Порівняти коміти

..
136 Коміти
Автор SHA1 Повідомлення Дата
Gabriel Gondim e8bc577e9d test(deployment): cover Remote Host Download to… wiring (#2763)
Covers Download to… in the Remote Host view: the Project view selection preselects the folder picker (its parent folder for a selected file), cancelling the picker downloads nothing, and the chosen folder receives the remote paths.

Co-authored-by: GGVidal <[email protected]>
2026-11-12 02:18:54 +08:00
rohitkumardandaтаrohitdanda b1348ee03a fix(lsp): open network-share files and upper-case drive letters from language servers (#2794)
On Windows `uri_to_path` rejected any `file:` URI host other than `localhost`, so network-share results (`file://server/share/a.ts`) from language servers were dropped from diagnostics and renames and opened as read-only virtual documents; such hosts now map to `\\server\share\...`, and `path_to_uri` writes UNC paths in the same RFC 8089 form. Lower-case or escaped drive letters (`file:///c%3A/x`) are upper-cased in `uri_to_path` and `to_ipc`, so the UI no longer treats `c:/x` and `C:/x` as different files. Non-Windows behaviour is unchanged.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:17:35 +08:00
rohitkumardandaтаrohitdanda e99d05d7a8 fix(vfs): retry saving on Windows while another process briefly holds the file (#2791)
The atomic save renames a temp file over the target, and Windows refuses that while another process (antivirus or indexer scan, sync client, a program reading its config) has the file open without delete sharing, so the save failed with "Access is denied" or a sharing violation. The rename is now retried for up to about a second on Windows errors 5, 32 and 33; other errors and other platforms fail at once.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:14:29 +08:00
rohitkumardandaтаrohitdanda 0e4cf1059a fix(workspace): keep repeated inserts at one position in workspace edits (#2793)
`WorkspaceEdit::deduplicate` dropped any edit already present for the file, including repeats inside one `TextDocumentEdit`, so a server edit inserting the same text twice at one position (two closing brackets, two blank lines) lost one insert. It now only drops edits an earlier alias entry for the same file already contributed and keeps every edit of one entry; the result stays idempotent.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:13:19 +08:00
rohitkumardandaтаrohitdanda b4bcd8cce7 fix(editor): keep a chosen file encoding when the file changes on disk (#2792)
Reloads after an external change (`reload_from_disk`) and after undoing a workspace edit (`reload_resource_buffers`) always re-ran encoding detection, so a file the user had reloaded or converted to a legacy encoding came back as mojibake and later saves used the guessed encoding. Documents now record that the encoding was chosen and both reload paths decode with it; files whose encoding was never chosen are still detected again.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:12:13 +08:00
rohitkumardandaтаrohitdanda cb5e7081fa fix(platform): keep delivering IDE events after one handler throws (#2790)
Broadcasts called their listeners in a bare loop, so one throwing handler kept the rest (including the catch-all "*" IDE event handlers) from seeing the event. The IDE event bus, local diagnostics, endpoint-change and navigation-modifier broadcasts now go through `notifyEach`, which logs a throwing listener and moves on. `onIdeEvent` wraps each subscription so unsubscribing one of two identical handlers keeps the other.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:09:24 +08:00
rohitkumardandaтаrohitdanda 034b2b9f65 fix(app): keep the window responsive while local history, shelves and run files load (#2789)
Tauri runs commands declared without `async` on the main thread, so Local History, shelf, change-list and run-configuration reads and writes, `.env` loading and the line diff of two arbitrary texts blocked input, painting and every other IPC call while they ran. They are now async and run on the blocking pool; IPC signatures are unchanged. `vcs_changelists_save` stays synchronous so its fire-and-forget saves land in order. A guard test keeps these commands async.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 02:06:22 +08:00
l0ng-ai 7ea3c59d05 fix(vcs): list local changes when the work tree has a directory git can't read (#2800)
gitoxide's status stops at the first IO error, so one unreadable directory failed the whole status and the Commit window showed no changes, where git warns and lists the rest. Status now asks git (porcelain v2) when gitoxide fails; a test checks the parser against gitoxide on a full status matrix.

Fixes #2769
2026-11-12 01:59:27 +08:00
l0ng-ai 6717b0948a fix(vcs): open a log commit's file diffs in one reusable preview tab (#2798)
Double-click, Enter or Show Diff on a file in the Git log's changes opened a new editor tab each time. It now opens the editor's preview tab, as the Commit window does, and while that tab shows one of the pane's diffs, selecting another file shows it there. Pinned tabs are left alone.

Fixes #2768
2026-11-12 01:54:10 +08:00
l0ng-ai 8302cd1d6f fix(debugger): clean up Delve's build and hide Run/Debug when their last tab closes (#2797)
Delve in DAP mode builds the debuggee as __debug_bin<digits> on every platform; the post-session sweep only accepted that form on Windows, so the binary stayed in the project on macOS and Linux. Closing the last session tab also left an empty Debug (or Run) tool window open; both now hide with their last tab.

Fixes #2766
2026-11-12 01:49:21 +08:00
l0ng-ai c05670f4fd test: add framework_files to the LanguageConfig literals in ide-dap and ide-lsp tests (#2796)
LanguageConfig gained framework_files in #2781; the test files that spell the struct out no longer compiled.
2026-11-12 01:42:20 +08:00
l0ng-ai 2f9658c367 fix(vcs): smooth-scroll again when moving between differences (#2795)
Next/Previous Difference also dispatched a scrollIntoView effect, which set scrollTop in the same measure as the smooth scroll and cut the animation off. The smooth scroll already centres the caret, so the effect is removed.

Fixes #2764
2026-11-12 01:38:03 +08:00
l0ng-ai 40aa7e4899 fix(java): resolve navigation and usages against the text just typed (#2759)
Right after typing in a Java file, Show Usages answered for the wrong symbol and Go to Declaration could do nothing. Usages, Find Usages and hierarchy now wait for the editor's edits to sync before sending the caret position, and the java-usages commands (usages, implementations, hierarchy, configuration) wait for jdtls to reconcile queued document changes before resolving a position (bundle 1.13.0, rebuilt reproducibly from source). File-system events (e.g. the build writing target/classes) now only drop cached navigation answers instead of discarding an in-flight Go to Declaration.
2026-11-12 01:33:25 +08:00
arookieofcтаClaude Code fba1f02375 feat(hotkeys): warn when the OS grabbed an IDE shortcut (#1876) (#2683)
* feat(hotkeys): warn when the OS grabbed an IDE shortcut (#1876)

Probe single-stroke Alt bindings at startup with RegisterHotKey and
notify once per session when another app owns one, with an action that
opens Keymap for reassignment.

Co-Authored-By: Claude Code <[email protected]>

* fix(hotkeys): satisfy clippy::collapsible_if on the F-key parse

Co-Authored-By: Claude Code <[email protected]>

---------

Co-authored-by: Claude Code <[email protected]>
2026-11-12 01:33:04 +08:00
l0ng-ai 053ca43368 fix(usages): say at once that Show Usages is searching (#2760)
Show Usages showed nothing until its popup opened (300 ms after the first usages, or when the whole answer came), so a slower search looked like the shortcut had done nothing. It now shows "Searching for usages…" at the caret until the popup or the answer replaces it, and pending hints (such as Go to Implementation's) appear after 50 ms instead of 250 ms.
2026-11-12 01:30:18 +08:00
Vadim LazukтаInf1mum 8fedc2efba fix(vcs): stop filtered logs from fanning one dashed lane per merge path (#2788)
A filtered log connects each visible commit to the nearest matching ancestor along every merge path, and in merge-heavy histories one row could collect dozens of dashed connections, each getting its own lane, so the graph column stretched the log horizontally. pruneFilteredEdges now drops a dashed connection whose target the commit's other connections already reach (shallowest first), so reachability is unchanged with one lane instead of one per hidden merge path. Real parents, collapsed fragments and unfiltered histories are untouched; parent/child navigation follows the same pruned graph.

Fixes #2670

Co-authored-by: Inf1mum <[email protected]>
2026-11-12 01:28:32 +08:00
zhangdagouтаPLZY 2f2fa79cbe fix(build-tools): select a clicked completion in Execute Maven Goal / Gradle Task instead of appending it (#2785)
Clicking a completion appended it to the command line and re-filtered the list, so repeated clicks kept accumulating goals. A click now selects the row and shows its command line in the input; another click switches to that row; the list filters only on typing; Enter or a double-click runs it. Goals and tasks already on the command line are no longer offered again.

Co-authored-by: PLZY <+[email protected]>
2026-11-12 01:26:36 +08:00
Deng Liuтаdengliu 65d49619dd fix(terminal): invalidate selection highlight cache on scrollback trim (#2784)
The selection-occurrence cache keyed matches by absolute buffer row and relied on onRender to invalidate edited lines, so a full scrollback trimming from row 0, CSI 3 J or clear() shifted cached margin and scrollback rows without invalidation, and a selection trimmed off row 0 left its boxes behind. The cache now anchors a marker at row 0 while populated (any top trim disposes it and clears the cache), invalidates active-screen rows across baseY advances on write, and clears the overlay when a trim removes the selection.

Co-authored-by: dengliu <[email protected]>
2026-11-12 01:24:20 +08:00
Andrea Basileтаandreabasile 7f28161869 feat(languages): Django templates (#2781)
Adds a Django Template language: .html/.txt files in a templates folder of a project with manage.py (new [[framework-files]] detection), a Lezer grammar for tags, variables, filters and comments with HTML mounted between them, block folding/indent/matching, {% %} / {{ }} / {# #} pairing, end-tag insertion and completion, per-line {# #} comments, HTML language server and Emmet in the HTML parts, a banner offering the language for HTML files with Django tags elsewhere, a fixture and docs. The built-in HTML syntax check skips Django templates, whose tags the HTML grammar reports as errors.

Refs #2581

Co-authored-by: andreabasile <[email protected]>
2026-11-12 01:22:23 +08:00
l0ng-ai fc1493e1a4 feat(go): report upgrades, vulnerabilities and deprecated modules when go.mod opens (#2787)
gopls reports available upgrades only after its "Check for upgrades" code lens runs, never reports deprecated modules, and checks vulnerabilities only on demand. Opening a go.mod now runs the upgrade check once per file and session, gopls' vulncheck setting is on for imports (Go, go.mod, go.work), and `go list -m -u` marks deprecated requirements on open and save. go.mod highlighting also covers godebug, tool and ignore.
2026-11-12 01:14:56 +08:00
l0ng-ai efda6834ad feat(search): list directories in Go to File and Search Everywhere (#2783)
Go to File and the Files section of Search Everywhere listed only files, so a
folder could not be found by name. Directories from the index's folder search
now rank with the files (a file first on equal scores), with a folder icon, and
choosing one selects it in the Project view. A file type filter or a
File.kt:42 query still lists files only.
2026-11-12 00:54:21 +08:00
zhangdagouтаPLZY e92e867019 feat(vcs): show an added or deleted file's diff as one full-width pane (#2779)
A diff of an added or deleted file showed an empty half next to the existing side. It now shows the side that exists alone, full width, with its lines tinted and the viewer-mode buttons disabled, as IDEA's one-side viewer does. The saved viewer mode is unchanged.

Co-authored-by: PLZY <[email protected]>
2026-11-12 00:47:35 +08:00
zhangdagouтаPLZY 8d8c7a481b fix(vcs): align the diff titles and find bar with the panes, and title revisions by commit and path (#2778)
After the side-by-side divider was dragged, the right half of the title row and of the find bar stayed at 50% and no longer lined up with the right pane. Both now follow the right pane's actual start. A revision side's title shows its short commit id (HEAD resolved) and the file's path in the repository.

Co-authored-by: PLZY <[email protected]>
2026-11-12 00:47:25 +08:00
zhangdagouтаPLZY 0b3ca2fc9a feat(vcs): put the left pane's line numbers next to the divider in side-by-side diffs (#2776)
The left editor of a side-by-side diff is mirrored: its line numbers sit on its right, next to the divider, as in IDEA. CodeMirror's lineNumbers() has no side option, so the viewer adds an equivalent line-number gutter on the after side for that pane.

Co-authored-by: PLZY <[email protected]>
2026-11-12 00:47:15 +08:00
Ernstjan FreriksтаEjaYF 2414773d61 fix(agents): attach files dropped from outside the project (#2767)
Dropping a file from the OS onto the agent message box did nothing when it was outside the project (a screen recording on the Desktop, say): the drop handler only turned project paths into @ references and skipped the rest silently. Such files now show as attachment chips next to the image thumbnails and reach the agent as resource links. A dropped image over the 5 MB inline limit, or one that cannot be read, is attached by path instead of inlined. The box highlights while an OS drag is over it. Remote windows still skip paths outside the project.

Co-authored-by: EjaYF <[email protected]>
2026-11-12 00:45:19 +08:00
rohitkumardandaтаrohitdanda e4181c36ea fix(agents): roll a local task back to where it started when HEAD moved (#2774)
A task working in the project's own checkout compared and rolled back its changes against the current HEAD. Once HEAD moved back or to another branch during the task (git reset --soft, an undone commit, a checkout), the user's own work showed up as the task's changes and Rollback overwrote it; in a repository without commits, Changes and Rollback failed.

Local tasks now use the commit they started from: files that are as they were at the start are not listed, and Rollback and the diff base use the start commit, or HEAD when the user committed on top of it so those commits are kept. In a repository without commits new files are listed and rollback deletes them. Worktree tasks are unchanged.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 00:42:35 +08:00
usong 5063f6756e fix(editor): correct library source encoding and Reload with Encoding (#2775)
Library sources in a non-UTF-8 charset (e.g. GBK) opened garbled, and Reload with Encoding failed on them with a read-only error: library and virtual documents bypassed the editor's encoding pipeline (archive entries read as strict UTF-8, jdt: documents labelled utf-8, set_encoding rejecting every read-only virtual document).

jar: and jdt: sources are now decoded through the same VFS pipeline as files on disk (detection, BOM, UTF-16, explicit reload) and report the charset used. Reload re-decodes virtual documents while keeping them read-only: jar: entries are re-read; for jdt: sources the jdtls source attachment encoding is updated (waiting for Maven's background refresh, retrying once) and the attached entry re-read. Reload is disabled for decompiled and server-generated content and for read-only containers such as the JDK.

Co-authored-by: usongon <[email protected]>
2026-11-12 00:41:22 +08:00
rohitkumardandaтаrohitdanda 1a2f5308a0 fix(editor): keep syncing edits after a document snapshot fails (#2777)
When an edit was rejected as out of sync and the rebase snapshot then failed, the in-flight edit was never cleared, so flush() returned early forever: save, format and refactorings waited and the backend kept the old text. A failed rebase snapshot now resends the edit (bounded by MAX_REBASES, then the existing hard resync), and idle waiters are no longer released while an edit is pending.

When hardResync's own snapshot failed, the in-flight and buffered changes were dropped although the editor still showed them, so later edits were expressed against text the backend never had. The difference between the confirmed and local text is now kept as the buffer and sent with the next flush.

Co-authored-by: rohitdanda <[email protected]>
2026-11-12 00:34:03 +08:00
l0ng-ai cd04593d04 refactor(settings): open Project Structure only as its own dialog (#2744)
Settings no longer lists a Project Structure page; File › Project Structure…
(⌘; / Ctrl+Alt+Shift+S) opens the standalone dialog, which already had the
same sections. Arrow keys in the dialog's sidebar follow its navigation order.
2026-11-12 00:31:51 +08:00
l0ng-ai 38241e7fae feat(deployment): download Remote Host items to a chosen folder or by dragging (#2780)
Remote Host › Download to… asks for a local folder (the Project view's selected
folder preselected) and copies only the selected files and folders, each under
its own name, instead of recreating the server path below the project root.
Dragging items from Remote Host onto a folder in the Project view does the same.
Download to Project keeps using the mappings.
2026-11-12 00:29:34 +08:00
l0ng-ai 2e6d6a05b9 fix(plugins): accept empty navigation anchor ranges as positions (#2773)
A document indexer that anchors a whole-file definition at the file's start
(an empty range at 0:0) had the whole document rejected with "invalid
navigation anchors". Empty ranges inside the document are now accepted:
Go to Declaration lands on them, while Find Usages and Rename skip them so
a rename never inserts the new name at that position. The error for a
document that is still rejected names the plugin and the file.
2026-11-12 00:11:15 +08:00
ntbang0901 c73d858895 feat(debugger): debug Kotlin with kotlin-lsp's debugger (#2651)
kotlin-debug-adapter (fwcd) is unmaintained: its evaluate only resolves plain local names and it reports breakpoints verified without checking. Kotlin debugging now uses the debugger inside JetBrains kotlin-lsp once its project import has finished, attached to the JVM the IDE starts through the existing JDWP link. kotlin-debug-adapter stays the fallback, with a notice saying why (not installed, still importing, import failed, cannot start).

kotlin-lsp's DAP quirks (step stop reasons, thread ids from earlier stops, frame names, file: URIs) are handled in one place, and its invalidated events reload the Variables pane and watches.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-12 00:05:41 +08:00
l0ng-ai 6e5486586c fix(i18n): restore the comma missing after the local data directory strings in zh.json (#2771) 2026-11-12 00:03:13 +08:00
arookieofc 93642b3741 feat(settings): custom local data directory with one-click migration (#1722) (#2677) 2026-11-11 23:53:24 +08:00
Gabriel Gondim b06f2e27ea feat(settings): Color Scheme › Console Colors for the terminal's ANSI colors (#2751)
Settings › Editor › Color Scheme gets a Console Colors page, as in IntelliJ IDEA: the console background and the 16 ANSI colors, used by the terminal and consoles. A light IDE theme no longer forces a light terminal. IDEA's CONSOLE_* keys from .icls schemes and IntelliJ theme imports set the same colors.

Closes #2698

Co-authored-by: GGVidal <[email protected]>
2026-11-11 23:40:04 +08:00
Muuтаmuutot 86f69b7fca feat(settings): complete the Python color page as PyCharm's full model (#2761)
The Python page of Settings › Editor › Color Scheme now has PyCharm's 38 attributes in its order and groups, all scoped to Python: an override lands in the editor block for Python files instead of recolouring every language, and unset values fall back to the shared scheme value. The seven attributes that had no default value (parentheses, brackets, braces, nested function, method call, type annotation, f-string format spec) now resolve, and .icls PY.* keys map to their Python attributes. Attribute ids and saved overrides are unchanged.

Co-authored-by: muutot <[email protected]>
2026-11-11 23:38:25 +08:00
Vadim LazukтаInf1mum fefa135f5a feat(docker): pick an SSH configuration for TCP/SSH Docker connections (#2737)
The TCP / SSH host kind of a Docker connection now offers the IDE's SSH Configurations (with … to open Settings › Tools › SSH Configurations), as IntelliJ IDEA does. Picking a configuration with OpenSSH config authentication fills the Engine API URL with ssh://[user@]host[:port], keeping the alias so the system ssh docker runs applies the entry's key, user and jump hosts; an existing ssh:// URL shows its configuration again. Password and key-pair configurations are listed but explained as unusable for now, since docker over SSH goes through the system ssh.

Refs #939

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 23:36:44 +08:00
Vadim LazukтаInf1mum 8fe2d35d62 feat(editor): pair <> where a generic type list starts (#2738)
Under Smart Keys › Insert paired brackets, < now pairs into <> where the syntax tree says a type argument / parameter list starts, as in IntelliJ IDEA: after a type name or generic declaration name in Java, TypeScript, Kotlin and C#, and Rust types, fn names and turbofish ::<. Comparisons (a < b, i <= n, x << 2), tags and literals never pair; > steps over the inserted half and Tab leaves it. Languages without a rule don't pair.

Closes #2700

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 23:34:24 +08:00
rohitkumardandaтаrohitdanda 21046c3047 fix(platform): open only http(s) and mailto URLs externally (#2755)
openExternalUrl handed any string to the system opener, and plugins and rendered content (Markdown, notebook output, editor links) reach it with arbitrary input, so file: or app-launching schemes such as ms-msdt: and search-ms: could be passed to the OS. It now opens only http:, https: and mailto: URLs and ignores anything else, including malformed or whitespace-padded strings.

Co-authored-by: rohitdanda <[email protected]>
2026-11-11 23:32:18 +08:00
l0ng-ai 017561a20b fix(libraries): list the JDK for an .idea project only when it is a Java one (#2765)
A project without a build manifest got a JDK under External Libraries as soon
as it had an .idea folder, which GoLand, PyCharm and WebStorm write too: a Go
project showed < jdk-25 >. The folder now counts only when misc.xml names a
Java SDK or language level, or a module file is a JAVA_MODULE.
2026-11-11 23:31:36 +08:00
l0ng-ai d3a68b2754 fix(lsp): ask again when a server withdraws a rename while analysing (#2762)
A server still analysing an edit may answer textDocument/rename with
ContentModified (the Dart analysis server does when the document changed
under it). That maps to a cancellation, so the rename the user confirmed was
dropped without a word. The rename is now asked again a few times, 300 ms
apart, and if it keeps being withdrawn the error says so.
2026-11-11 23:28:22 +08:00
l0ng-ai e98a560523 fix(run): open .NET Project configurations as the form with a project field (#2757)
The .NET Project type had no way back from a command line to its fields, and a
new configuration starts without params, so the editor fell back to the plain
Command/Arguments form: there was no .NET project field, and Debug failed with
"Select a .NET project". dotnet run command lines the form can express now map
back to its fields, for new configurations and ones typed as a command.

Fixes #2733
2026-11-11 23:21:53 +08:00
l0ng-ai 6983e8ec2f feat(color-scheme): add a C# page to Settings › Editor › Color Scheme (#2756)
C# gets its own page like Go, Java, Kotlin and PHP: every Language Defaults
attribute can be overridden for C# editors only, with a C# preview. Language
sub-page ids are now slugged (C# → csharp): the '#' in editor.colorScheme.c#
made the page unreachable from the settings tree.
2026-11-11 23:17:46 +08:00
l0ng-ai dc355602a9 feat(http-client): go to a {{variable}}'s definition (#2754)
Navigate › Declaration (⌘-click, ⌘B) on {{name}} in a .http file now lists
its definitions in lookup order: the file's @name = … line, then the key in
each environment (and $shared) of the nearest http-client.private.env.json
and http-client.env.json. Dynamic variables ($uuid…) have none.
2026-11-11 23:13:42 +08:00
l0ng-ai 924fa9259d fix(plugins): refuse a command while its plugin waits for an answer in another window (#2753)
A plugin runs one call at a time, and a prompt waits for its answer without a
limit. A command run from another window while the plugin's prompt was open
queued silently behind it and ran only after the prompt was answered. It is
now refused with a message naming the reason, and the window that holds the
prompt is brought to the front.

Fixes #2673
2026-11-11 23:04:11 +08:00
l0ng-ai 24deeb0dbb fix(i18n): drop repeated zh catalog keys and test that every key appears once (#2752)
Merges had added eleven zh.json keys a second time (Docker, Shell, the language
names, Add Service ({kind}), Others, Servers). JSON.parse silently keeps the last
copy, so nothing failed. The catalogs test now reads each catalog file and fails
on a key listed twice.
2026-11-11 22:58:21 +08:00
renyun2таrenyun2 0ce7f6e430 feat(agents): expose Cursor effort and Fast as separate options (#2705)
Cursor's ACP agent only offers thinking effort and a separate Fast switch when the client advertises parameterizedModelPicker. OpenIDEA now advertises it, and when an ACP agent is picked in the Home composer before any session has reported its options, a short discovery session (session/new without MCP servers, killed afterwards or on timeout, once per agent per run) records its config options and effort levels. Effort and Fast show inside the model control, a new Cursor task defaults Fast to off, and config changes show immediately and roll back if the agent rejects them.

Co-authored-by: renyun2 <[email protected]>
2026-11-11 22:54:44 +08:00
misswellтаmisswell 84fd0d0a60 fix(build): skip JaCoCo instrumentation in IDE compile builds (#2686)
Projects that bind jacoco-maven-plugin's offline instrument goal to process-classes had target/classes rewritten by the IDE's own Build/Rebuild and launch builds, so a later Debug/Run without the JaCoCo agent failed with NoClassDefFoundError. The Maven steps of Build and Rebuild now pass -Djacoco.skip=true; artifact (package) builds and coverage runs are unchanged, and Automatic launches stay incremental.

Fixes #2676

Co-authored-by: misswell <[email protected]>
2026-11-11 22:50:57 +08:00
Kamal ElFadlтаelfadl 96c99e0396 feat(agents): add a command execution policy for ACP agents (#1165)
Settings › Agents › Command Execution Policy (global or per project) lists commands an agent may run without asking and commands it is never allowed to run. A permission request for a shell command is answered from the policy: a blocked pattern (glob, checked against the whole command and every part of a chain) rejects it; it is approved only when every part of a &&/||/;/|/& chain matches an allowed entry exactly or up to a word boundary and none uses command substitution, redirection, subshells, here-docs or leading variable assignments. Everything else prompts as before. The allowed list starts empty; the blocked list ships common destructive commands.

Closes #1152

Co-authored-by: elfadl <[email protected]>
2026-11-11 22:50:17 +08:00
arookieofcтаa-rookie-of-C-language fd4838fc36 feat(agents): create, remove and import custom ACP agents (#2690)
Settings › Agents › Coding Agents can add a custom ACP agent (name, id, command, arguments, environment) without writing its agent file by hand, remove one, and import the agent_servers entries of a JetBrains acp.json. Built-in and ACP Registry ids are refused, ids are validated as file stems on save and on remove, and colliding or incomplete acp.json entries are skipped with a reason.

Closes #228

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:45:36 +08:00
arookieofcтаa-rookie-of-C-language 4873905884 feat(agents): accept worktree task changes into the local checkout (#2704)
The Changes pane of a worktree task gets an Accept action next to Rollback: after confirmation it copies the task's changed files (all, or the selected ones) from the task worktree into the local checkout, uncommitted. A file the checkout has changed itself is left alone and listed in the notification instead of being overwritten; running and local tasks are refused.

Closes #1777

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:40:49 +08:00
arookieofcтаa-rookie-of-C-language 6b37788971 test(lsp): go.mod/go.work reach gopls with the language id it understands (#2719)
Pins the go.mod serving contract: opening go.mod through LspManager with the bundled registry sends didOpen to gopls with languageId go.mod, scoped to the module root, and gopls's pushed diagnostics reach Problems. The gomod/gowork gopls entries now use the same initialization options as the go entry, so startup options no longer depend on which file launches the server first.

Refs #2248

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:33:33 +08:00
arookieofcтаa-rookie-of-C-language 44bf648bee fix(tomcat): rebuild the WAR when web resources or dependency modules change (#2722)
The pre-launch up-to-date check only compared the WAR with the WAR module's own src and build files, so a save in a dependency module (mapper XML in a DAO module), in web content outside src (custom warSourceDirectory / webResources), or in any module when the config had no cwd (the check fell back to the project root) left the old WAR deployed. The check now resolves the WAR module from the selected artifact when cwd is missing, includes its web resource directories, and includes the sources and build files of every module in the build model.

Fixes #2685

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-11 22:32:36 +08:00
Vadim LazukтаInf1mum a0c3249368 fix(linux): keep the first-map hint under Mutter's auto-maximize share (#2520)
Wayland has no primary monitor, so the first-map size hint fell back to scale 1 with no work-area limit and requested the saved physical size as logical. On a HiDPI output the first mapped area exceeded Mutter's 80% share and the window opened maximized before the saved geometry was restored. The hint now converts with the largest output scale and shrinks proportionally to stay within the tightest output's share; X11 keeps its primary-monitor path.

Fixes #2400

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 22:29:05 +08:00
l0ng-ai 757412cb73 fix(security): reach content roots in plugin quick fixes and keep diagrams local
- A plugin quick fix may change or open files in the project's content
  roots (a project window group's members, extra Project Structure
  folders), still never git's own files; before, it was disabled for
  every file outside the primary root.
- `git init` (New Project › Create Git repository) uses the configured
  git executable again.
- Mermaid output is cleaned before it is shown: remote image sources,
  external `url()`, `@import` and address strings in its CSS are
  removed, so `style`/`classDef` values and HTML labels fetch nothing.
- The reply image e2e test loads a web image only after its button is
  clicked; the post-checkout hook test runs in a trusted scope.
- Capability docs list `project.write` as high risk.
2026-11-11 22:18:48 +08:00
Fabio Nunes Cerqueira a497178b3e fix(security): close the findings of GHSA-442p-gxfg-qxv9
- Restricted Mode git: reads run without a signature verifier, hooks
  (including config-defined ones), an external diff, filters or a lazy
  fetch, also inside submodules; every repository and every git starts
  restricted until the project is trusted.
- project.write and plugin quick fixes cannot reach git's own files under
  any spelling or through links; project.write is a high-risk grant.
- Built-in web server: each served root has its own port and token; a
  file outside the project is served with its folder minus dot names, or
  alone when that folder is the home folder or above it; no fixed port.
- Images, references, Markdown links and language-server links reach no
  network share other than the one already open.
- Marketplace: entries whose id or version no manifest may carry are
  dropped; an index read over the network names nothing on this machine;
  network reads are bounded in size and time.
- Agent replies load remote images on a click; the Markdown preview and
  Mermaid configuration no longer fetch from other hosts.
2026-11-11 22:18:48 +08:00
ntbang0901 8b179992e8 feat(profiler): profile allocations and live objects of JVM runs (#2650)
The Profiler gets two more modes for JVM runs through async-profiler: allocations (where the program allocates, in bytes) and live objects (what is still alive at the end). A live-object profile collects garbage first: run executors get a beforeStop hook, bounded to 12 s, that runs a GC before the stop signal; a program that ends by itself says that unreachable objects may show as live.

Part 3 of #2217.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 22:13:38 +08:00
wangjunboit 85541b3663 feat(vcs): show a diff in a separate window (#2231)
Changes trees get Show Diff in New Window: the diff opens in its own native window (labelled diff-*, with the same capability as comparison windows), with the full diff toolbar, editable working-copy side and a save prompt for unsaved edits on close. Showing the same diff again reuses its window.

Co-authored-by: wangjunboit <[email protected]>
2026-11-11 22:07:04 +08:00
l0ng-ai 0418a2ce01 fix(java): explain a Maven build plugin that doesn't resolve when m2e only logs it (#2745)
When a build plugin such as maven-resources-plugin can't be resolved
(e.g. the default version is missing from a company mirror), m2e fails
the import's last step and every build but puts no marker on pom.xml,
so Problems stays empty and the "Maven project could not be configured"
notification never appeared. The notification now also fires for these
logged failures, once per plugin whichever way jdtls reports it.
2026-11-11 21:39:00 +08:00
l0ng-ai c8639b38ba fix(libraries): write only SDKs in angle brackets under External Libraries (#2743)
Every External Libraries group was shown as < name >, so a Maven or Gradle project listed hundreds of bracketed dependencies. As in IntelliJ IDEA, only SDKs (JDK, Python, Ruby, Go, Rust) keep the brackets; dependency groups such as Maven: g:a:v are shown as they are.

Refs #1977
2026-11-11 21:34:48 +08:00
Vadim LazukтаInf1mum f04658612f fix(plugins): plugins reach a project window group's member files (#2739)
In a project window group plugins only saw the container: projects() and project-of() reported the container root, read/write/find refused member paths, Find in Files dropped member matches and the document indexer skipped member documents. The plugin host now treats the members like the editor and language servers do: projects() lists them, project-of() resolves to the deepest member, the sandbox path checks accept paths inside any member (files outside every root stay refused), and find-files, symbol search and the indexer cover member files. No WIT change.

Fixes #2506

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:31:23 +08:00
Qingтаqtvbwfn d5cb379cae feat(services): show only services the user added (#2740)
The Services tool window listed services nobody added: an injected Docker connection, Spring Boot by default and every configuration that had ever run. Now only added types, connections and registries are listed, removal takes effect at once, an untouched injected Docker connection is dropped from older docker.json files, and the detail pane has a single scrollbar with console controls only (Run/Debug/Stop stay on the window toolbar).

Co-authored-by: qtvbwfn <[email protected]>
2026-11-11 21:28:41 +08:00
Muuтаmuutot e3a818b607 feat(color-scheme): group Python color attributes as PyCharm does (#2735)
The Python page of Settings › Editor › Color Scheme listed its attributes flat. They now follow PyCharm's tree (String › f-string, Docstring, Braces and Operators, Functions, Parameters, Special names…), in its order; only display names change, so .icls files and existing overrides keep working.

Co-authored-by: muutot <[email protected]>
2026-11-11 21:26:31 +08:00
Vadim LazukтаInf1mum 0b1872937a fix(lsp): hint when Vue 2 Options API template navigation lacks strictFunctionTypes (#2728)
Vue 2.7's Options API types only infer methods and computed into this under strictFunctionTypes; without it template go-to-definition and hover return nothing. When vue-language-server serves a Vue 2 project whose jsconfig/tsconfig lacks the option (and doesn't extend a base), the IDE now shows a warning with the fix.

Fixes #1163

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:25:01 +08:00
Vadim LazukтаInf1mum f11cd361a7 fix(color-scheme): import .xml scheme files and tolerate real-world IDEA variants (#2731)
The import chooser only offered .icls, so schemes distributed as .xml (Material Theme UI) could not be picked. It now offers .xml too, and the parser accepts a BOM, an encoding declaration, a DOCTYPE, a scheme wrapped in a settings export and a scheme without a name (named after its file); a rejected file shows the reason.

Fixes #2714

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:23:38 +08:00
Vadim LazukтаInf1mum 7034fe1640 fix(debugger): offer to reinstall Delve when it is too old for the Go toolchain (#2730)
A Delve older than the Go toolchain refuses the launch with "Version of Delve is too old for Go version …". The debugger now recognises it and offers Reinstall and Debug, which runs the managed go install …/dlv@latest and starts the session again.

Fixes #2721

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:23:03 +08:00
Vadim LazukтаInf1mum fc264d3f5c fix(ui): stop double-clicking tree rows from selecting label text (#2726)
The second press of a double-click started the browser's word selection, which in the virtualized tree spilled onto the rows an expanding folder had just inserted. Tree cancels the default of repeated presses; the first press still selects and dblclick still activates.

Fixes #2713

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:22:02 +08:00
Vadim LazukтаInf1mum 8e90341b3a feat(build-tools): a Skip Tests toggle for the Gradle tool window (#2734)
The Gradle tool window gets an optional Skip Tests toggle (off by default, per project): task runs and saved configurations started from the tree get -x test, unless the command line already excludes test or the build has no test task (where -x test would fail the run). A run that skips tests is not marked as a test run.

Fixes #2688

Co-authored-by: Inf1mum <[email protected]>
2026-11-11 21:20:58 +08:00
l0ng-ai 3644956755 feat(terminal): activate the project's conda environment in new terminals (#2742)
Activate virtualenv only handled venvs, so a conda project interpreter left new terminals in (base). New terminals now run conda activate <prefix> through the installation's shell hook (cmd, PowerShell, bash, zsh, POSIX sh, fish), finding the installation from the env itself, the parent of envs/, CONDA_EXE or PATH.

Fixes #2697
2026-11-11 21:18:24 +08:00
l0ng-ai 90e7e3162e feat(mcp): run the IDE's tools without confirmation in brave mode (#2741)
Settings › Agents › IDE MCP Server gets Run tools without confirmation (brave mode): builds, run configurations and the other tools that change a project start without asking, so an unattended agent isn't left waiting on a confirmation. Read from the IDE's own settings only; the project layer already refuses mcpServer.*.

Fixes #2660
2026-11-11 21:17:24 +08:00
Yann 833ae4c3c4 feat(editor): add and remove UTF-8 BOM (#2717)
The encoding status menu of a UTF-8 file gets Add UTF-8 BOM / Remove UTF-8 BOM, and the status bar shows "UTF-8 BOM" for a file that has one. The change marks the document modified and is written on save (at once with auto-save).

Co-authored-by: YannABC <[email protected]>
2026-11-11 21:00:48 +08:00
Elmir Abkerimov 584baa3f71 feat(editor): code vision position per hint and a "no usages" switch (#2716)
Settings › Editor › Inlay Hints › Code Vision gets a Position for Usages and for Code author — Top (a row above the declaration, the default) or Right (after the end of the declaration's first line) — placed per hint, and Show "no usages" (on by default) to leave the usages hint off declarations nothing uses. The options sit under the hint they belong to.

Co-authored-by: w4terlaw <[email protected]>
2026-11-11 20:58:46 +08:00
Elmir Abkerimov 98010e6513 fix(editor): start every selected row at the text's left edge (#2724)
CodeMirror starts selection rows that run through a line start at the left padding rounded down to whole pixels, and a row that starts at a character at the real padding; with the 2.5px inset the two were half a pixel apart, a visible step on Retina. Rows that start at the real edge now move to the rounded one, and the sliver drawn on a line the selection ends at the start of is dropped after the rows are grown.

Co-authored-by: w4terlaw <[email protected]>
2026-11-11 20:56:38 +08:00
Muuтаmuutot a3640f655f fix(perf): name language server memory rows by their workspace root (#2702)
The memory indicator labelled every language server row with the project folder, so with additional content roots the servers of each root showed as identical rows. Each row now names the server's own workspace root.

Co-authored-by: muutot <[email protected]>
2026-11-11 20:56:07 +08:00
Shohruh Ismoilovтаshohruh-genius 8aaa7daf83 fix(project): keep workspace project icons stable on removal (#2699)
Removing a member from a workspace reset every detected project type, so the remaining members briefly showed folder icons and fell back to name order. The cached types of members still in the same workspace are kept while the rescan runs.

Co-authored-by: shohruh-genius <[email protected]>
2026-11-11 20:54:16 +08:00
Bin 6a9a970df4 fix(editor): use JSX comments for expression children (#2703)
Ctrl+/ on a JSX expression child such as {label} read the JavaScript comment tokens at the opening brace; it now wraps the line in the surrounding JSX comment pair, {/* … */}, and takes it back out.

Co-authored-by: ffgenius <[email protected]>
2026-11-11 20:53:53 +08:00
l0ng-ai a309b7e6fa fix(java): import Gradle builds on the JVM the IDE's Gradle runs use (#2736)
With Gradle JVM at Project JDK the IDE's Gradle runs use the project JDK (or an installed JDK the Gradle version supports), but the jdtls import ran Gradle on jdtls's own JDK 21, so Gradle < 8.5 failed to import and no dependency resolved. The import now takes JAVA_HOME and GRADLE_USER_HOME from gradle_build_env for Gradle builds.

Refs #2681
2026-11-11 20:52:49 +08:00
Zhu Minhaoтаzmh69695328 6971936f78 feat(java): run jdtls's Gradle import with the build settings (#2701)
jdtls's Gradle import ran on its own defaults (project wrapper only, ~/.gradle, the server's JDK), so a project without gradlew was never imported and one whose caches live in a configured Gradle user home resolved no dependencies. Settings › Build Tools › Gradle now reach java.import.gradle.*: home with the wrapper disabled for a local distribution, plus user.home and java.home, resolved like the IDE's own Gradle runs and merged into the existing settings.

Co-authored-by: zmh69695328 <[email protected]>
2026-11-11 20:49:28 +08:00
l0ng-ai 9d781a2f83 fix(agents): start and delete tasks without waiting on slow git work (#2732)
Starting a task in the local checkout recorded the already-changed files through the full change listing (diff, numstat, a read of every untracked file, quadratic lookups); the snapshot now uses the status alone and the listing uses hashed lookups. Deleting a task now drops it from the list before its worktree folder is deleted.

Refs #2667, #2666
2026-11-11 20:46:13 +08:00
l0ng-ai 3260e5e224 fix(vcs): open the source file on F4 in the changes trees (#2729)
F4 in the Commit tool window opened the selected file, then the tree handled the same key as Enter and opened the diff preview, which took the focus. Tree takes an onJumpToSource handler for F4 / Cmd+Down; the commit and Git Log changed-files trees use it to open the file itself.

Fixes #2696
2026-11-11 20:38:27 +08:00
l0ng-ai ed575a00d9 docs: show sponsor QR codes at the same size with labels (#2725) 2026-11-11 20:33:59 +08:00
l0ng-ai 2b41e61ce9 docs: add Alipay and WeChat Pay sponsor QR codes (#2723) 2026-11-11 20:29:16 +08:00
l0ng-ai 908740b3f2 feat(plugins): a marketplace that helps pick plugins (#2718)
* feat(plugins): a marketplace that helps pick plugins

Rows show each plugin's description, publisher, downloads and an inline
Install / Update button; plugins without an icon get a tinted initial.
While browsing, the list opens with suggestions for the open project and
the maintainers' picks, then all community plugins by downloads; the
official plugins come last as examples. The first plugin is selected so
the details are never empty, and they show release notes, the homepage
and why a plugin is suggested.

The index gains optional homepage, notes, downloads, featured and suggest
fields; openidea-plugin index fills homepage and notes from the manifest.

* fix(plugins): plugin icons fill their tile like the initials

* feat(plugins): drop the Preview badge from the plugins page

* feat(plugins): open the plugins page on Available
2026-11-11 20:13:28 +08:00
l0ng-ai e702e4da7b chore: v0.1.42 2026-11-11 19:31:09 +08:00
arookieofc 91023de207 chore(ci): route heap dump dialogs through the platform wrapper; clippy 1.99 and Windows test fixes (#2711)
Heap dump open/save use the platform dialog wrapper (GTK fallback on Linux). ide-heap allows clippy 1.99's large_enum_variant on the hot bit table. dialog.test compares paths with forward slashes on Windows.
2026-11-11 19:30:15 +08:00
l0ng-ai 02ee2880fe fix: translate New Project popup and error text; green CI on main (#2715)
Translate the project popup actions and the non-empty target error; translate the JDK field label. Fix clippy in ide-jdwp, formatting, RUSTSEC-2026-0486 (rustix 1.1.6) and agent E2E tests that expected the removed floating Quote button.
2026-11-11 19:25:09 +08:00
l0ng-ai 5a316e81d6 fix: Compile File sees its module's classes; list external agent sessions per project off the main thread (#2707)
Compile File passed javac only the dependencies and upstream modules, so any file using
another class of its own module failed with "cannot find symbol". The module's own
output folders now come first on the classpath, and its source folders (generated ones
included; main ones for a test) are the -sourcepath with -implicit:none, so classes that
were never built resolve from source while only the file itself is written.

Listing sessions started outside OpenIDEA parsed every Claude Code transcript of every
project on a synchronous command, which runs on the main thread and froze the window
(1.2 GB of transcripts here). It now reads only the transcript folders of the open
project, keeps sessions whose working directory is in the project, reads Codex rollouts
line by line, and runs on the blocking pool; importing does the same.
2026-11-11 18:22:28 +08:00
l0ng-ai 7c2ca4d061 fix: build Maven again when outputs were deleted; main menu in a separate toolbar (#2706)
Build Project / Build Module skipped Maven when no input changed, even after the
modules' target/classes were deleted outside the IDE (mvn clean in a terminal).
The up-to-date record now also lists the output folders that existed after the
build, and a build is skipped only while they all still exist.

The "Always show the main menu" toggle duplicated Main menu › Merged with main
toolbar, showed on macOS where it does nothing, and kept the titles in a title
bar too narrow for them. It is replaced by a third Main menu choice, Separate
toolbar: the menu titles get their own row below the title bar and never
collapse into the hamburger button.
2026-11-11 18:09:01 +08:00
ntbang0901 8d09479a65 feat(debugger): heap snapshots of a debugged JVM (#2649)
Capture a heap dump of the debugged JVM over the attach listener (GC.heap_dump, after checking free disk space and the JDK's rules for suspended VMs) or open an .hprof file, and browse it in an editor tab: classes with shallow and retained sizes, biggest objects, dominators, the path from an object to a GC root, and a comparison of two snapshots. A snapshot captured at a stop adds a Retained Size column to that stop's Memory tab.

New crate ide-heap: HotSpot hprof parser (bounds-checked; unfinished or crafted dumps are refused), object graph, dominator tree (semi-NCA), retained sizes and GC-root paths. Snapshots live in private directories removed with their window; an opened dump is analyzed from a private copy so changes to the user's file can't affect the mapped data.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 17:43:07 +08:00
Bin 8fafde10b4 fix(editor): keep tab actions at the pane edge (#2682)
The trailing slot of the editor tab strip had flex: none, so the split / more buttons followed the last tab. An automatic left margin keeps them at the right edge of the pane without changing how wrapped tabs share space.

Co-authored-by: ffgenius <[email protected]>
2026-11-11 17:35:16 +08:00
Muhammad N. Fadhil b1cd917a96 fix(gradle): process resources before launches from the language server's classpath (#2678)
The Java language server copies resources unchanged into Buildship's bin/<source set>, so launches with "Build and run using: OpenIDEA" saw unfiltered resources (e.g. `version: @version@`). The Gradle sync now records which source sets Gradle changes resources for (filters, expand, renames, includes/excludes, extra sources); before such a launch the IDE runs only those processResources tasks and copies their output over the plain copies. Builds that copy resources as they are run no extra Gradle step; if Gradle fails, the launch continues with a hint.

Closes #2675

Co-authored-by: MuhammadNFadhil <[email protected]>
2026-11-11 17:33:43 +08:00
l0ng-ai 605d5f4985 feat(new-project): project generators for mainstream languages and frameworks (#2689)
File › New Project offers 38 generators grouped by language (Java, Spring Boot, Quarkus, Micronaut, Kotlin, Ktor, Scala, Python/Django/FastAPI/Flask, Node/Vite/Next.js/Nuxt/Angular/SvelteKit/Astro/NestJS/Express, Go, Rust, C/C++, Zig, .NET, PHP/Laravel/Symfony, Ruby/Rails, Dart/Flutter, Swift, Elixir/Phoenix, Haskell, static website, empty). Each is a declaration in generators/<id>.toml run by the new ide-scaffold crate: built-in templates, the framework's official CLI run non-interactively, or a project service zip (Spring Initializr, code.quarkus.io, Micronaut Launch) whose metadata becomes form fields. Projects are built in a staging directory and moved into place only on success. The dialog renders each form from its schema, flags missing toolchains with install hints and streams progress; created projects open trusted. New Project is also on the welcome screen and the title bar project menu.

Closes #709
2026-11-11 17:22:57 +08:00
l0ng-ai 7d52f058a7 chore: record ide-search's encoding_rs dev-dependency in Cargo.lock (#2695) 2026-11-11 17:17:30 +08:00
l0ng-ai c44558b4aa fix(lsp): report a server missing while a variable it requires is unset (#2671)
With pwsh installed but PSES_BUNDLE_PATH unset, PowerShell Editor Services was launched with a broken path and restarted in a loop with a crash notification each time. Language servers can now declare requires-env; while one of those variables is unset or empty (in the server's env and the IDE's environment) the server counts as missing and shows its install hint instead of starting. powershell.toml declares PSES_BUNDLE_PATH; the field is documented.
2026-11-11 17:16:11 +08:00
l0ng-ai 8455761181 fix(lsp): show only the crashed instance's stderr, without colour codes (#2668)
A language server crash notification quoted stderr verbatim: coloured output showed raw escape sequences, and since the buffer was never cleared between restarts each report repeated earlier instances' output. Stderr lines are now stripped of ANSI escapes when recorded and the buffer is cleared when a new instance starts; strip_ansi moves to ide-core::output_encoding so ide-build and ide-lsp share it.
2026-11-11 17:16:07 +08:00
l0ng-ai dabf9a954e fix(search): find text in GBK and UTF-16 files (#2665)
Closed files that are not UTF-8 were searched as raw bytes: Chinese in GBK files never matched and ASCII matches showed mojibake, UTF-16 files were skipped as binary, and comment/string-limited searches skipped non-UTF-8 files. The engine now checks UTF-8 while streaming and searches non-UTF-8 files again in text decoded like the editor does (BOM / UTF-16 as detected, else the project's fallback encoding). Find in Files, Search Everywhere › Text, MCP and plugin search share this path.
2026-11-11 17:14:42 +08:00
ntbang0901 a3d1e57f3e feat(debugger): memory view for a paused JVM (#2648)
Adds a Memory tab to the Debug tool window for JVM sessions: loaded classes with instance counts and shallow sizes (with deltas between refreshes), read over the HotSpot attach listener (`GC.class_histogram`), falling back to counting reachable objects through the debugger on JDKs that cannot answer while suspended. While paused, a class's instances, an instance's fields and its referrers are listed through a JDWP link (the Kotlin adapter's path, and a jdtls bundle `java/jvm-inspect` for java-debug). Debug launches started by the IDE get `-XX:+StartAttachListener` unless the user's VM options decide otherwise (not on Windows).

New crates: ide-jdwp, ide-jvm-attach, ide-jvm-inspect. The bundled jar rebuilds byte-for-byte from build.py against the pinned JDT LS plugins.

Co-authored-by: ntbang0901 <[email protected]>
2026-11-11 17:10:21 +08:00
l0ng-ai 0ff3f188de feat(agents): add built-in Qoder (#2687)
Qoder's global CLI starts with `qodercli --acp` like Qoder CN; both editions share the icon.
2026-11-11 16:49:13 +08:00
wangfeiтаhapput 1654d22756 feat(agents): add built-in Qoder CN support (#2647)
Qoder CN appears in the built-in agent list and starts with `qoderclicn --acp` through the existing ACP driver, with sign-in guidance and a row in the agent guides. Its icon is a 16px glyph with a dark-theme variant.

Co-authored-by: happut <[email protected]>
2026-11-11 16:46:45 +08:00
Emanyzwww 14d9cdff11 feat(app): show a What's New page after an update (#2664)
After an update the restarted app opens what the new version brought as an editor tab, and Help > What's New reopens it at any time. Its checkbox turns the automatic opening off for later updates; the same switch sits on System Settings > Updates next to the update checks. The update balloon keeps a one-paragraph summary.

The release workflow fills the updater manifest's notes from the release body without its Download section, the install records them, and a version installed outside the app reads them from the manifest. The first start of a different version is recognised from the last run version kept in settings.
2026-11-11 16:42:32 +08:00
Muuтаmuutot f7a507a5db feat(vcs): allow rewriting history of pushed commits (#2644)
Edit Commit Message, Fixup, Squash Into, Drop Commit, Squash Commits and Interactively Rebase from Here are no longer disabled for pushed commits. A rewrite that reaches commits already on the upstream asks for confirmation first (pushing afterwards needs a force push); on a protected branch such a rewrite is refused, while its local commits can still be rewritten. Undo Commit follows the same rule.

Co-authored-by: muutot <[email protected]>
2026-11-11 16:41:45 +08:00
Charlie.Weiтаcharli117 c67e07a4f9 feat(plugins): cancel plugin commands per request (#2620)
Plugin commands could only be cancelled as a whole plugin. Local IPC and remote RPC now take a request id: a cancelled request is dropped before the plugin is activated, a running command gets a short grace period to exit and is interrupted after it, and other calls to the same plugin keep running. The frontend runCommand takes an AbortSignal that settles the caller at once and forwards the cancellation.

Co-authored-by: charli117 <[email protected]>
2026-11-11 16:37:16 +08:00
misswellтаmisswell 4e08893eae fix(run): build Maven modules through Maven on Rebuild (#2617)
A Rebuild launch of a Maven module used jdtls' java/buildWorkspace(true), which only recompiles sources: stale classes stayed in target/classes, and JDT rejected sources javac accepts (a package that doesn't match its directory), so Debug stopped with "Build has compile errors" although mvn clean compile succeeded. Rebuild now always delegates Maven modules to their Maven build (clean compile / clean test-compile), and a successful clean build clears the stale JDT diagnostics. Automatic and Make keep the JDT path.

Co-authored-by: misswell <[email protected]>
2026-11-11 16:33:19 +08:00
l0ng-ai 779070c3d7 feat(vcs): open diff follows the change tree selection; file buttons in the diff toolbar (#2628)
With a diff tab open from the Commit or Local Changes tree, selecting another file in the tree shows that file in the same tab and keeps focus in the tree; with no diff open, selecting opens nothing. The diff toolbar gets Compare Previous File / Compare Next File buttons when the diff has sibling files.

Closes #2492
2026-11-11 16:30:53 +08:00
l0ng-ai 4e86161c3c fix(run): debug Maven tests from the gutter and attach when Surefire starts the fork (#2684)
The run gutter and context menu offered Debug only for configurations with a debug plan or adapter, so Maven / Gradle tests (debugged through a JDWP agent) had none. Debugging a Maven test attached to the pinned Surefire port right away and java-debug's 30-second retry could run out while Maven recompiled; it now attaches when Maven prints the T E S T S banner just before starting the fork. A failed attach to a JVM started for the debugger now stops that run (as for Python), so its suspended JVM doesn't keep it alive.

Fixes #2656
2026-11-11 16:28:14 +08:00
arookieofc d7393b53f3 feat(menu): add always-show main menu option (#1521) (#2661)
* feat(menu): add always-show main menu option (#1521)

* fix(i18n): add zh catalog entries for always-show main menu strings
2026-11-11 16:22:52 +08:00
Deng Liuтаdengliu 276a91973c fix(terminal): use editor search colors, keep the active match above the selection, and seed ⌘F (#2606)
The terminal Find bar painted matches with the old text-search tokens; it now uses --bg-search-match / --bg-search-match-current (blended over the panel background) like the editor. The active match's decoration goes to xterm's top layer so it stays visible above the selection the search places, and ⌘F with the Find bar already open seeds it from a fresh user selection while keeping the query when the selection is the current match.

Co-authored-by: dengliu <[email protected]>
2026-11-11 16:17:36 +08:00
l0ng-ai fc75b15519 perf(editor): find a type's overridden methods with one subtype walk (#2680)
The gutter's "is implemented / overridden" markers sent one implementation request per visible method, and jdtls builds the type hierarchy again for each, so a large interface in a big project kept it busy long enough to hold up navigation. When outline names carry the signature (jdtls), a type's methods are now resolved from one walk of its subtypes and their outlines, as the super-method markers are; used only while it takes fewer requests than it replaces, and a method whose name a subtype reuses with another signature (generic overloads) still gets its own request.

Refs #2314
2026-11-11 16:04:10 +08:00
arookieofc 8798c2a542 fix(remote): sync outside-root classpath entries for SSH Java launches (#2641) (#2669) 2026-11-11 15:55:30 +08:00
Deng Liuтаdengliu 8ea2787490 fix(editor): keep selection within its own row when font glyphs exceed line height (#2674)
With a font whose glyph box is taller than the line height (e.g. font size 18–20 with the default 22px line height), selecting text on one line also painted the rows above and below: growToLineBoxes probed the rows at the glyph box's top and bottom edges, which then fell in the neighbouring rows, and .cm-openidea-selected painted an unclipped inline background at glyph height. Outer rows are now resolved at each marker's vertical midpoint and row seams align to the line boxes (block widgets and zero-width trailing markers handled), and the inline selection background is capped at 1lh in the editor, notebook cells and diff/merge viewers, with a plain colour fallback where lh is unsupported.

Co-authored-by: dengliu <[email protected]>
2026-11-11 15:54:26 +08:00
l0ng-ai dc4e8403de fix(java): keep a language server JDK that ran usable while it is unchanged (#2679)
A JDK whose java -version succeeded was probed again every 10 seconds; on a busy machine one probe could time out and the working JDK was reported as "cannot run on this machine". A successful probe now stays valid while the executable's size, modification time and permissions are unchanged; failed probes are still retried after 10 seconds, and a java that hangs on its first probe is still rejected.

Fixes #2470
2026-11-11 15:50:10 +08:00
l0ng-ai 4a0efbae25 fix(agents): keep a run or turn the reader opened open across remounts
A run's open state lived in its row, so it closed when the row remounted: when a
finished turn folded its work away, or when a run grew from one call to two.
Opened runs and turns are now remembered by their first entry for the
conversation's lifetime.
2026-11-11 15:36:22 +08:00
l0ng-ai cb3c0bafd7 fix(boot): count the startup timeout only while the window shows
A window behind others or on another Space gets no animation frames, and the
splash waits for two of them before startup finishes. The 30-second timeout kept
running meanwhile and reported a healthy but slow start, such as a cold dev
build, as "OpenIDEA could not start". It now pauses while the document is hidden
and resumes with the time left.
2026-11-11 15:34:30 +08:00
l0ng-ai 71072dbe4b fix(agents): pace streaming reasoning, cut only between graphemes, cap at 60 Hz
- Reasoning in view streams at the reply's pace instead of in bursts.
- The painted end of a streaming text falls between graphemes (Intl.Segmenter):
  an emoji with joiners or a skin tone, a flag or a letter with combining marks
  no longer flashes in parts. A chunk ending mid-cluster holds that fragment back
  until the rest arrives.
- The reveal commits at most at 60 Hz, so a 120 Hz display doesn't re-render the
  Markdown twice as often.
2026-11-11 15:33:53 +08:00
l0ng-ai 18b286e7e0 fix(agents): one spinner per conversation, counts on one line
Work in progress shimmers on its label (a run's counts and current call, a
running tool, "Thinking") instead of showing a spinner beside it; the working
row at the conversation's end is the only spinner. A run's counts keep their
line, and what runs now ellipsizes in the space left.
2026-11-11 15:32:40 +08:00
l0ng-ai 9474af4de1 fix(agents): steadier tool-call runs, working row in the stream, selection and scroll fixes
- Reasoning and tool calls fold into one run including running calls, so a run
  keeps its place while a call finishes. Its summary names what runs now; it no
  longer leads with "Reasoning". A folded thought shows its first line. An open
  run is bounded and follows its latest call.
- The working / starting / retrying row with its timer ends the conversation
  instead of sitting above the composer, for sessions and subagents alike.
- Selecting text no longer pops up a Quote button; Quote stays in the context menu.
- A selection clamped to the conversation ends in its last selectable text, not on
  its edge: WebKit painted a selection ending there into the composer, as a
  triple-click on the last paragraph showed.
- Sending a message while scrolled up returns to the end, where it shows.
2026-11-11 15:20:09 +08:00
arookieofc 8044d99ea0 feat(build): compile single Java files and skip unchanged Maven builds (#2476) (#2646)
* feat(build): compile a single Java file with javac (#2476)

* feat(build): add Compile File action for single Java sources (#2476)

* feat(build): skip unchanged Maven menu builds via fingerprint records (#2476)

* fix(build): drop shadowed fixture model in unix-only classpath test (#2476)

* fix(build): pass cancel into shared executeBuild helper (#2476)

* chore: retrigger CI after mass-cancel wave (#2646)

* chore: probe CI queue recovery (#2646)
2026-11-11 15:03:42 +08:00
Charlie.Weiтаcharli117 0abaf6c777 fix(plugins): keep plugin settings that match their defaults (#2619)
Choosing a plugin setting's default value deleted the override, so in a project the global value showed through instead of the user's choice. The setting control now always writes the chosen value.

Co-authored-by: charli117 <[email protected]>
2026-11-11 14:45:40 +08:00
LU XIANGRUIтаwildcatDownstairs 0d9b08ac8c fix(color-scheme): import WebStorm JavaScript and TypeScript colours from .icls (#2634)
WebStorm stores JavaScript and TypeScript colours under JS.* / TS.* keys, which the .icls importer did not map, and DEFAULT_SEMICOLON / _COMMA / _DOT all mapped to Braces and Operators, so a scheme recolouring semicolons painted every brace and paren. JS.* and TS.* now map onto the Language Defaults attributes with TS > JS > PY > defaults precedence, and the separator keys are no longer mapped.

Co-authored-by: wildcatDownstairs <[email protected]>
2026-11-11 14:45:32 +08:00
Maskerтаzed1995 e54580265d fix(agents): show Codex ACP collaboration mode in the composer (#2631)
codex-acp reports its Default/Plan choice as a collaboration_mode config option, which the composer dropped because it only rendered model, mode, permissions, approval and thought_level. collaboration_mode is now one of the composer's ordered settings, so a Codex session can switch to Plan.

Co-authored-by: zed1995 <[email protected]>
2026-11-11 14:45:24 +08:00
LU XIANGRUIтаwildcatDownstairs 4fa700b337 fix(editor): collapse the bookmark gutter fully in files without bookmarks (#2632)
The bookmark gutter's 2px side padding applied to the empty spacer element too, so the column kept 4px in files without bookmarks. The padding now applies only to elements that hold a marker, so the column is 0px until the file has a bookmark.

Co-authored-by: wildcatDownstairs <[email protected]>
2026-11-11 14:45:16 +08:00
DrPepperтаmakise2060 173d5edcdc fix(run): render the Run console with the terminal font (#2615)
The Run console hard-coded the --font-mono stack and --fs-code, so Settings › Tools › Terminal › Font (family, fallback, size, or Use editor font) never reached process output. resolveMonoFont() now resolves the terminal font choice for both xterm and the DOM console, and the console's virtual row height follows the font size (round(size × 1.5), 20px at the default 13px) so rows don't overlap.

Co-authored-by: makise2060 <[email protected]>
2026-11-11 14:43:59 +08:00
Gabriel Gondim 4d60921f24 feat(database): list all databases on a SQL Server host (#2629)
SQL Server data sources without single-database mode now show a database level in the tree, like PostgreSQL: every online database the login can access, each expanding to its schemas, with the Schemas tab database picker and the Single database mode option. The backend already listed them; listsDatabases() only enabled the level for Postgres. A failed connect shows an error under the node instead of leaving it on Loading, and saving Properties reloads what the tree had loaded for that source so expanded nodes don't stay on Loading.

Closes #2600

Co-authored-by: GGVidal <[email protected]>
2026-11-11 14:40:52 +08:00
Paulem 7784f3cdef fix(dialog): fall back to a GTK file chooser on Linux when no portal or zenity is available (#2394)
On Linux sessions with neither an xdg-desktop-portal nor zenity (antiX), rfd's xdg-portal backend returned nothing and every Open / Save picker silently did nothing. All pickers now go through src/platform/dialog.ts; on Linux the backend checks once for a reachable portal or zenity and otherwise shows a GTK FileChooserDialog. Other platforms and sessions keep the desktop's dialog. dialog.test.ts keeps every picker (including Move Worktree, keymap import, Mermaid export and library sources) on the wrapper.

Fixes #2386

Co-authored-by: Paulem79 <[email protected]>
2026-11-11 14:25:03 +08:00
misswellтаmisswell b010397711 fix(lsp): mark progress tasks that stop reporting (#757)
A $/progress task that reports nothing for 3 minutes (jdtls's Maven import waiting on a repository that never answers) kept the server Busy forever, which also held wait_idle and ReferencesIncomplete::Indexing. Such a task now stays open with its last message and fraction, marked "no progress for 180 s", and no longer counts as busy; a late report clears the mark and the same task finishes normally.

Co-authored-by: misswell <[email protected]>
2026-11-11 14:18:16 +08:00
arookieofc 108da537f4 feat(agents): list and import sessions started outside OpenIDEA (#2640)
* feat(agents): list and import sessions started outside OpenIDEA

Sessions started in a terminal, another editor or the CLIs never showed
up in OpenIDEA, so users had to re-explain their work (#312). The agent
history popup now lists them under Sessions from other apps: Claude Code
transcripts and Codex rollouts on this machine, with title, working
directory and age. Importing one creates a task whose first turn loads
and replays the transcript through the existing session/load path.

* i18n: add zh translation for 'Sessions from other apps'
2026-11-11 12:42:01 +08:00
Deng Liuтаdengliu c658b6e59f feat(terminal): highlight visible occurrences of selected text (#2638)
Selecting text in the terminal highlights its other occurrences in the visible rows, like the editor's identifier-under-caret highlight. Only the viewport (plus a few rows of margin) is scanned, matches are cached per logical line and drawn in one overlay layer, recomputation is coalesced to one pass per animation frame, and highlights are cleared during heavy output and redrawn once it settles. Wrapped lines and wide characters map to the right cells; multi-line, very short or whitespace-only selections, the alternate buffer and Find-bar selections are skipped. Settings › Tools › Terminal › Highlight occurrences of selected text turns it off.

Co-authored-by: dengliu <[email protected]>
2026-11-11 12:41:33 +08:00
Deng Liuтаdengliu 990a1c7a87 feat(agent): retry session setup after ACP authenticate on auth error (#2637)
When session/load or session/new fails with an authentication error and the agent advertises a non-terminal `agent` authentication method in `initialize`, the IDE now calls ACP `authenticate` with that method once per session setup and retries the failed call before falling back to the sign-in error. Terminal-login methods (`_meta.terminal-auth`) keep using the existing sign-in guidance.

Co-authored-by: dengliu <[email protected]>
2026-11-11 12:37:34 +08:00
l0ng-ai a58a7a0b8c fix(project): keep the project root icon outside workspaces (#2657)
Project type detection only runs for workspace members. A single project
root no longer takes the icon of its first matching build marker, which
labelled a Cargo + Vite repository as a Vite project.
2026-11-11 12:36:12 +08:00
arookieofc c4b47da9d1 fix(maven): link Windows path variants and warn on outside-root picks (#2639)
* fix(maven): link Windows path variants and warn on outside-root picks

The Add Maven Projects dialog hands back whatever form Windows uses:
drive-letter case may differ from the stored root and long paths come
back \?\ -prefixed, so in-root picks were silently dropped and the
button looked dead (#2626). Normalize both sides before comparing
(case-insensitively on Windows) and notify which picks were skipped
because Maven only links POMs inside the open project.

* i18n: tag Maven outside-root notice with tx and add zh entries
2026-11-11 12:33:15 +08:00
l0ng-ai 9d64134b68 fix(agents): keep list selection unbroken in agent replies (#2655)
Draw list markers as non-selectable pseudo-elements instead of native
markers. WebKit paints a selection box around each native marker, which
left a broken stripe down the list gutter when a reply was selected.
2026-11-11 12:32:27 +08:00
l0ng-ai ca945b13d2 feat(plugins): show available updates on the Installed tab (#2654)
Installed plugins with a newer, loadable marketplace version get an Update
badge in the list and an "Update to x.y.z" button in their details, which
installs it from the marketplace.
2026-11-11 12:32:22 +08:00
l0ng-ai 97c58cc808 fix(agents): keep a conversation drag-selection out of the composer (#2653)
Dragging a selection from the last message down past the conversation
highlighted the composer's text field. During such a drag everything outside
the conversation is made unselectable, but WebKit keeps editable text
selectable regardless of user-select, so the selection reached the textarea.
Text fields outside the selection scope now also drop out of hit testing while
the drag lasts.
2026-11-11 12:32:17 +08:00
l0ng-ai e158658afb fix(agents): drop the status bar's Changes toggle (#2652)
The session status bar ended with a Changes toggle that did the same as the
"N changed files" chip right below it. The chip opens and closes the Changes
pane and also shows the line counts, the pane has its own hide button, and the
status-bar toggle was disabled whenever there were no changes, so the status
bar now keeps only the Terminal button. The unused "Show changes" string goes
from the locale catalogs.
2026-11-11 12:32:12 +08:00
l0ng-ai db4bb52694 fix(agents): drop the running tool's name from the working row (#2645)
The tool in progress already shows as the last card of the conversation, and
next to it the row's elapsed time read as that tool's duration rather than the
turn's. The row now shows Working… with the turn's elapsed time, and still
shows retries and the starting state.
2026-11-11 12:31:37 +08:00
l0ng-ai 65bb39e08c test(plugins): give the official examples test a longer command budget (#2643)
The plugin_data_tools harness caps plugin commands at 250 ms to test interruption; the official examples' first command compiles and scans the project and ran over that on Windows runners. That test now uses a 10 s budget.
2026-11-11 12:08:25 +08:00
853 змінених файлів з 66911 додано та 3291 видалено

No files matched your search

+2
Переглянути файл
@@ -0,0 +1,2 @@
custom:
- https://github.com/l0ng-ai/openidea#sponsor
Бінарний файл не відображається.

Після

Ширина:  |  Висота:  |  Розмір: 70 KiB

Бінарний файл не відображається.

Після

Ширина:  |  Висота:  |  Розмір: 97 KiB

+22 -1
Переглянути файл
@@ -251,7 +251,28 @@ jobs:
core.setFailed(`latest.json would miss ${missing.join(", ")}`);
return;
}
const manifest = { version: tag.replace(/^v/, ""), notes: "", pub_date: new Date().toISOString(), platforms };
// The update's notes: the release body without its Download section. That
// section only matters on the release page, and What's new follows it.
const { data: release } = await github.rest.repos.getRelease({ owner, repo, release_id });
const withoutDownloadSection = (body) => {
const kept = [];
let skip = 0;
for (const line of body.split(/\r?\n/)) {
const heading = /^(#{1,6})\s+(.*)$/.exec(line);
if (heading) {
const level = heading[1].length;
if (!skip && level <= 3 && /^download$/i.test(heading[2].trim())) {
skip = level;
continue;
}
if (skip && level <= skip) skip = 0;
}
if (!skip) kept.push(line);
}
return kept.join("\n").trim();
};
const notes = withoutDownloadSection(String(release.body ?? ""));
const manifest = { version: tag.replace(/^v/, ""), notes, pub_date: new Date().toISOString(), platforms };
const old = assets.find((a) => a.name === "latest.json");
if (old) await github.rest.repos.deleteReleaseAsset({ owner, repo, asset_id: old.id });
await github.rest.repos.uploadReleaseAsset({
+1
Переглянути файл
@@ -12,6 +12,7 @@
"src/editor/core/cm/angular/parser*.ts",
"src/editor/core/cm/jsp/parser*.ts",
"src/editor/core/cm/astro/parser*.ts",
"src/editor/core/cm/django/parser*.ts",
"plugins/themes.md",
"pnpm-lock.yaml",
"**/*.toml",
Створено
+121 -42
Переглянути файл
@@ -4145,7 +4145,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "ide-agent"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
@@ -4169,7 +4169,7 @@ dependencies = [
[[package]]
name = "ide-analysis"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ast-grep-core",
"ide-core",
@@ -4183,7 +4183,7 @@ dependencies = [
[[package]]
name = "ide-android"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"serde",
@@ -4194,7 +4194,7 @@ dependencies = [
[[package]]
name = "ide-build"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"blake3",
"chardetng",
@@ -4212,7 +4212,7 @@ dependencies = [
[[package]]
name = "ide-cli"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"serde",
@@ -4226,7 +4226,7 @@ dependencies = [
[[package]]
name = "ide-clipboard"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"gtk",
"objc2 0.6.5",
@@ -4236,7 +4236,7 @@ dependencies = [
[[package]]
name = "ide-core"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
@@ -4253,7 +4253,7 @@ dependencies = [
[[package]]
name = "ide-coverage"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-run",
@@ -4267,15 +4267,20 @@ dependencies = [
[[package]]
name = "ide-dap"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"futures",
"ide-core",
"ide-heap",
"ide-jdwp",
"ide-jsonrpc",
"ide-jvm-attach",
"ide-jvm-inspect",
"ide-language",
"libc",
"memchr",
"parking_lot",
"percent-encoding",
"portable-pty",
"serde",
"serde_json",
@@ -4288,7 +4293,7 @@ dependencies = [
[[package]]
name = "ide-db"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"bb8",
"blake3",
@@ -4318,7 +4323,7 @@ dependencies = [
[[package]]
name = "ide-docker"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"serde",
@@ -4331,7 +4336,7 @@ dependencies = [
[[package]]
name = "ide-extension"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4352,7 +4357,7 @@ dependencies = [
[[package]]
name = "ide-fonts"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"objc2-core-foundation",
"objc2-core-text",
@@ -4364,7 +4369,7 @@ dependencies = [
[[package]]
name = "ide-github"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-vcs",
@@ -4375,9 +4380,27 @@ dependencies = [
"which",
]
[[package]]
name = "ide-heap"
version = "0.1.42"
dependencies = [
"criterion",
"ide-core",
"libc",
"memmap2",
"proptest",
"serde",
"serde_json",
"specta",
"tempfile",
"thiserror 2.0.21",
"tracing",
"uuid",
]
[[package]]
name = "ide-host"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4397,9 +4420,11 @@ dependencies = [
"ide-db",
"ide-docker",
"ide-extension",
"ide-heap",
"ide-http",
"ide-index",
"ide-jsonrpc",
"ide-jvm-attach",
"ide-language",
"ide-lsp",
"ide-mcp",
@@ -4407,6 +4432,7 @@ dependencies = [
"ide-plugin",
"ide-remote",
"ide-run",
"ide-scaffold",
"ide-sdk",
"ide-search",
"ide-spell",
@@ -4443,7 +4469,7 @@ dependencies = [
[[package]]
name = "ide-http"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-vfs",
@@ -4462,7 +4488,7 @@ dependencies = [
[[package]]
name = "ide-index"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"blake3",
"criterion",
@@ -4482,9 +4508,21 @@ dependencies = [
"tracing",
]
[[package]]
name = "ide-jdwp"
version = "0.1.42"
dependencies = [
"ide-jvm-inspect",
"proptest",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tracing",
]
[[package]]
name = "ide-jsonrpc"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"encoding_rs",
"futures",
@@ -4499,9 +4537,29 @@ dependencies = [
"tracing",
]
[[package]]
name = "ide-jvm-attach"
version = "0.1.42"
dependencies = [
"libc",
"tempfile",
"thiserror 2.0.21",
]
[[package]]
name = "ide-jvm-inspect"
version = "0.1.42"
dependencies = [
"futures",
"serde",
"serde_json",
"specta",
"thiserror 2.0.21",
]
[[package]]
name = "ide-language"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"blake3",
"criterion",
@@ -4557,13 +4615,15 @@ dependencies = [
[[package]]
name = "ide-lsp"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
"futures",
"globset",
"ide-core",
"ide-jsonrpc",
"ide-jvm-inspect",
"ide-language",
"ide-text",
"ide-vfs",
@@ -4583,7 +4643,7 @@ dependencies = [
[[package]]
name = "ide-mcp"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"bytes",
"dunce",
@@ -4602,7 +4662,7 @@ dependencies = [
[[package]]
name = "ide-notebook"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"futures",
"ide-core",
@@ -4625,7 +4685,7 @@ dependencies = [
[[package]]
name = "ide-plugin"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"blake3",
"dunce",
@@ -4648,9 +4708,10 @@ dependencies = [
[[package]]
name = "ide-profiler"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-jvm-attach",
"ide-run",
"serde",
"serde_json",
@@ -4661,7 +4722,7 @@ dependencies = [
[[package]]
name = "ide-remote"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dunce",
"globset",
@@ -4680,7 +4741,7 @@ dependencies = [
[[package]]
name = "ide-run"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dunce",
"encoding_rs",
@@ -4702,9 +4763,25 @@ dependencies = [
"zip",
]
[[package]]
name = "ide-scaffold"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-tools",
"ide-vcs",
"insta",
"serde",
"serde_json",
"specta",
"tempfile",
"toml 0.9.12+spec-1.1.0",
"tracing",
]
[[package]]
name = "ide-sdk"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dunce",
"ide-core",
@@ -4724,9 +4801,10 @@ dependencies = [
[[package]]
name = "ide-search"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"criterion",
"encoding_rs",
"globset",
"grep-matcher",
"grep-regex",
@@ -4744,7 +4822,7 @@ dependencies = [
[[package]]
name = "ide-spell"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"parking_lot",
@@ -4757,7 +4835,7 @@ dependencies = [
[[package]]
name = "ide-ssh"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"flate2",
@@ -4778,7 +4856,7 @@ dependencies = [
[[package]]
name = "ide-syntax"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"ide-language",
@@ -4793,7 +4871,7 @@ dependencies = [
[[package]]
name = "ide-terminal"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"ide-core",
"libc",
@@ -4806,7 +4884,7 @@ dependencies = [
[[package]]
name = "ide-text"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"criterion",
"ide-core",
@@ -4819,7 +4897,7 @@ dependencies = [
[[package]]
name = "ide-tools"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"flate2",
"hex",
@@ -4840,7 +4918,7 @@ dependencies = [
[[package]]
name = "ide-vcs"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"blake3",
"dunce",
@@ -4861,7 +4939,7 @@ dependencies = [
[[package]]
name = "ide-vfs"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"chardetng",
"dunce",
@@ -4885,11 +4963,11 @@ dependencies = [
[[package]]
name = "ide-web"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"dirs 6.0.0",
"ide-core",
"open",
"parking_lot",
"percent-encoding",
"serde",
"specta",
@@ -4901,7 +4979,7 @@ dependencies = [
[[package]]
name = "ide-workspace"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"blake3",
"dunce",
@@ -6479,7 +6557,7 @@ dependencies = [
[[package]]
name = "openidea"
version = "0.1.41"
version = "0.1.42"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -6513,6 +6591,7 @@ dependencies = [
"ide-profiler",
"ide-remote",
"ide-run",
"ide-scaffold",
"ide-sdk",
"ide-search",
"ide-spell",
@@ -7719,9 +7798,9 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.1.5"
version = "1.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
checksum = "e5e76742e5294153d4dde1207dbec5bdd81eef6fabdc0d2118babadabfd858aa"
dependencies = [
"bitflags 2.13.2",
"errno",
+7 -1
Переглянути файл
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.41"
version = "0.1.42"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
@@ -43,10 +43,15 @@ ide-ssh = { path = "crates/ide-ssh" }
ide-remote = { path = "crates/ide-remote" }
ide-host = { path = "crates/ide-host" }
ide-mcp = { path = "crates/ide-mcp" }
ide-scaffold = { path = "crates/ide-scaffold" }
ide-notebook = { path = "crates/ide-notebook" }
ide-extension = { path = "crates/ide-extension" }
ide-plugin = { path = "crates/ide-plugin" }
ide-cli = { path = "crates/ide-cli" }
ide-jvm-attach = { path = "crates/ide-jvm-attach" }
ide-jvm-inspect = { path = "crates/ide-jvm-inspect" }
ide-jdwp = { path = "crates/ide-jdwp" }
ide-heap = { path = "crates/ide-heap" }
# serialization & IPC typing
serde = { version = "1", features = ["derive"] }
@@ -82,6 +87,7 @@ trash = "5"
blake3 = "1"
dunce = "1"
dirs = "6"
memmap2 = "0.9"
zip = { version = "9.2.3", default-features = false, features = ["deflate"] }
# search, index, language
+13
Переглянути файл
@@ -19,6 +19,8 @@
<a href="#plugins">Plugins</a>
&nbsp;·&nbsp;
<a href="#community">Community</a>
&nbsp;·&nbsp;
<a href="#sponsor">Sponsor</a>
</p>
https://github.com/user-attachments/assets/234dd54c-1255-4b3a-b35c-533d78d737c0
@@ -68,6 +70,17 @@ the [official plugins](plugins/official). The plugin API is still a preview.
- Contributing: see [CONTRIBUTING.md](CONTRIBUTING.md) and the [contributor docs](https://openidea.sh/en/contribute/setup).
## Sponsor
If OpenIDEA saves you time, you can support its development with Alipay or WeChat Pay.
<table>
<tr>
<td align="center"><img src=".github/assets/sponsor-alipay.png" alt="Alipay QR code" width="200"><br>Alipay</td>
<td align="center"><img src=".github/assets/sponsor-wechat.png" alt="WeChat Pay QR code" width="200"><br>WeChat Pay</td>
</tr>
</table>
## License
[Apache 2.0](LICENSE). Bundled third-party assets are listed in [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md).
+3 -4
Переглянути файл
@@ -1,11 +1,10 @@
# Third-party licenses
## Flutter and Vite project icons
## Project type icons
`src/ui/brands/flutter.svg` and `vite.svg` use the unmodified logo paths from
`src/ui/brands/*.svg` use the unmodified logo paths from
[Simple Icons](https://github.com/simple-icons/simple-icons), retrieved through
Iconify (`simple-icons:flutter`, `simple-icons:vite`) and filled with the project
icon colors. Licensed under [CC0 1.0 Universal](https://github.com/simple-icons/simple-icons/blob/develop/LICENSE.md).
Iconify (`simple-icons:angular`, `simple-icons:astro`, `simple-icons:cplusplus`, `simple-icons:dart`, `simple-icons:django`, `simple-icons:dotnet`, `simple-icons:elixir`, `simple-icons:express`, `simple-icons:fastapi`, `simple-icons:flask`, `simple-icons:flutter`, `simple-icons:go`, `simple-icons:haskell`, `simple-icons:html5`, `simple-icons:kotlin`, `simple-icons:ktor`, `simple-icons:laravel`, `simple-icons:nestjs`, `simple-icons:nextdotjs`, `simple-icons:nodedotjs`, `simple-icons:nuxt`, `simple-icons:openjdk`, `simple-icons:phoenixframework`, `simple-icons:php`, `simple-icons:python`, `simple-icons:quarkus`, `simple-icons:ruby`, `simple-icons:rubyonrails`, `simple-icons:rust`, `simple-icons:scala`, `simple-icons:springboot`, `simple-icons:svelte`, `simple-icons:swift`, `simple-icons:symfony`, `simple-icons:vite`, `simple-icons:zig`) and filled with the brand colors (the standard icon grey for black logos). Licensed under [CC0 1.0 Universal](https://github.com/simple-icons/simple-icons/blob/develop/LICENSE.md).
## Project Lombok
+102
Переглянути файл
@@ -0,0 +1,102 @@
//! A short-lived ACP session to learn models, effort levels and other config options
//! before the user's first task (Cursor and other ACP agents do not expose these in
//! `initialize` alone).
use super::{Conn, Dialect};
use crate::config;
use crate::model::AgentConfigOption;
use crate::provider::ProviderSpec;
use crate::{Shared, client_info};
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::sync::Arc;
use std::time::Duration;
const DISCOVER_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(3) } else { Duration::from_secs(120) };
/// Opens `session/new`, reads `configOptions`, and records per-model `thought_level` choices.
pub(crate) async fn run(shared: &Arc<Shared>, spec: &ProviderSpec) -> Result<(), String> {
let root = shared.paths.project_root.clone();
let launch = spec.clone();
let launch = tokio::task::spawn_blocking({
let root = root.clone();
move || launch.resolve_for_launch(&root)
})
.await
.map_err(|e| e.to_string())?;
let cwd = dunce::canonicalize(&root).unwrap_or(root);
let stderr: Arc<Mutex<Vec<String>>> = Arc::default();
let lines = stderr.clone();
let on_stderr: Box<dyn Fn(String) + Send + Sync> = Box::new(move |l| lines.lock().push(l));
let client = Dialect::wrap(Arc::new(ide_jsonrpc::NoopHandler));
let peer = (shared.connector)(&launch, &cwd, client.clone(), on_stderr)
.map_err(|e| format!("Couldn't start {} (`{}`): {e}", launch.name, launch.command))?;
let conn = Conn::new(peer.clone(), client);
let work = async {
conn.initialize(&client_info()).await.map_err(|e| e.to_string())?;
let cwd_s = cwd.to_string_lossy().into_owned();
// No project MCP during discovery: Cursor can spend a long time spawning servers.
let new: Value = conn
.request("session/new", json!({ "cwd": cwd_s, "mcpServers": [] }), None)
.await
.map_err(|e| e.to_string())?;
let mut options = config::options_of(&new);
config::add_client_options(&launch.id, &mut options);
let model_option = options.iter().find(|o| o.category.as_deref() == Some("model"));
let effort = levels_for_model_from_slice(&options);
let levels: Vec<(String, Vec<AgentConfigOption>)> = match (model_option, effort) {
(Some(model), Some(effort)) if !model.values.is_empty() => {
model.values.iter().map(|v| (v.value.clone(), effort.clone())).collect()
}
(Some(model), Some(effort)) => model.current.clone().map(|m| vec![(m, effort)]).unwrap_or_default(),
_ => Vec::new(),
};
let option_ids: Vec<_> = options.iter().map(|o| o.id.as_str()).collect();
tracing::info!(
"agent {}: settings discovery got {} config option(s) {:?}, {} model effort list(s)",
launch.id,
options.len(),
option_ids,
levels.len()
);
shared.remember_discovered(&launch.id, &options, &[], &levels);
Ok::<(), String>(())
};
let result = tokio::time::timeout(DISCOVER_TIMEOUT, work).await;
peer.kill();
match result {
Ok(Ok(())) => Ok(()),
Ok(Err(e)) => Err(match stderr.lock().last() {
Some(line) => format!("{e}: {line}"),
None => e,
}),
Err(_) => {
let tail = stderr.lock().join("\n");
if tail.is_empty() {
Err("the agent did not answer in time".into())
} else {
Err(format!("the agent did not answer in time:\n{tail}"))
}
}
}
}
fn is_thought_level(o: &AgentConfigOption) -> bool {
o.category.as_deref() == Some("thought_level")
|| matches!(o.id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
}
fn levels_for_model_from_slice(options: &[AgentConfigOption]) -> Option<Vec<AgentConfigOption>> {
let levels: Vec<_> = options
.iter()
.filter(|o| is_thought_level(o))
.cloned()
.map(|mut o| {
if o.category.as_deref() != Some("thought_level") {
o.category = Some("thought_level".into());
}
o
})
.collect();
(!levels.is_empty()).then_some(levels)
}
+15
Переглянути файл
@@ -14,6 +14,7 @@
//! - Agent → client ([`Dialect`]): `session/update`s are normalized ([`normalize`]) and
//! permission requests get the v1 `toolCall`.
pub(crate) mod discover;
mod normalize;
use ide_jsonrpc::{Handler, Peer, RpcError, codes};
@@ -57,6 +58,9 @@ fn client_capabilities() -> Value {
"terminal": false,
// Subagents as sessions of their own (ACP subagents RFD; v2 baseline).
"subagents": {},
// Cursor exposes model parameters (e.g. Composer `fast`) as separate config options
// only when the client advertises this flag (undocumented Cursor ACP extension).
"_meta": { "parameterizedModelPicker": true },
})
}
@@ -178,6 +182,17 @@ impl Negotiated {
pub fn meta(&self, pointer: &str) -> Option<&Value> {
self.capabilities.get("_meta")?.pointer(pointer)
}
/// First authentication method the agent can run over ACP (`type` absent or `"agent"`).
pub fn agent_auth_method(&self) -> Option<&str> {
self.auth_methods.iter().find_map(|m| {
let kind = m.get("type").and_then(Value::as_str).unwrap_or("agent");
(kind == "agent" && m.pointer("/_meta/terminal-auth").is_none())
.then(|| m.get("id").and_then(Value::as_str))
.flatten()
.filter(|id| !id.is_empty())
})
}
}
/// The client side of one agent connection: wraps the task's [`Handler`] and, once
+104 -2
Переглянути файл
@@ -93,9 +93,81 @@ fn the_negotiating_request_carries_both_versions() {
assert!(p["capabilities"].is_object());
assert_eq!(p["clientCapabilities"]["subagents"], json!({}));
assert_eq!(p["clientCapabilities"]["terminal"], false);
assert_eq!(
p["clientCapabilities"]["_meta"]["parameterizedModelPicker"], true,
"Cursor and similar agents need parameterized model options (e.g. fast on/off)"
);
// What we sent in the capture is what claude-agent-acp negotiated v2 and v1 from.
let sent = &frames(TURN, "out")[0]["params"];
assert_eq!(sent["clientCapabilities"], p["clientCapabilities"]);
let sent = frames(TURN, "out")[0]["params"]["clientCapabilities"].clone();
let mut expected = sent;
expected["_meta"] = p["clientCapabilities"]["_meta"].clone();
assert_eq!(expected, p["clientCapabilities"]);
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent() {
#[derive(Default)]
struct DiscoveryAgent {
peer: std::sync::OnceLock<Peer>,
calls: parking_lot::Mutex<Vec<(String, Value)>>,
}
impl Handler for DiscoveryAgent {
fn on_notification(&self, _: &str, _: Value) {}
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push((method.to_owned(), params));
match method {
"initialize" => Ok(json!({
"protocolVersion": 1,
"agentCapabilities": { "loadSession": true }
})),
"session/new" => Ok(json!({
"sessionId": "discover",
"configOptions": [
{"id": "model", "category": "model", "type": "select", "currentValue": "m1",
"options": [{"value": "m1", "name": "One"}, {"value": "m2", "name": "Two"}]},
{"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
})),
_ => Ok(json!({})),
}
}
}
let agent = Arc::new(DiscoveryAgent::default());
let tmp = tempfile::tempdir().unwrap();
let agent_for_connector = agent.clone();
let connector: crate::Connector = Arc::new(move |_, _, client, _| {
let (ca, cb) = tokio::io::duplex(1 << 20);
let (car, caw) = tokio::io::split(ca);
let (cbr, cbw) = tokio::io::split(cb);
let agent_peer = Peer::connect_framed(cbr, cbw, agent_for_connector.clone(), Framing::Lines);
let _ = agent_for_connector.peer.set(agent_peer);
Ok(Peer::connect_framed(car, caw, client, Framing::Lines))
});
let mgr = crate::AgentManager::with_connector(
crate::AgentPaths {
project_root: tmp.path().to_path_buf(),
store_dir: tmp.path().join("store"),
worktrees_dir: tmp.path().join("worktrees"),
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
},
tokio::runtime::Handle::current(),
connector,
);
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert_eq!(calls.iter().map(|(m, _)| m.as_str()).collect::<Vec<_>>(), ["initialize", "session/new"]);
assert_eq!(calls[1].1["mcpServers"], json!([]));
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
let fast = cursor.config.iter().find(|o| o.id == "fast").expect("fast option");
assert_eq!(fast.category.as_deref(), Some("model_config"));
assert_eq!(fast.current.as_deref(), Some("false"));
assert!(cursor.thought_levels.contains_key("m1") && cursor.thought_levels.contains_key("m2"));
assert!(cursor.thought_levels["m1"].iter().any(|o| o.category.as_deref() == Some("thought_level")));
}
#[test]
@@ -135,6 +207,36 @@ fn the_shape_decides_the_version_not_the_number() {
assert_eq!((n.generation, n.load_session(), n.session("fork")), (2, false, false));
}
#[test]
fn picks_non_terminal_agent_auth_method() {
let v1 = Negotiated::from_initialize(&json!({
"protocolVersion": 1,
"authMethods": [
{ "id": "cli_login", "_meta": { "terminal-auth": { "command": "copilot", "args": ["login"] } } },
{ "id": "terminal_login", "type": "terminal", "args": ["login"] },
{ "id": "api_key", "type": "env_var" },
{ "id": "github_oauth", "name": "GitHub OAuth" }
]
}));
assert_eq!(v1.agent_auth_method(), Some("github_oauth"));
let v2 = Negotiated::from_initialize(&json!({
"protocolVersion": 2,
"info": { "name": "x", "version": "1" },
"authMethods": [
{ "type": "terminal", "methodId": "cli", "name": "CLI", "args": ["login"] },
{ "type": "agent", "methodId": "oauth", "name": "OAuth" }
]
}));
assert_eq!(v2.agent_auth_method(), Some("oauth"));
let terminal_only = Negotiated::from_initialize(&json!({
"protocolVersion": 1,
"authMethods": [{ "id": "cli", "type": "terminal", "args": ["login"] }]
}));
assert_eq!(terminal_only.agent_auth_method(), None);
}
#[test]
fn meta_and_auth_methods_pass_through() {
// The native shims advertise their extensions in `agentCapabilities._meta`.
+24 -1
Переглянути файл
@@ -107,7 +107,15 @@ pub(crate) fn parse_options(options: &Value) -> Vec<AgentConfigOption> {
Some(AgentConfigOption {
name: str_of(o, "name").unwrap_or_else(|| id.clone()),
description: str_of(o, "description"),
category: str_of(o, "category").or_else(|| matches!(id.as_str(), "model" | "mode").then(|| id.clone())),
category: str_of(o, "category").or_else(|| {
matches!(id.as_str(), "model" | "mode")
.then(|| id.clone())
.or_else(|| {
matches!(id.as_str(), "effort" | "reasoning_effort" | "thinking" | "reasoning")
.then(|| "thought_level".into())
})
.or_else(|| matches!(id.as_str(), "fast").then(|| "model_config".into()))
}),
current: str_of(o, "currentValue"),
values,
via: AgentConfigVia::Option,
@@ -333,6 +341,21 @@ mod tests {
assert!(bypass_approvals(&url_mode));
}
#[test]
fn effort_and_fast_categories_are_inferred_from_ids() {
let c = options_of(&json!({
"configOptions": [
{"id": "effort", "type": "select", "currentValue": "high",
"options": [{"value": "low"}, {"value": "high"}]},
{"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}
]
}));
assert_eq!(c.len(), 2);
assert_eq!(c[0].category.as_deref(), Some("thought_level"));
assert_eq!(c[1].category.as_deref(), Some("model_config"));
}
#[test]
fn command_sources_in_updates_and_stored_tasks() {
let update = json!({"availableCommands": [
+325
Переглянути файл
@@ -0,0 +1,325 @@
//! Sessions the user started outside OpenIDEA (a terminal, another editor, the CLI):
//! scanned so they can be listed and imported as tasks instead of re-explained (#312).
//!
//! Only the native providers keep transcripts on disk in a readable form: Claude Code
//! (`<config>/projects/*/<id>.jsonl`, see `native::claude::history`) and Codex
//! (`<home>/.codex/sessions/**`, rollout files with a `session_meta` line).
use super::native::claude::history;
use serde::Serialize;
use specta::Type;
use std::io::BufRead;
use std::path::{Path, PathBuf};
/// A session on this machine that no task holds yet.
#[derive(Clone, Debug, Serialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct ExternalSession {
/// The [`crate::AgentProvider`] id that owns it (`claude`, `codex`).
pub provider: String,
/// The id resuming it names (`--resume`, `thread/resume`).
pub session_id: String,
pub title: String,
/// Where it ran, when the transcript names one.
pub cwd: Option<String>,
/// File modification time, ms since the epoch.
pub updated_ms: f64,
}
/// Modification time of `path` in ms, else 0.
fn mtime(path: &Path) -> f64 {
path.metadata()
.and_then(|m| m.modified())
.ok()
.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok())
.map(|d| d.as_millis() as f64)
.unwrap_or(0.0)
}
/// Short display text: collapsed whitespace, truncated to `MAX_TITLE` chars.
fn short(text: &str) -> String {
const MAX_TITLE: usize = 80;
let flat: String = text.split_whitespace().collect::<Vec<_>>().join(" ");
if flat.chars().count() <= MAX_TITLE {
flat
} else {
format!("{}…", flat.chars().take(MAX_TITLE - 1).collect::<String>())
}
}
/// The text of a Claude Code transcript entry's message, when it carries prose.
fn message_text(e: &serde_json::Value) -> Option<String> {
match e.pointer("/message/content") {
Some(serde_json::Value::String(s)) => Some(s.clone()),
Some(serde_json::Value::Array(blocks)) => {
let text: Vec<&str> = blocks
.iter()
.filter(|b| b.get("type").and_then(serde_json::Value::as_str) == Some("text"))
.filter_map(|b| b.get("text").and_then(serde_json::Value::as_str))
.collect();
(!text.is_empty()).then(|| text.join("\n"))
}
_ => None,
}
}
/// Whether a session that ran in `cwd` belongs to the project at `root` (in it or below it).
fn in_project(cwd: Option<&str>, root: &Path) -> bool {
cwd.is_some_and(|cwd| Path::new(cwd).starts_with(root))
}
/// Claude Code transcripts of the project at `root`: `<config>/projects/<escaped cwd>/<id>.jsonl`.
/// Only the folders named after `root` or a folder below it are read, so other projects'
/// transcripts (often gigabytes) are never opened.
pub fn claude_sessions(config_dir: &Path, root: &Path) -> Vec<ExternalSession> {
let mut out = Vec::new();
let projects = config_dir.join("projects");
let escaped = history::escape(root);
let dirs: Vec<PathBuf> = std::fs::read_dir(&projects)
.into_iter()
.flatten()
.flatten()
.filter(|d| {
let name = d.file_name().to_string_lossy().into_owned();
name == escaped || name.starts_with(&format!("{escaped}-"))
})
.map(|d| d.path())
.collect();
for dir in dirs {
let files: Vec<PathBuf> = std::fs::read_dir(&dir)
.into_iter()
.flatten()
.flatten()
.map(|d| d.path())
.filter(|p| p.extension().is_some_and(|e| e == "jsonl"))
.collect();
for path in files {
let id = path.file_stem().and_then(|s| s.to_str()).unwrap_or_default().to_owned();
if id.is_empty() {
continue;
}
let entries = history::read(&path).unwrap_or_default();
if entries.is_empty() {
continue;
}
let title = history::title(&entries)
.or_else(|| entries.iter().filter(|e| history::is_turn_start(e)).filter_map(message_text).next())
.map(|t| short(&t))
.unwrap_or_else(|| format!("Claude Code session {}", &id[..id.len().min(8)]));
let cwd = entries
.iter()
.filter_map(|e| e.get("cwd").and_then(serde_json::Value::as_str))
.next()
.map(str::to_owned);
// Escaping is lossy (`a/b-c` and `a/b/c` share a folder): the transcript decides.
if !in_project(cwd.as_deref(), root) {
continue;
}
out.push(ExternalSession {
provider: "claude".to_owned(),
session_id: id,
title,
cwd,
updated_ms: mtime(&path),
});
}
}
out
}
/// The `session_meta` line of a Codex rollout file: `(id, cwd)`, across shapes.
fn codex_meta(line: &serde_json::Value) -> Option<(String, Option<String>)> {
let payload =
line.get("payload").filter(|p| p.get("type").and_then(serde_json::Value::as_str) == Some("session_meta"));
let inner = payload.and_then(|p| p.get("payload")).or_else(|| {
line.get("type").and_then(serde_json::Value::as_str).filter(|t| *t == "session_meta").and(Some(line))
})?;
let id = inner.get("id").and_then(serde_json::Value::as_str)?.to_owned();
let cwd = inner.get("cwd").and_then(serde_json::Value::as_str).map(str::to_owned);
Some((id, cwd))
}
/// A prose line of a Codex rollout file (a user message), across shapes.
fn codex_user_text(line: &serde_json::Value) -> Option<String> {
let payload = line.get("payload")?;
let ty = payload.get("type").and_then(serde_json::Value::as_str)?;
if !(ty == "user_message" || ty == "userMessage" || ty == "event_msg") {
return None;
}
let inner = payload.get("payload").unwrap_or(&serde_json::Value::Null);
for v in [payload, inner] {
if let Some(s) = v.get("message").and_then(serde_json::Value::as_str) {
return Some(s.to_owned());
}
if let Some(s) = v.get("text").and_then(serde_json::Value::as_str) {
return Some(s.to_owned());
}
}
None
}
/// One rollout file's session, or `None` when it names no `session_meta`.
fn codex_file(path: &Path) -> Option<ExternalSession> {
let file = std::fs::File::open(path).ok()?;
let mut id: Option<(String, Option<String>)> = None;
let mut title: Option<String> = None;
// `session_meta` is first, but cap the scan: giant rollouts must not stall listing.
for line in std::io::BufReader::new(file).lines().take(400) {
let Ok(line) = line else { break };
let Ok(v) = serde_json::from_str::<serde_json::Value>(&line) else { continue };
if id.is_none()
&& let Some(meta) = codex_meta(&v)
{
id = Some(meta);
}
if title.is_none()
&& let Some(t) = codex_user_text(&v)
{
title = Some(short(&t));
}
if id.is_some() && title.is_some() {
break;
}
}
let (session_id, cwd) = id?;
Some(ExternalSession {
provider: "codex".to_owned(),
title: title.unwrap_or_else(|| format!("Codex session {}", &session_id[..session_id.len().min(8)])),
session_id,
cwd,
updated_ms: mtime(path),
})
}
/// Collects `*.jsonl` below `dir`, up to 4 levels (`sessions/YYYY/MM/DD/<rollout>`).
fn rollout_files(dir: &Path, depth: usize, out: &mut Vec<PathBuf>) {
if depth == 0 {
return;
}
for entry in std::fs::read_dir(dir).into_iter().flatten().flatten() {
let path = entry.path();
if path.is_dir() {
rollout_files(&path, depth - 1, out);
} else if path.extension().is_some_and(|e| e == "jsonl") {
out.push(path);
}
}
}
/// Codex sessions of the project at `root`: `<home>/.codex/sessions/**`.
pub fn codex_sessions(codex_home: &Path, root: &Path) -> Vec<ExternalSession> {
let mut files = Vec::new();
rollout_files(&codex_home.join("sessions"), 4, &mut files);
files.iter().filter_map(|p| codex_file(p)).filter(|s| in_project(s.cwd.as_deref(), root)).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use std::io::Write;
fn write(path: &Path, lines: &[serde_json::Value]) {
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let mut f = std::fs::File::create(path).unwrap();
for line in lines {
writeln!(f, "{line}").unwrap();
}
}
fn claude_user(uuid: &str, text: &str) -> serde_json::Value {
json!({"type": "user", "uuid": uuid, "parentUuid": null,
"message": {"role": "user", "content": text}, "cwd": "/work/app"})
}
#[test]
fn lists_claude_transcripts_with_titles() {
let tmp = tempfile::tempdir().unwrap();
write(
&tmp.path().join("projects/-work-app/abc123.jsonl"),
&[
claude_user("u1", "fix the login bug"),
json!({"type": "assistant", "uuid": "a1", "parentUuid": "u1",
"message": {"role": "assistant", "content": []}}),
json!({"type": "ai-title", "aiTitle": "Login fix"}),
],
);
write(&tmp.path().join("projects/-work-app/empty.jsonl"), &[]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("claude", "abc123"));
assert_eq!(s.title, "Login fix");
assert_eq!(s.cwd.as_deref(), Some("/work/app"));
assert!(s.updated_ms > 0.0);
}
#[test]
fn claude_title_falls_back_to_the_first_prompt() {
let tmp = tempfile::tempdir().unwrap();
write(&tmp.path().join("projects/-work-app/def456.jsonl"), &[claude_user("u1", "hello there")]);
let sessions = claude_sessions(tmp.path(), Path::new("/work/app"));
assert_eq!(sessions.len(), 1);
assert_eq!(sessions[0].title, "hello there");
}
#[test]
fn lists_codex_rollouts_with_meta() {
let tmp = tempfile::tempdir().unwrap();
write(
&tmp.path().join("sessions/2026/01/02/rollout-01.jsonl"),
&[
json!({"timestamp": "t", "payload": {"type": "session_meta",
"payload": {"id": "thread-1", "cwd": "/work/app"}}}),
json!({"timestamp": "t", "payload": {"type": "user_message",
"payload": {"message": "add tests"}}}),
],
);
write(&tmp.path().join("sessions/notes.jsonl"), &[json!({"nope": true})]);
let sessions = codex_sessions(tmp.path(), Path::new("/work"));
assert_eq!(sessions.len(), 1);
let s = &sessions[0];
assert_eq!((s.provider.as_str(), s.session_id.as_str()), ("codex", "thread-1"));
assert_eq!(s.title, "add tests");
assert_eq!(s.cwd.as_deref(), Some("/work/app"));
}
#[test]
fn missing_directories_list_nothing() {
let tmp = tempfile::tempdir().unwrap();
assert!(claude_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
assert!(codex_sessions(&tmp.path().join("nope"), Path::new("/work")).is_empty());
}
#[test]
fn only_sessions_of_the_project_are_listed() {
let tmp = tempfile::tempdir().unwrap();
let user = |cwd: &str| {
json!({"type": "user", "uuid": "u1", "parentUuid": null,
"message": {"role": "user", "content": "hi"}, "cwd": cwd})
};
write(&tmp.path().join("projects/-work-app/in-root.jsonl"), &[user("/work/app")]);
write(&tmp.path().join("projects/-work-app-web/below.jsonl"), &[user("/work/app/web")]);
// `/work/app-old` escapes like a folder below `/work/app`; its transcript says otherwise.
write(&tmp.path().join("projects/-work-app-old/sibling.jsonl"), &[user("/work/app-old")]);
write(&tmp.path().join("projects/-other/elsewhere.jsonl"), &[user("/other")]);
let mut ids: Vec<String> =
claude_sessions(tmp.path(), Path::new("/work/app")).into_iter().map(|s| s.session_id).collect();
ids.sort();
assert_eq!(ids, ["below", "in-root"]);
let rollout = |name: &str, cwd: &str| {
write(
&tmp.path().join(format!("codex/sessions/2026/01/02/{name}.jsonl")),
&[json!({"payload": {"type": "session_meta", "payload": {"id": name, "cwd": cwd}}})],
)
};
rollout("mine", "/work/app/api");
rollout("theirs", "/other");
let codex: Vec<String> = codex_sessions(&tmp.path().join("codex"), Path::new("/work/app"))
.into_iter()
.map(|s| s.session_id)
.collect();
assert_eq!(codex, ["mine"]);
}
}
+330 -44
Переглянути файл
@@ -10,15 +10,18 @@
mod acp;
mod config;
mod conversation;
mod external;
pub mod mcp;
mod model;
mod native;
mod provider;
pub mod registry;
pub mod security;
mod skills;
mod timeline;
mod worktree;
pub use external::ExternalSession;
pub use model::*;
use conversation::{Conversation, SavedSubagent, Session};
@@ -37,6 +40,9 @@ use std::time::Duration;
/// Receives the events of one project's tasks.
pub type AgentSink = Arc<dyn Fn(AgentEvent) + Send + Sync>;
/// Returns the active command execution security policy for the project.
pub type SecurityHook = Arc<dyn Fn() -> ide_core::settings::AgentSecuritySettings + Send + Sync>;
/// Entry changes are batched for this long before they are sent.
const FLUSH_DELAY: Duration = Duration::from_millis(40);
/// Updates that arrive this late after `session/load` returns are still replayed history.
@@ -100,6 +106,7 @@ struct Shared {
runtime: tokio::runtime::Handle,
connector: Connector,
sink: RwLock<Option<AgentSink>>,
security: RwLock<Option<SecurityHook>>,
known: Mutex<HashMap<String, Known>>,
providers: RwLock<Vec<provider::ProviderSpec>>,
/// Last probe of each provider.
@@ -175,9 +182,15 @@ impl Shared {
let shared = self.clone();
// Keep the lookup alive if a composer unmounts while it is waiting.
self.runtime.spawn(async move {
if let Ok(found) =
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await
{
tracing::info!("agent {}: settings discovery started", spec.id);
let discovered = if spec.driver == provider::Driver::Acp {
acp::discover::run(&shared, &spec).await.map(|_| None)
} else if spec.driver == provider::Driver::Claude {
native::claude::discover(spec.clone(), shared.paths.project_root.clone()).await.map(Some)
} else {
return;
};
if let Ok(Some(found)) = discovered {
let commands =
config::commands_of(&json!({ "availableCommands": found.commands }), &spec.id);
let options = config::options_of(&json!({ "configOptions": found.options }));
@@ -196,6 +209,8 @@ impl Shared {
shared.remember_discovered(&spec.id, &options, &commands, &levels);
})
.await;
} else if let Err(e) = discovered {
tracing::info!("agent {}: settings discovery failed: {e}", spec.id);
}
let _ = done.send(true);
});
@@ -218,12 +233,17 @@ impl Shared {
fn knows_all_levels(&self, provider: &str) -> bool {
let cache = self.known.lock();
let Some(known) = cache.get(provider) else { return false };
known
let models: Vec<&str> = known
.config
.iter()
.filter(|o| o.category.as_deref() == Some("model"))
.flat_map(|o| &o.values)
.all(|v| known.thought_levels.contains_key(&v.value))
.flat_map(|o| o.values.iter().map(|v| v.value.as_str()))
.collect();
// No advertised models yet: nothing to attach effort levels to.
if models.is_empty() {
return false;
}
models.iter().all(|id| known.thought_levels.contains_key(*id))
}
fn root_key(&self) -> String {
@@ -432,6 +452,7 @@ impl AgentManager {
runtime,
connector,
sink: RwLock::new(None),
security: RwLock::new(None),
known: Mutex::new(known),
providers,
probes: Mutex::new(HashMap::new()),
@@ -475,6 +496,11 @@ impl AgentManager {
tasks
}
/// Sets the callback used to read the active command execution security policy.
pub fn set_security_hook(&self, hook: SecurityHook) {
*self.shared.security.write() = Some(hook);
}
/// The agents (custom ones re-read), whether each is installed, and what they offered last time.
pub fn providers(&self) -> Vec<AgentProvider> {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -499,7 +525,9 @@ impl AgentManager {
let new = specs.iter().find(|p| &p.id == id);
matches!((old, new), (Some(a), Some(b)) if a.command == b.command && a.args == b.args && a.env == b.env)
});
*prev = specs;
*prev = specs.clone();
drop(probes);
drop(prev);
list
}
@@ -520,6 +548,30 @@ impl AgentManager {
provider::set_builtin_env(&dir, id, env)
}
/// Saves a new custom ACP agent through its agent file.
pub fn save_custom_agent(&self, input: CustomAgentInput) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::save_custom(&dir, &input)
}
/// Removes a custom agent's file (registry installs go through `registry_remove`).
pub fn remove_custom_agent(&self, id: &str) -> IdeResult<()> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::remove_custom(&dir, id)
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom agents.
pub fn import_acp_agents(&self, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::import_acp(&dir, text)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
@@ -550,10 +602,16 @@ impl AgentManager {
let mut offered = offered;
if task.is_none()
&& (offered.is_empty() || !self.shared.has_config(&provider) || !self.shared.knows_all_levels(&provider))
&& let Some(spec) = self.shared.spec(&provider).filter(|s| s.driver == provider::Driver::Claude)
&& let Some(spec) = self
.shared
.spec(&provider)
.filter(|s| matches!(s.driver, provider::Driver::Claude | provider::Driver::Acp))
{
let claude = spec.driver == provider::Driver::Claude;
self.shared.fetch_settings(spec).await;
offered = self.shared.known_commands(&provider);
if claude {
offered = self.shared.known_commands(&provider);
}
}
for &(name, description) in builtin_commands(&provider) {
if !offered.iter().any(|c| c.name == name) {
@@ -585,6 +643,93 @@ impl AgentManager {
Ok(timeline.entries.clone())
}
/// Session ids the tasks already hold (native drivers reuse the CLI's own id).
fn imported_session_ids(&self) -> HashSet<String> {
self.tasks.read().values().filter_map(|t| t.state.lock().session_id.clone()).collect()
}
/// Sessions of this project on this machine that no task holds yet, newest first. Reads
/// transcripts from disk: call it off the UI and command threads.
pub fn external_sessions(&self) -> Vec<ExternalSession> {
let mut out = Vec::new();
let root = self.shared.paths.project_root.clone();
if let Some(spec) = self.shared.spec("claude")
&& let Some(dir) = native::claude::cli::config_dir(&spec.env)
{
out.extend(external::claude_sessions(&dir, &root));
}
let home = self.shared.paths.home_dir.clone().or_else(home_dir);
if let Some(home) = home {
out.extend(external::codex_sessions(&home.join(".codex"), &root));
}
let known = self.imported_session_ids();
out.retain(|s| !known.contains(&s.session_id));
out.sort_by(|a, b| b.updated_ms.total_cmp(&a.updated_ms));
out.truncate(50);
out
}
/// Imports an external session as a task: its first turn loads and replays it (#312).
pub fn import_session(&self, provider: &str, session_id: &str) -> IdeResult<AgentTask> {
if !matches!(provider, "claude" | "codex") {
return Err(IdeError::invalid(format!("cannot import {provider} sessions")));
}
let spec = self.shared.spec(provider).ok_or_else(|| IdeError::invalid(format!("unknown agent {provider}")))?;
if self.imported_session_ids().contains(session_id) {
return Err(IdeError::invalid("that session is already a task"));
}
let found = self
.external_sessions()
.into_iter()
.find(|s| s.provider == provider && s.session_id == session_id)
.ok_or_else(|| IdeError::not_found(format!("unknown {provider} session {session_id}")))?;
let root = self.shared.paths.project_root.clone();
let cwd = found.cwd.map(PathBuf::from).filter(|p| p.is_absolute()).unwrap_or_else(|| root.clone());
let config = self.shared.known_config(&spec.id);
let now = now_ms();
let id = uuid::Uuid::new_v4().simple().to_string();
let task = AgentTask {
project_root: Some(IpcPath(root)),
id,
title: found.title,
provider: spec.id.clone(),
model: model_of(&config),
status: AgentStatus::Ready,
isolation: Isolation::Local,
cwd: IpcPath(cwd),
branch: None,
base: None,
base_commit: None,
error: None,
created_ms: now,
updated_ms: now,
config,
commands: self.shared.known_commands(&spec.id),
usage: None,
queue: Vec::new(),
capabilities: self.shared.probes.lock().get(&spec.id).map(|p| p.capabilities).unwrap_or_default(),
resets_ms: None,
retry: None,
background: Box::default(),
};
let file = TaskFile {
task: task.clone(),
session_id: Some(session_id.to_owned()),
entries: Vec::new(),
wanted: Vec::new(),
before: BTreeMap::new(),
saved: None,
subagents: Vec::new(),
pending_rollback: 0,
title_renamed: true,
};
let handle = Task::restore(file, self.shared.clone());
self.tasks.write().insert(task.id.clone(), handle.clone());
handle.emit_task();
handle.persist();
Ok(task)
}
/// Creates the task (and its worktree) and sends `req.prompt` as its first message.
pub async fn create(&self, req: NewAgentTask) -> IdeResult<AgentTask> {
let spec = self
@@ -799,14 +944,38 @@ impl AgentManager {
}
o.clone()
};
let previous = target.current.clone();
// Cursor's `session/set_config_option` (Fast, effort) can take seconds. Show the
// choice first so the switch and slider move immediately.
apply_config_choice(&task, option, &value);
task.persist();
task.emit_task();
let live = task.live.lock().clone().filter(|l| !l.peer.is_closed());
let mut model_options =
target.category.as_deref() != Some("model") || target.current.as_deref() == Some(value.as_str());
target.category.as_deref() != Some("model") || previous.as_deref() == Some(value.as_str());
if let Some(live) = live.filter(|_| option != config::CURSOR_APPROVAL) {
let (method, mut params) = config::set_request(&target, &value);
params["sessionId"] = json!(live.session);
let reply: Value =
live.peer.request(method, params, None).await.map_err(|e| IdeError::internal(task.describe(e)))?;
let reply: Value = match live.peer.request(method, params, None).await {
Ok(reply) => reply,
Err(e) => {
{
let mut s = task.state.lock();
s.wanted.retain(|c| c.id != option);
if let Some(prev) = previous.clone() {
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: prev });
}
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = previous.clone();
}
s.task.model = model_of(&s.task.config).or(s.task.model.take());
}
task.persist();
task.emit_task();
return Err(IdeError::internal(task.describe(e)));
}
};
if let Some(options) = reply.get("configOptions") {
config::merge_update(&mut task.state.lock().task.config, options);
model_options = true;
@@ -817,8 +986,6 @@ impl AgentManager {
if !model_options {
s.task.config.retain(|o| o.category.as_deref() != Some("thought_level"));
}
s.wanted.retain(|c| c.id != option);
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: value.clone() });
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = Some(value);
}
@@ -1145,15 +1312,6 @@ impl AgentManager {
};
// A copy of the task (a fork) may work in the same worktree.
let shared = self.tasks.read().values().any(|t| t.id != id && t.state.lock().task.cwd.0 == cwd);
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
if let Some(peer) = live {
let _ = tokio::time::timeout(EXIT_WAIT, peer.exited()).await;
}
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
self.tasks.write().remove(id);
*task.deleted.lock() = true;
let _ = std::fs::remove_file(self.shared.paths.store_dir.join(format!("{id}.json")));
@@ -1163,22 +1321,36 @@ impl AgentManager {
{
let _ = std::fs::remove_dir_all(self.shared.paths.store_dir.join(saved));
}
// The list drops the task now; removing a large worktree's folder can take a while.
self.shared.emit(AgentEvent::Removed { id: id.to_owned() });
if remove_worktree && !shared && isolation == Isolation::Worktree && cwd.exists() {
// On Windows the agent's processes lock the folder (their working directory,
// open files) until they have exited.
if let Some(peer) = live {
let _ = tokio::time::timeout(EXIT_WAIT, peer.exited()).await;
}
let worktrees = self.shared.paths.worktrees_dir.clone();
blocking(move || worktree::remove(&root, &worktrees, &cwd)).await?;
}
Ok(())
}
/// Files the task changed since it started.
pub async fn changes(&self, id: &str) -> IdeResult<Vec<AgentChange>> {
let task = self.task(id)?;
let (cwd, base) = task.cwd_and_base();
let at = task.base();
let before = task.state.lock().before.clone();
blocking(move || Ok(worktree::without_untouched(worktree::changes(&cwd, &base)?, &before))).await
blocking(move || at.changes(&before)).await
}
/// Content of `path` (in the task's directory) when the task started.
pub async fn base_text(&self, id: &str, path: PathBuf) -> IdeResult<Option<String>> {
let (cwd, base) = self.task(id)?.cwd_and_base();
blocking(move || worktree::file_at(&cwd, &path, &base)).await
let at = self.task(id)?.base();
blocking(move || match at.rev()? {
Some(rev) => worktree::file_at(at.cwd(), &path, &rev),
None => Ok(None),
})
.await
}
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as they
@@ -1188,16 +1360,17 @@ impl AgentManager {
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before rolling back its changes"));
}
let (cwd, base) = task.cwd_and_base();
let at = task.base();
let (before, saved) = {
let s = task.state.lock();
(s.before.clone(), s.saved.as_ref().map(|name| self.shared.paths.store_dir.join(name)))
};
let restamped = blocking(move || {
let changes = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before);
let changes = at.changes(&before)?;
let rev = at.rev()?;
let mut restamped = Vec::new();
for change in changes.iter().filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path))) {
worktree::restore(&cwd, &base, change, &before, saved.as_deref())?;
worktree::restore(at.cwd(), rev.as_deref(), change, &before, saved.as_deref())?;
// Back to the user's own edits: not a change of the task any more.
if before.contains_key(&change.rel_path) {
restamped.push((change.rel_path.clone(), worktree::stamp(&change.path.0)));
@@ -1216,6 +1389,41 @@ impl AgentManager {
Ok(())
}
/// Accept: copies the task's changed files among `rel_paths` (all with `None`) from its
/// worktree into the local checkout, uncommitted, for the user to review and commit.
/// Files the checkout changed itself are left alone and reported. Only worktree tasks
/// have anything to accept.
pub async fn accept_changes(&self, id: &str, rel_paths: Option<Vec<String>>) -> IdeResult<AgentAcceptOutcome> {
let task = self.task(id)?;
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before accepting its changes"));
}
let (isolation, cwd, base, root, before) = {
let s = task.state.lock();
(
s.task.isolation,
s.task.cwd.0.clone(),
s.task.base_commit.clone(),
s.task.project_root.clone().map(|p| p.0).unwrap_or_else(|| self.shared.paths.project_root.clone()),
(*s.before).clone(),
)
};
if isolation != Isolation::Worktree {
return Err(IdeError::invalid("the task works in the local checkout: there is nothing to accept"));
}
let Some(base) = base else {
return Err(IdeError::invalid("the task has no base revision to accept from"));
};
blocking(move || {
let wanted: Vec<AgentChange> = worktree::without_untouched(worktree::changes(&cwd, &base)?, &before)
.into_iter()
.filter(|c| rel_paths.as_ref().is_none_or(|p| p.contains(&c.rel_path)))
.collect();
worktree::accept(&root, &cwd, &base, &wanted)
})
.await
}
/// Stops every agent (project closed).
pub fn shutdown(&self) {
for task in self.tasks.read().values() {
@@ -1266,15 +1474,15 @@ impl Task {
})
}
fn cwd_and_base(&self) -> (PathBuf, String) {
fn base(&self) -> worktree::TaskBase {
let s = self.state.lock();
// In the project's own checkout the user commits too: compare against the current HEAD,
// as the Commit tool window does, so commits made since the task started drop out.
let base = match s.task.isolation {
Isolation::Local => None,
Isolation::Worktree => s.task.base_commit.clone(),
};
(s.task.cwd.0.clone(), base.unwrap_or_else(|| "HEAD".into()))
let cwd = s.task.cwd.0.clone();
match s.task.isolation {
Isolation::Local => worktree::TaskBase::Local { cwd, start: s.task.base_commit.clone() },
Isolation::Worktree => {
worktree::TaskBase::Worktree { cwd, base: s.task.base_commit.clone().unwrap_or_else(|| "HEAD".into()) }
}
}
}
fn emit_task(&self) {
@@ -1623,6 +1831,30 @@ impl Task {
})
}
/// Runs `method` and, on an auth error, tries the agent's own ACP `authenticate` once before retrying.
async fn request_with_auth(
&self,
peer: &acp::Conn,
init: &acp::Negotiated,
tried_auth: &mut bool,
method: &str,
params: Value,
) -> Result<Value, RpcError> {
let first = peer.request(method, params.clone(), None).await;
if !*tried_auth
&& let Err(RpcError::Server { code, ref message, .. }) = first
&& acp::auth_required(code, message)
&& let Some(method_id) = init.agent_auth_method()
{
*tried_auth = true;
match peer.request::<_, Value>("authenticate", json!({ "methodId": method_id }), None).await {
Ok(_) => return peer.request(method, params, None).await,
Err(e) => tracing::debug!(task = %self.id, method_id, error = %e, "agent authenticate failed"),
}
}
first
}
async fn ensure_session(self: &Arc<Self>) -> Result<Live, String> {
let _guard = self.connecting.lock().await;
if let Some(live) = self.live.lock().clone()
@@ -1665,12 +1897,12 @@ impl Task {
// Agents that compare canonical paths fail in a symlinked directory (macOS `/tmp`).
let cwd_s = dunce::canonicalize(&cwd).unwrap_or(cwd).to_string_lossy().into_owned();
let mut tried_auth = false;
let mut session = None;
if let (Some(id), true) = (saved, can_load) {
self.replaying.store(true, Ordering::Release);
let loaded: Result<Value, RpcError> = peer
.request("session/load", json!({ "sessionId": id, "cwd": cwd_s, "mcpServers": mcp_servers }), None)
.await;
let params = json!({ "sessionId": &id, "cwd": &cwd_s, "mcpServers": &mcp_servers });
let loaded = self.request_with_auth(&peer, &init, &mut tried_auth, "session/load", params).await;
tokio::time::sleep(REPLAY_GRACE).await;
self.replaying.store(false, Ordering::Release);
match loaded {
@@ -1682,8 +1914,9 @@ impl Task {
let (session_id, result) = match session {
Some(s) => s,
None => {
let v: Value = peer
.request("session/new", json!({ "cwd": cwd_s, "mcpServers": mcp_servers }), None)
let params = json!({ "cwd": &cwd_s, "mcpServers": &mcp_servers });
let v = self
.request_with_auth(&peer, &init, &mut tried_auth, "session/new", params)
.await
.map_err(|e| self.sign_in_error(&e, Some(&sign_in_message)).unwrap_or_else(|| self.describe(e)))?;
let id = v.get("sessionId").and_then(Value::as_str).ok_or("the agent returned no session")?.to_owned();
@@ -2009,6 +2242,44 @@ impl Task {
.iter()
.map(|question| question.answer_key.clone().unwrap_or_else(|| question.question.clone()))
.collect();
if let Some(hook) = self.shared.security.read().clone()
&& let Some(command) = security::extract_command(params)
{
match security::evaluate_command(&command, &hook()) {
security::PolicyDecision::Blocked(pattern) => {
let reject = options.iter().find(|o| o.kind.starts_with("reject")).cloned();
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
options,
questions: Vec::new(),
chosen: Some(format!("Blocked by policy ({pattern})")),
});
return match reject {
Some(option) => json!({ "outcome": { "outcome": "selected", "optionId": option.id } }),
None => json!({ "outcome": { "outcome": "cancelled" } }),
};
}
security::PolicyDecision::Allowed => {
if let Some(allow) = options
.iter()
.find(|o| o.kind == "allow_once")
.or_else(|| options.iter().find(|o| o.kind.starts_with("allow")))
.cloned()
{
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
options,
questions: Vec::new(),
chosen: Some(format!("{} (whitelisted)", allow.name)),
});
return json!({ "outcome": { "outcome": "selected", "optionId": allow.id } });
}
}
security::PolicyDecision::AskUser => {}
}
}
let (tx, rx) = std::sync::mpsc::channel();
let entry =
self.push_entry(AgentEntry::Permission { id, title, options: options.clone(), questions, chosen: None });
@@ -2322,7 +2593,7 @@ fn with_details(message: String, data: Option<&Value>) -> String {
}
}
fn client_info() -> Value {
pub(crate) fn client_info() -> Value {
json!({ "name": "OpenIDEA", "version": env!("CARGO_PKG_VERSION") })
}
@@ -2381,6 +2652,16 @@ fn retry_of(update: &Value, now: f64) -> Option<AgentRetry> {
})
}
fn apply_config_choice(task: &Task, option: &str, value: &str) {
let mut s = task.state.lock();
s.wanted.retain(|c| c.id != option);
s.wanted.push(AgentConfigChoice { id: option.to_owned(), value: value.to_owned() });
if let Some(o) = s.task.config.iter_mut().find(|o| o.id == option) {
o.current = Some(value.to_owned());
}
s.task.model = model_of(&s.task.config).or(s.task.model.take());
}
fn model_of(config: &[AgentConfigOption]) -> Option<String> {
config.iter().find(|o| o.category.as_deref() == Some("model")).and_then(|o| o.current.clone())
}
@@ -2429,6 +2710,11 @@ fn now_ms() -> f64 {
std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0.0, |d| d.as_millis() as f64)
}
/// The user's home, for tools that keep state there (Codex sessions).
fn home_dir() -> Option<PathBuf> {
std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE")).map(PathBuf::from)
}
fn write_file(path: &Path, bytes: &[u8]) -> Result<(), String> {
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
+46
Переглянути файл
@@ -602,3 +602,49 @@ pub struct McpServerInput {
fn default_true() -> bool {
true
}
/// Payload for creating a custom ACP agent (always the ACP driver); the id
/// becomes the `<config>/agents/<id>.toml` file name.
#[derive(Clone, Debug, Deserialize, Type)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CustomAgentInput {
pub id: String,
pub name: String,
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: BTreeMap<String, String>,
}
/// What importing one entry of a JetBrains `acp.json` file did.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AcpImportOutcome {
/// The `agent_servers` key the entry was imported from.
pub name: String,
/// The id its agent file would get (also when skipped).
pub id: String,
pub saved: bool,
/// Why the entry was skipped; empty when it was saved.
#[serde(default)]
pub reason: String,
}
/// What accepting a worktree task's changes copied into the local checkout.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptOutcome {
/// Relative paths copied into the checkout (uncommitted), including ones already identical.
pub accepted: Vec<String>,
/// Files left alone because the checkout has its own edits to them.
pub skipped: Vec<AgentAcceptSkipped>,
}
/// A file accept left alone, with why.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentAcceptSkipped {
pub path: String,
pub reason: String,
}
+1 -1
Переглянути файл
@@ -135,7 +135,7 @@ impl Cli for Process {
}
/// Claude Code's configuration directory (`CLAUDE_CONFIG_DIR`, else `~/.claude`).
pub(super) fn config_dir(env: &[(String, String)]) -> Option<PathBuf> {
pub(crate) fn config_dir(env: &[(String, String)]) -> Option<PathBuf> {
let set = env
.iter()
.find(|(k, _)| k == "CLAUDE_CONFIG_DIR")
+4 -4
Переглянути файл
@@ -31,11 +31,11 @@ pub(super) fn transcript(config_dir: &Path, cwd: &Path, id: &str) -> PathBuf {
}
/// The project directory name Claude Code uses for `cwd`.
pub(super) fn escape(cwd: &Path) -> String {
pub(crate) fn escape(cwd: &Path) -> String {
cwd.to_string_lossy().chars().map(|c| if c.is_ascii_alphanumeric() { c } else { '-' }).collect()
}
pub(super) fn read(path: &Path) -> std::io::Result<Vec<Value>> {
pub(crate) fn read(path: &Path) -> std::io::Result<Vec<Value>> {
let text = std::fs::read_to_string(path)?;
Ok(text.lines().filter_map(|l| serde_json::from_str(l).ok()).collect())
}
@@ -111,7 +111,7 @@ fn block_ids<'a>(e: &'a Value, ty: &'a str, key: &'a str) -> impl Iterator<Item
/// Whether `e` is a message that started a turn: the user's own, not a tool result, a
/// steer, or text Claude Code added.
pub(super) fn is_turn_start(e: &Value) -> bool {
pub(crate) fn is_turn_start(e: &Value) -> bool {
if kind(e) != "user" || e.get("isMeta").and_then(Value::as_bool) == Some(true) {
return false;
}
@@ -149,7 +149,7 @@ pub(super) fn cut(chain: &[&Value], turns: usize) -> Result<Option<String>, Stri
}
/// The newest title Claude Code gave the session.
pub(super) fn title(entries: &[Value]) -> Option<String> {
pub(crate) fn title(entries: &[Value]) -> Option<String> {
entries
.iter()
.rev()
+2 -2
Переглянути файл
@@ -7,8 +7,8 @@
//! <id>.json`): loading that session, from this connection or a fresh one, starts it that
//! way.
mod cli;
mod history;
pub(crate) mod cli;
pub(crate) mod history;
mod settings;
mod tools;
mod translate;
+239 -1
Переглянути файл
@@ -2,7 +2,7 @@
//! `<config>/agents/<id>.toml` (written by hand, or by installing one from the ACP
//! Registry, see `registry`).
use crate::{AgentCommandReadOnlyReason, AgentProvider, AgentSource};
use crate::{AcpImportOutcome, AgentCommandReadOnlyReason, AgentProvider, AgentSource, CustomAgentInput};
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -59,6 +59,8 @@ const BUILTIN: &[(&str, &str, Driver, &[&str], &str)] = &[
("cursor", "Cursor", Driver::Acp, &["cursor-agent", "acp"], "Run `cursor-agent login` in a terminal."),
("opencode", "OpenCode", Driver::Acp, &["opencode", "acp"], "Run `opencode auth login` in a terminal."),
("qwen", "Qwen Code", Driver::Acp, &["qwen", "--experimental-acp"], "Run `qwen` in a terminal and sign in."),
("qoder", "Qoder", Driver::Acp, &["qodercli", "--acp"], "Run `qodercli login` in a terminal."),
("qodercn", "Qoder CN", Driver::Acp, &["qoderclicn", "--acp"], "Run `qoderclicn login` in a terminal."),
("kimi", "Kimi CLI", Driver::Acp, &["kimi", "acp"], "Run `kimi` in a terminal and sign in."),
("goose", "Goose", Driver::Acp, &["goose", "acp"], "Run `goose configure` in a terminal."),
("auggie", "Auggie", Driver::Acp, &["auggie", "--acp"], "Run `auggie login` in a terminal."),
@@ -276,6 +278,127 @@ fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAg
Ok(())
}
/// Saves a new custom agent; refuses built-in and ACP Registry ids, taken ids
/// and an empty command. The id is also the agent file's name, so it must be
/// a safe file stem.
pub(crate) fn save_custom(custom_dir: &Path, input: &CustomAgentInput) -> IdeResult<()> {
let id = validated_custom_id(&input.id)?;
if builtin_spec(&id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent; change its command instead")));
}
if input.command.trim().is_empty() {
return Err(IdeError::invalid("the agent command must not be empty"));
}
let file = custom_dir.join(format!("{id}.toml"));
if file.try_exists()? {
if CustomAgent::read(&file).is_ok_and(|a| a.registry.is_some()) {
return Err(IdeError::invalid(format!(
"{id} is installed from the ACP Registry; uninstall it there to replace it"
)));
}
return Err(IdeError::invalid(format!("there is already a custom agent '{id}'")));
}
CustomAgent {
name: Some(input.name.clone()),
driver: Driver::Acp,
command: input.command.clone(),
args: input.args.clone(),
env: input.env.clone(),
registry: None,
}
.write(&file)?;
Ok(())
}
/// Removes a custom agent's file; refuses built-in, ACP Registry and missing ids.
pub(crate) fn remove_custom(custom_dir: &Path, id: &str) -> IdeResult<()> {
let id = validated_custom_id(id)?;
let id = id.as_str();
if builtin_spec(id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent and cannot be removed")));
}
let file = custom_dir.join(format!("{id}.toml"));
let agent =
CustomAgent::read(&file).map_err(|_| IdeError::not_found(format!("there is no custom agent '{id}'")))?;
if agent.registry.is_some() {
return Err(IdeError::invalid(format!("{id} is installed from the ACP Registry; uninstall it there")));
}
match std::fs::remove_file(&file) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e.into()),
_ => Ok(()),
}
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom
/// agents, one outcome per entry; entries that collide, lack a command or
/// have no usable name are skipped with a reason.
pub(crate) fn import_acp(custom_dir: &Path, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
#[derive(Deserialize)]
struct AcpJson {
#[serde(default)]
agent_servers: BTreeMap<String, AcpServer>,
}
#[derive(Deserialize)]
struct AcpServer {
#[serde(default)]
command: String,
#[serde(default)]
args: Vec<String>,
#[serde(default)]
env: BTreeMap<String, String>,
}
let parsed: AcpJson =
serde_json::from_str(text).map_err(|e| IdeError::invalid(format!("not an acp.json file: {e}")))?;
let mut outcomes = Vec::with_capacity(parsed.agent_servers.len());
for (name, server) in parsed.agent_servers {
let id = slugify_id(&name);
let mut outcome = AcpImportOutcome { name, id: id.clone(), saved: false, reason: String::new() };
if id.is_empty() {
outcome.reason = "the name has no usable characters for an id".to_owned();
} else if builtin_spec(&id).is_some() {
outcome.reason = format!("{id} is a built-in agent");
} else if server.command.trim().is_empty() {
outcome.reason = "the entry has no command".to_owned();
} else if custom_dir.join(format!("{id}.toml")).try_exists()? {
outcome.reason = format!("there is already a custom agent '{id}'");
} else {
CustomAgent {
name: Some(outcome.name.clone()),
driver: Driver::Acp,
command: server.command,
args: server.args,
env: server.env,
registry: None,
}
.write(&custom_dir.join(format!("{id}.toml")))?;
outcome.saved = true;
}
outcomes.push(outcome);
}
Ok(outcomes)
}
/// A custom agent id doubles as a file stem: letters, digits, `-` and `_` only.
fn validated_custom_id(id: &str) -> IdeResult<String> {
if !id.is_empty() && id.chars().all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_') {
return Ok(id.to_owned());
}
Err(IdeError::invalid(format!("'{id}' is not a usable agent id (letters, digits, '-' and '_' only)")))
}
/// Turns an `acp.json` server name into a file-stem id (`"My Agent!"` → `"my-agent"`).
fn slugify_id(name: &str) -> String {
let mut id = String::with_capacity(name.len());
for c in name.chars() {
if c.is_ascii_alphanumeric() {
id.push(c.to_ascii_lowercase());
} else if !id.ends_with('-') {
id.push('-');
}
}
id.trim_matches('-').to_owned()
}
/// An agent file, `<config>/agents/<id>.toml`.
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
@@ -409,6 +532,45 @@ mod tests {
use super::*;
use serde_json::json;
#[test]
fn qoder_global_and_cn_editions_are_both_built_in() {
let info = |id: &str| describe(&load(None).into_iter().find(|s| s.id == id).unwrap());
let (global, cn) = (info("qoder"), info("qodercn"));
assert_eq!((global.name.as_str(), global.command.as_str()), ("Qoder", "qodercli --acp"));
assert_eq!((cn.name.as_str(), cn.command.as_str()), ("Qoder CN", "qoderclicn --acp"));
assert_eq!(global.login_hint.as_deref(), Some("Run `qodercli login` in a terminal."));
}
#[test]
fn qodercn_is_available_without_a_file_and_preserves_existing_customizations() {
let dir = tempfile::tempdir().unwrap();
let qodercn = load(None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(qodercn.driver, Driver::Acp);
assert!(!is_unlisted(&qodercn));
let info = describe(&qodercn);
assert_eq!(info.name, "Qoder CN");
assert_eq!(info.command, "qoderclicn --acp");
assert_eq!(info.source, AgentSource::Builtin);
assert!(info.command_editable);
assert_eq!(info.login_hint.as_deref(), Some("Run `qoderclicn login` in a terminal."));
std::fs::write(
dir.path().join("qodercn.toml"),
"command = \"/custom/qoderclicn\"\nargs = [\"--acp\"]\n[env]\nK = \"v\"\n",
)
.unwrap();
let specs = load(Some(dir.path()));
assert_eq!(specs.iter().filter(|s| s.id == "qodercn").count(), 1);
let custom = specs.iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(custom.command, "/custom/qoderclicn");
assert_eq!(custom.env, [("K".into(), "v".into())]);
set_builtin_command(dir.path(), "qodercn", None).unwrap();
let reset = load(Some(dir.path())).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(reset.command, "qoderclicn");
assert_eq!(reset.args, ["--acp"]);
assert_eq!(reset.env, custom.env);
}
#[test]
fn login_hints_cover_all_builtin_commands_and_preserve_separate_login_tools() {
for spec in builtins() {
@@ -712,4 +874,80 @@ mod tests {
assert_eq!(spec.source, AgentSource::Registry { version: "0.10.0".into(), update: None });
assert!(toml::from_str::<CustomAgent>("command = \"x\"\n[registry]\nid = \"x\"\n").is_err());
}
fn custom_input(id: &str) -> CustomAgentInput {
CustomAgentInput {
id: id.into(),
name: "Mine".into(),
command: "/opt/mine".into(),
args: vec!["acp".into()],
env: BTreeMap::from([("K".to_owned(), "v".to_owned())]),
}
}
#[test]
fn custom_agents_save_list_and_remove() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap();
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
assert_eq!(mine.source, AgentSource::Custom);
remove_custom(dir.path(), "mine").unwrap();
assert!(!load(Some(dir.path())).iter().any(|s| s.id == "mine"));
remove_custom(dir.path(), "mine").unwrap_err();
}
#[test]
fn custom_agents_refuse_builtin_registry_and_bad_ids() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("claude")).unwrap_err();
save_custom(dir.path(), &custom_input("../evil")).unwrap_err();
save_custom(dir.path(), &custom_input("")).unwrap_err();
let mut empty_command = custom_input("mine");
empty_command.command = " ".into();
save_custom(dir.path(), &empty_command).unwrap_err();
save_custom(dir.path(), &custom_input("mine")).unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap_err();
remove_custom(dir.path(), "claude").unwrap_err();
// An id that isn't a plain file stem never reaches outside the agents directory.
let outside = dir.path().join("outside.toml");
std::fs::write(&outside, "command = \"/bin/sh\"\n").unwrap();
let agents = dir.path().join("agents");
std::fs::create_dir(&agents).unwrap();
remove_custom(&agents, "../outside").unwrap_err();
assert!(outside.exists());
std::fs::write(
dir.path().join("amp-acp.toml"),
"command = \"/registry/amp\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n",
)
.unwrap();
save_custom(dir.path(), &custom_input("amp-acp")).unwrap_err();
remove_custom(dir.path(), "amp-acp").unwrap_err();
}
#[test]
fn acp_json_imports_servers_and_skips_collisions() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("taken")).unwrap();
let outcomes = import_acp(
dir.path(),
r#"{"agent_servers": {
"My Agent!": {"command": "/opt/agent", "args": ["--acp"], "env": {"K": "v"}},
"taken": {"command": "/opt/taken"},
"claude": {"command": "/opt/claude"},
"!!!": {"command": "/opt/noname"},
"silent": {"args": []}
}}"#,
)
.unwrap();
let saved: Vec<&str> = outcomes.iter().filter(|o| o.saved).map(|o| o.id.as_str()).collect();
assert_eq!(saved, ["my-agent"]);
assert!(outcomes.iter().filter(|o| !o.saved).all(|o| !o.reason.is_empty()));
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "my-agent").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("My Agent!", "/opt/agent"));
assert_eq!(mine.source, AgentSource::Custom);
import_acp(dir.path(), "not json").unwrap_err();
assert!(import_acp(dir.path(), "{}").unwrap().is_empty());
}
}
+487
Переглянути файл
@@ -0,0 +1,487 @@
//! Command execution security policy (Whitelist / Blacklist) for coding agents.
use ide_core::settings::AgentSecuritySettings;
use serde_json::Value;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum PolicyDecision {
/// Command matched a blacklist pattern (`pattern`).
Blocked(String),
/// Command matched the whitelist and no blacklist pattern.
Allowed,
/// Neither matched; prompt the user as usual.
AskUser,
}
/// Extracts the shell command string from an ACP `session/request_permission` payload
/// when the request is for executing a command.
pub fn extract_command(params: &Value) -> Option<String> {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"))?;
let raw_cmd = tool.pointer("/rawInput/command").or_else(|| tool.pointer("/rawInput/cmd"));
if let Some(cmd) = raw_cmd.and_then(command_from_value) {
return Some(cmd);
}
let kind = tool.get("kind").and_then(Value::as_str).unwrap_or_default();
let name = tool.get("name").and_then(Value::as_str).unwrap_or_default();
let title =
tool.get("title").and_then(Value::as_str).or_else(|| params.get("title").and_then(Value::as_str))?.trim();
if let Some(cmd) = title.strip_prefix("Run ").map(str::trim).filter(|s| !s.is_empty()) {
return Some(cmd.to_owned());
}
if (kind == "execute" || matches!(name, "Bash" | "PowerShell" | "Execute" | "Run" | "terminal"))
&& !title.is_empty()
&& title != "Terminal"
&& title != "Run a command"
{
return Some(title.to_owned());
}
None
}
fn command_from_value(val: &Value) -> Option<String> {
match val {
Value::String(s) => {
let trimmed = s.trim();
(!trimmed.is_empty()).then(|| trimmed.to_owned())
}
Value::Array(items) => {
let words: Vec<&str> = items.iter().filter_map(Value::as_str).collect();
if words.is_empty() {
return None;
}
let cmd = match words.as_slice() {
[_, flag, script] if flag.starts_with('-') && flag.ends_with('c') => (*script).trim().to_owned(),
_ => words.join(" ").trim().to_owned(),
};
(!cmd.is_empty()).then_some(cmd)
}
_ => None,
}
}
/// Checks whether a single pattern matches `command` using prefix + glob rules:
/// - If `pattern` contains `*` or `?`, matches using glob wildcards (where `*` matches
/// any sequence of characters). A trailing ` *` in the pattern also matches the bare
/// command without arguments (e.g. `git status *` matches `git status`).
/// - Otherwise, matches when `command` equals `pattern` or starts with `pattern` followed
/// by whitespace (prefix match at word boundary).
pub fn matches_command(pattern: &str, command: &str) -> bool {
let pat = pattern.trim();
let cmd = command.trim();
if pat.is_empty() || cmd.is_empty() {
return false;
}
if pat.contains('*') || pat.contains('?') {
if let Some(stem) = pat.strip_suffix(" *")
&& cmd == stem.trim()
{
return true;
}
wildcard_match(pat.as_bytes(), cmd.as_bytes())
} else {
cmd == pat || cmd.strip_prefix(pat).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
}
fn wildcard_match(pat: &[u8], text: &[u8]) -> bool {
let (mut p, mut t) = (0, 0);
let (mut star_p, mut star_t) = (None, 0);
while t < text.len() {
if p < pat.len() && (pat[p] == b'?' || pat[p] == text[t]) {
p += 1;
t += 1;
} else if p < pat.len() && pat[p] == b'*' {
star_p = Some(p);
p += 1;
star_t = t;
} else if let Some(sp) = star_p {
p = sp + 1;
star_t += 1;
t = star_t;
} else {
return false;
}
}
while p < pat.len() && pat[p] == b'*' {
p += 1;
}
p == pat.len()
}
/// Matches an allowed command entry against a sub-command.
/// An allow entry is a whole command or a prefix only up to a word boundary:
/// e.g. `git status` matches `git status` and `git status -s`, but NOT `git statusx`.
/// Any accidental trailing `*` in the allow entry is stripped so `git status*` behaves
/// cleanly as a prefix at word boundaries instead of a glob wildcard.
pub fn matches_allow_entry(entry: &str, command: &str) -> bool {
let mut entry = entry.trim();
if let Some(stripped) = entry.strip_suffix('*') {
entry = stripped.trim_end();
}
let cmd = command.trim();
if entry.is_empty() || cmd.is_empty() {
return false;
}
cmd == entry || cmd.strip_prefix(entry).is_some_and(|rest| rest.starts_with(char::is_whitespace))
}
/// Checks whether a word is an environment variable assignment (`IDENTIFIER=...`).
fn is_env_assignment_word(word: &str) -> bool {
let Some((name, _)) = word.split_once('=') else {
return false;
};
if name.is_empty() {
return false;
}
let mut chars = name.chars();
match chars.next() {
Some(c) if c.is_ascii_alphabetic() || c == '_' => {}
_ => return false,
}
chars.all(|c| c.is_ascii_alphanumeric() || c == '_')
}
/// Returns true if the sub-command starts with an environment variable assignment,
/// e.g. `FOO=bar cmd` or `PORT=3000`.
fn has_leading_env_assignment(cmd: &str) -> bool {
let trimmed = cmd.trim_start();
let bytes = trimmed.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
}
i += 1;
continue;
}
if b == b'\'' || b == b'"' {
quote = Some(b);
i += 1;
continue;
}
if b.is_ascii_whitespace() {
break;
}
i += 1;
}
let first_word = &trimmed[..i];
is_env_assignment_word(first_word)
}
/// Checks whether `command` contains shell constructs that must never be auto-approved:
/// - Command substitution: `$(...)` or backticks
/// - Redirection: `>`, `>>`, `<`
/// - Here-docs: `<<`
/// - Subshells: `(...)`
/// - Leading environment assignments: `VAR=val cmd`
pub fn has_unsafe_shell_syntax(command: &str) -> bool {
let cmd = command.trim();
if cmd.is_empty() {
return false;
}
if has_leading_env_assignment(cmd) {
return true;
}
let bytes = cmd.as_bytes();
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if b == b'\\' && quote != Some(b'\'') && i + 1 < bytes.len() {
i += 2;
continue;
}
if let Some(q) = quote {
if b == q {
quote = None;
i += 1;
continue;
}
// Inside double quotes: $(...) and `...` are evaluated by the shell.
if q == b'"' {
if b == b'`' {
return true;
}
if b == b'$' && i + 1 < bytes.len() && bytes[i + 1] == b'(' {
return true;
}
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'`' => return true,
b'$' if i + 1 < bytes.len() && bytes[i + 1] == b'(' => return true,
b'<' | b'>' => return true,
b'(' | b')' => return true,
_ => i += 1,
}
}
false
}
/// Splits a shell command into sub-commands separated by `;`, `&&`, `||`, `|`, or `\n`
/// (outside single/double quotes) so compound commands cannot bypass policy checks.
fn split_subcommands(command: &str) -> Vec<&str> {
let bytes = command.as_bytes();
let mut out = Vec::new();
let mut start = 0;
let mut i = 0;
let mut quote: Option<u8> = None;
while i < bytes.len() {
let b = bytes[i];
if let Some(q) = quote {
if b == b'\\' && q == b'"' && i + 1 < bytes.len() {
i += 2;
continue;
}
if b == q {
quote = None;
}
i += 1;
continue;
}
match b {
b'\'' | b'"' => {
quote = Some(b);
i += 1;
}
b'\\' if i + 1 < bytes.len() => {
i += 2;
}
b';' | b'\n' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
i += 1;
start = i;
}
b'&' | b'|' => {
let seg = command[start..i].trim();
if !seg.is_empty() {
out.push(seg);
}
if i + 1 < bytes.len() && bytes[i + 1] == b {
i += 2;
} else {
i += 1;
}
start = i;
}
_ => i += 1,
}
}
let tail = command[start..].trim();
if !tail.is_empty() {
out.push(tail);
}
out
}
/// Evaluates `command` against `settings`:
/// 1. Blacklist takes precedence: if the full command or any sub-command matches a
/// `blocked_commands` entry, returns [`PolicyDecision::Blocked`].
/// 2. Whitelist: allowed_commands cannot auto-approve when empty.
/// 3. Every sub-command in a chain must be safe (no command substitution, redirection,
/// subshell, or env assignments) and match at least one `allowed_commands` entry
/// exactly or up to word boundary; otherwise falls back to [`PolicyDecision::AskUser`].
pub fn evaluate_command(command: &str, settings: &AgentSecuritySettings) -> PolicyDecision {
let cmd = command.trim();
if cmd.is_empty() {
return PolicyDecision::AskUser;
}
let subcommands = split_subcommands(cmd);
for pat in &settings.blocked_commands {
let pat = pat.trim();
if pat.is_empty() {
continue;
}
if matches_command(pat, cmd) || subcommands.iter().any(|sub| matches_command(pat, sub)) {
return PolicyDecision::Blocked(pat.to_owned());
}
}
if settings.allowed_commands.is_empty() || subcommands.is_empty() {
return PolicyDecision::AskUser;
}
let all_allowed = subcommands.iter().all(|sub| {
let s = sub.trim();
if s.is_empty() || has_unsafe_shell_syntax(s) {
return false;
}
settings.allowed_commands.iter().any(|entry| matches_allow_entry(entry, s))
});
if all_allowed {
return PolicyDecision::Allowed;
}
PolicyDecision::AskUser
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn prefix_and_glob_matching() {
assert!(matches_command("git status", "git status"));
assert!(matches_command("git status", "git status -s"));
assert!(!matches_command("git status", "git statusbar"));
assert!(matches_command("git status*", "git status"));
assert!(matches_command("git status*", "git status --short"));
assert!(matches_command("npm test *", "npm test"));
assert!(matches_command("npm test *", "npm test unit"));
assert!(matches_command("rm -rf *", "rm -rf /tmp/foo"));
assert!(matches_command("rm -rf *", "rm -rf /"));
}
#[test]
fn blacklist_takes_precedence_and_catches_chained_commands() {
let settings = AgentSecuritySettings::default();
assert_eq!(evaluate_command("rm -rf /tmp/build", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command(":(){ :|:& };:", &settings), PolicyDecision::Blocked(":(){ :|:& };:".into()));
}
#[test]
fn allow_entry_matching_word_boundary() {
assert!(matches_allow_entry("git status", "git status"));
assert!(matches_allow_entry("git status", "git status -s"));
assert!(!matches_allow_entry("git status", "git statusbar"));
assert!(!matches_allow_entry("git status", "git statusx"));
// Trailing asterisk in allow entry normalizes cleanly to word boundary prefix
assert!(matches_allow_entry("git status*", "git status"));
assert!(matches_allow_entry("git status*", "git status --short"));
assert!(!matches_allow_entry("git status*", "git statusbar"));
assert!(matches_allow_entry("ls", "ls"));
assert!(matches_allow_entry("ls", "ls -la"));
assert!(!matches_allow_entry("ls", "lsof"));
assert!(matches_allow_entry("cargo check", "cargo check"));
assert!(matches_allow_entry("cargo check", "cargo check --workspace"));
assert!(!matches_allow_entry("cargo check", "cargo checkmate"));
}
#[test]
fn unsafe_shell_syntax_detection() {
// Command substitutions: $(...) and `...`
assert!(has_unsafe_shell_syntax("echo $(whoami)"));
assert!(has_unsafe_shell_syntax("echo \"user: $(whoami)\""));
assert!(has_unsafe_shell_syntax("echo `whoami`"));
assert!(has_unsafe_shell_syntax("echo \"`whoami`\""));
// Redirections: >, >>, <
assert!(has_unsafe_shell_syntax("git status > /tmp/out"));
assert!(has_unsafe_shell_syntax("git status >> /tmp/out"));
assert!(has_unsafe_shell_syntax("cat < /tmp/in"));
assert!(has_unsafe_shell_syntax("cat << 'EOF'"));
// Subshells: (...)
assert!(has_unsafe_shell_syntax("(cd /tmp && ls)"));
assert!(has_unsafe_shell_syntax("git status && (exit 1)"));
// Environment assignments: VAR=val ...
assert!(has_unsafe_shell_syntax("FOO=bar git status"));
assert!(has_unsafe_shell_syntax("PORT=3000 npm test"));
assert!(has_unsafe_shell_syntax("_SECRET=123 cargo test"));
assert!(has_unsafe_shell_syntax("A=\"b c\" git status"));
// Safe constructs with characters inside quotes or command flags
assert!(!has_unsafe_shell_syntax("git commit -m \"fix <bug> and (issue)\""));
assert!(!has_unsafe_shell_syntax("git commit -m 'fix <bug>'"));
assert!(!has_unsafe_shell_syntax("cargo test --test=unit"));
assert!(!has_unsafe_shell_syntax("git status -s"));
}
#[test]
fn default_settings_has_no_auto_approval_and_blocks_dangerous_defaults() {
let settings = AgentSecuritySettings::default();
assert!(settings.allowed_commands.is_empty(), "allowed_commands must start empty");
// Safe commands ask user by default
assert_eq!(evaluate_command("git status", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("ls", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo check", &settings), PolicyDecision::AskUser);
// Blacklist patterns are still blocked by default
assert_eq!(evaluate_command("rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
assert_eq!(evaluate_command("rm -rf /tmp/test", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn whitelist_allows_safe_single_and_chained_commands_only() {
let settings = AgentSecuritySettings {
allowed_commands: vec!["git status".into(), "git diff".into(), "cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
};
assert_eq!(evaluate_command("git status -s", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git diff HEAD~1", &settings), PolicyDecision::Allowed);
assert_eq!(evaluate_command("git status && cargo test", &settings), PolicyDecision::Allowed);
// Word boundary: git statusx must not match git status
assert_eq!(evaluate_command("git statusx", &settings), PolicyDecision::AskUser);
// Unsafe syntax (redirection, command substitution, env assignment) falls back to AskUser
assert_eq!(evaluate_command("git status > out.txt", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("git status $(whoami)", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("FOO=bar git status", &settings), PolicyDecision::AskUser);
// Chaining an unapproved command must still ask the user.
assert_eq!(evaluate_command("git status && curl https://example.com", &settings), PolicyDecision::AskUser);
assert_eq!(evaluate_command("cargo publish", &settings), PolicyDecision::AskUser);
// Blacklist still takes precedence over whitelist
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
#[test]
fn extracts_command_from_v1_v2_and_codex_payloads() {
let claude = json!({
"toolCall": {
"title": "git status",
"kind": "execute",
"rawInput": { "command": "git status --short" }
}
});
assert_eq!(extract_command(&claude).as_deref(), Some("git status --short"));
let codex = json!({
"toolCall": {
"title": "Run cargo test",
"kind": "execute",
"rawInput": { "command": ["/bin/zsh", "-lc", "cargo test --workspace"] }
}
});
assert_eq!(extract_command(&codex).as_deref(), Some("cargo test --workspace"));
let write = json!({
"toolCall": {
"title": "Write foo.txt",
"kind": "edit",
"rawInput": { "file_path": "foo.txt" }
}
});
assert_eq!(extract_command(&write), None);
}
}
+419 -3
Переглянути файл
@@ -35,11 +35,19 @@ struct FakeAgent {
/// Without `loadSession` (a session lives only as long as its process).
no_load: bool,
auth_on_new: bool,
auth_on_load: bool,
authenticate_method: Option<String>,
authenticated: AtomicBool,
request_error: Option<(i64, String)>,
auth_methods: Vec<Value>,
/// Hold the late mode reply until the prompt's permission request is already waiting.
hold_late_mode: bool,
empty_config: AtomicBool,
/// Blocks `session/new` until discovery times out (see `acp::discover` test timeout).
hold_session_new: AtomicBool,
/// Sleeps this long at the start of `session/set_config_option`, after noting the time.
delay_config: Duration,
config_entered: Mutex<Option<std::time::Instant>>,
/// Config options `session/new` reports after the model one (per-model efforts, say).
extra_options: Mutex<Vec<Value>>,
model_options: Mutex<HashMap<String, Vec<Value>>>,
@@ -127,9 +135,9 @@ impl Handler for FakeAgent {
fn on_request(&self, method: &str, params: Value) -> Result<Value, (i64, String)> {
self.calls.lock().push(method.to_owned());
if (method == "session/new" && self.auth_on_new)
|| (method == "session/prompt" && params.pointer("/prompt/0/text") == Some(&json!("auth")))
{
let needs_auth = !self.authenticated.load(Ordering::Acquire)
&& ((method == "session/new" && self.auth_on_new) || (method == "session/load" && self.auth_on_load));
if needs_auth || (method == "session/prompt" && params.pointer("/prompt/0/text") == Some(&json!("auth"))) {
return Err(self
.request_error
.clone()
@@ -154,6 +162,9 @@ impl Handler for FakeAgent {
)
}
"session/new" => {
if self.hold_session_new.load(Ordering::Acquire) {
std::thread::sleep(Duration::from_secs(5));
}
if self.empty_config.load(Ordering::Acquire) {
return Ok(json!({"sessionId": "s1"}));
}
@@ -176,7 +187,21 @@ impl Handler for FakeAgent {
self.say(&session, "replayed history");
Ok(json!({}))
}
"authenticate" if self.authenticate_method.is_some() => {
let id = params.get("methodId").and_then(Value::as_str).unwrap_or_default();
self.calls.lock().push(format!("authenticate {id}"));
if Some(id) == self.authenticate_method.as_deref() {
self.authenticated.store(true, Ordering::Release);
Ok(json!({}))
} else {
Err((codes::INVALID_PARAMS, format!("unexpected methodId {id}")))
}
}
"session/set_config_option" | "session/set_mode" => {
if method == "session/set_config_option" && !self.delay_config.is_zero() {
*self.config_entered.lock() = Some(std::time::Instant::now());
std::thread::sleep(self.delay_config);
}
self.calls.lock().push(format!("{method} {}", params.get("value").or(params.get("modeId")).unwrap()));
if params["configId"] == "model" && !self.model_options.lock().is_empty() {
return Ok(json!({"configOptions": self.options_for(params["value"].as_str().unwrap())}));
@@ -696,6 +721,91 @@ async fn registry_sign_in_errors_use_the_advertised_terminal_login() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn authenticates_and_retries_session_setup_once_on_auth_error() {
let tmp = tempfile::tempdir().unwrap();
let methods = vec![
json!({"id": "cli_login", "_meta": {"terminal-auth": {"command": "copilot", "args": ["login"]}}}),
json!({"id": "github_oauth", "name": "GitHub OAuth"}),
];
let agent = Arc::new(FakeAgent {
auth_on_new: true,
authenticate_method: Some("github_oauth".into()),
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods.clone(),
..Default::default()
});
let mgr = manager(tmp.path(), agent.clone());
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "copilot".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
assert_eq!(
agent
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/new", "authenticate", "authenticate github_oauth", "session/new", "session/prompt"],
);
// Resuming after an IDE restart also authenticates once and retries `session/load`.
mgr.shutdown();
let agent2 = Arc::new(FakeAgent {
auth_on_load: true,
auth_on_new: true,
authenticate_method: Some("github_oauth".into()),
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods.clone(),
..Default::default()
});
let mgr2 = manager(tmp.path(), agent2.clone());
mgr2.prompt(&task.id, text("again"), false).await.unwrap();
tokio::time::sleep(Duration::from_millis(50)).await;
wait_for(&mgr2, &task.id, AgentStatus::Ready).await;
assert_eq!(
agent2
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/load", "authenticate", "authenticate github_oauth", "session/load", "session/prompt"],
);
// If `authenticate` fails during `session/load`, fallback to `session/new` does not try it again.
mgr2.shutdown();
let agent3 = Arc::new(FakeAgent {
auth_on_load: true,
auth_on_new: true,
request_error: Some((-32000, "Authentication required".into())),
auth_methods: methods,
..Default::default()
});
let mgr3 = manager(tmp.path(), agent3.clone());
mgr3.prompt(&task.id, text("again"), false).await.unwrap();
wait_for(&mgr3, &task.id, AgentStatus::Failed).await;
assert_eq!(
agent3
.calls
.lock()
.iter()
.filter(|c| c.starts_with("session/") || c.starts_with("authenticate"))
.collect::<Vec<_>>(),
["session/load", "authenticate", "session/new"],
);
mgr3.shutdown();
}
#[tokio::test(flavor = "multi_thread")]
async fn commands_are_remembered_per_project_and_merged_with_skills_on_disk() {
let tmp = tempfile::tempdir().unwrap();
@@ -2533,6 +2643,92 @@ async fn session_settings_win_over_discovery_in_memory_and_on_disk() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn provider_reload_does_not_start_acp_discovery() {
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.reload_providers();
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(agent.calls.lock().is_empty(), "reload must not spawn cursor-agent");
}
#[tokio::test(flavor = "multi_thread")]
async fn composer_commands_discover_cursor_effort_and_fast() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![
json!({"id": "effort", "type": "select", "currentValue": "medium",
"options": [{"value": "low"}, {"value": "medium"}, {"value": "high"}]}),
json!({"id": "fast", "type": "select", "currentValue": "false",
"options": [{"value": "false", "name": "Off"}, {"value": "true", "name": "On"}]}),
]),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
mgr.commands("cursor", None).await.unwrap();
let calls = agent.calls.lock();
assert!(calls.iter().any(|m| m.as_str() == "session/new"));
assert_eq!(calls.iter().filter(|m| m.as_str() == "initialize").count(), 1);
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert_eq!(cursor.config.iter().find(|o| o.id == "fast").unwrap().current.as_deref(), Some("false"));
assert!(
cursor
.thought_levels
.values()
.next()
.is_some_and(|levels| { levels.iter().any(|o| o.category.as_deref() == Some("thought_level")) })
);
}
#[tokio::test(flavor = "multi_thread")]
async fn config_choice_shows_before_the_agent_answers() {
let agent = Arc::new(FakeAgent {
extra_options: Mutex::new(vec![json!({
"id": "fast", "category": "model_config", "type": "select", "currentValue": "true",
"options": [{"value": "true", "name": "On"}, {"value": "false", "name": "Off"}]
})]),
delay_config: Duration::from_millis(300),
..Default::default()
});
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let shown: Arc<Mutex<Option<std::time::Instant>>> = Arc::default();
let mark = shown.clone();
mgr.subscribe(Arc::new(move |e| {
if let AgentEvent::Task { task } = e
&& task.config.iter().any(|o| o.id == "fast" && o.current.as_deref() == Some("false"))
&& mark.lock().is_none()
{
*mark.lock() = Some(std::time::Instant::now());
}
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("again"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
mgr.set_config(&task.id, "fast", "false".into()).await.unwrap();
let shown = (*shown.lock()).expect("the switch moved");
let entered = (*agent.config_entered.lock()).expect("the agent received the change");
assert!(shown < entered, "the composer updates before the agent answers");
}
#[tokio::test(flavor = "multi_thread")]
async fn discovery_times_out_when_session_new_hangs() {
let agent = Arc::new(FakeAgent { hold_session_new: AtomicBool::new(true), ..Default::default() });
let mgr = manager(tempfile::tempdir().unwrap().path(), agent.clone());
let started = std::time::Instant::now();
mgr.commands("cursor", None).await.unwrap();
assert!(started.elapsed() < Duration::from_secs(10), "discovery should time out quickly in tests");
let cursor = mgr.providers().into_iter().find(|p| p.id == "cursor").unwrap();
assert!(!cursor.config.iter().any(|o| o.category.as_deref() == Some("model")));
}
#[tokio::test(flavor = "multi_thread")]
async fn discovered_effort_levels_fill_models_no_session_has_reported() {
let tmp = tempfile::tempdir().unwrap();
@@ -2672,3 +2868,223 @@ async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edi
mgr.delete(&task.id, false).await.unwrap();
assert!(!dir.join("store").join(format!("{}.before", task.id)).exists());
}
/// A git repository in a temporary folder, for local tasks whose checkout moves under them.
struct Checkout {
_tmp: tempfile::TempDir,
dir: PathBuf,
}
impl Checkout {
fn new() -> Self {
let tmp = tempfile::tempdir().unwrap();
let dir = dunce::canonicalize(tmp.path()).unwrap();
let checkout = Self { _tmp: tmp, dir };
checkout.git(&["init", "-q", "-b", "main"]);
checkout.git(&["config", "core.autocrlf", "false"]);
checkout.write(".gitignore", "store/\nworktrees/\nagents/\nmodels.json\n");
checkout
}
fn git(&self, args: &[&str]) {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args(args)
.current_dir(&self.dir)
.status()
.unwrap()
.success();
assert!(ok, "{args:?}");
}
fn commit(&self, message: &str) {
self.git(&["add", "."]);
self.git(&["commit", "-q", "-m", message]);
}
fn write(&self, rel: &str, text: &str) {
std::fs::write(self.dir.join(rel), text).unwrap();
}
fn read(&self, rel: &str) -> String {
std::fs::read_to_string(self.dir.join(rel)).unwrap()
}
/// A local task, stopped so it can be rolled back.
async fn task(&self) -> (AgentManager, AgentTask) {
let mgr = manager(&self.dir, Arc::new(FakeAgent::default()));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("Hello"),
provider: "claude".into(),
config: Vec::new(),
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
mgr.cancel(&task.id, None).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
(mgr, task)
}
}
async fn changed(mgr: &AgentManager, id: &str) -> Vec<String> {
mgr.changes(id).await.unwrap().into_iter().map(|c| c.rel_path).collect()
}
#[tokio::test(flavor = "multi_thread")]
async fn work_uncommitted_by_a_reset_during_a_local_task_is_not_rolled_back() {
let repo = Checkout::new();
repo.write("a.txt", "base\n");
repo.write("mine.txt", "base\n");
repo.commit("init");
repo.write("mine.txt", "my work\n");
repo.commit("my work");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The user takes the last commit back into the working tree (`git reset --soft`, an
// undone commit): its changes are the user's, not the agent's.
repo.git(&["reset", "-q", "--soft", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"]);
let base = mgr.base_text(&task.id, repo.dir.join("a.txt")).await.unwrap();
assert_eq!(base.as_deref(), Some("base\n"));
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n");
assert_eq!(repo.read("mine.txt"), "my work\n", "the user's work survives the rollback");
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn a_branch_switched_to_during_a_local_task_is_left_alone() {
let repo = Checkout::new();
repo.write("a.txt", "base\n");
repo.write("other.txt", "base\n");
repo.commit("init");
repo.git(&["checkout", "-q", "-b", "feature"]);
repo.write("other.txt", "feature\n");
repo.write("only-feature.txt", "feature\n");
repo.commit("feature");
repo.git(&["checkout", "-q", "main"]);
repo.write("only-main.txt", "main\n");
repo.commit("main");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
// The agent's uncommitted edit comes along to a branch that doesn't contain the task's start.
repo.git(&["checkout", "-q", "feature"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"]);
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n");
assert_eq!(repo.read("other.txt"), "feature\n");
assert_eq!(repo.read("only-feature.txt"), "feature\n");
assert!(!repo.dir.join("only-main.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn commits_made_during_a_local_task_are_kept_by_its_rollback() {
let repo = Checkout::new();
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.write("b.txt", "agent\n");
// The user commits what the agent did to `a.txt` so far: it drops out of the task's changes.
repo.git(&["add", "a.txt"]);
repo.git(&["commit", "-q", "-m", "keep"]);
assert_eq!(changed(&mgr, &task.id).await, ["b.txt"]);
repo.write("a.txt", "agent\nagain\n");
assert_eq!(changed(&mgr, &task.id).await, ["a.txt", "b.txt"]);
mgr.rollback_changes(&task.id, None).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "back to the commit, not to before it");
assert!(!repo.dir.join("b.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_changes() {
let repo = Checkout::new();
repo.write("mine.txt", "mine\n");
let (mgr, task) = repo.task().await;
repo.write("new.txt", "agent\n");
assert_eq!(changed(&mgr, &task.id).await, ["new.txt"]);
assert_eq!(mgr.base_text(&task.id, repo.dir.join("new.txt")).await.unwrap(), None);
mgr.rollback_changes(&task.id, None).await.unwrap();
assert!(!repo.dir.join("new.txt").exists());
assert_eq!(repo.read("mine.txt"), "mine\n");
assert!(changed(&mgr, &task.id).await.is_empty());
}
#[tokio::test(flavor = "multi_thread")]
async fn security_policy_auto_approves_whitelist_and_rejects_blacklist() {
use ide_core::settings::AgentSecuritySettings;
// 1. Whitelist auto-approves "cargo test" without waiting for manual respond().
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent);
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["rm -rf *".into()],
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run tests"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
let entries = mgr.entries(&task.id, None).unwrap();
assert!(
entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose yes"))),
"{entries:?}"
);
assert!(
entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("whitelisted")
)),
"{entries:?}"
);
// 2. Blacklist auto-rejects "cargo test" when matched by blocked_commands.
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec!["cargo test*".into()],
}));
let blocked_task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("run blocked"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &blocked_task.id, AgentStatus::Ready).await;
let blocked_entries = mgr.entries(&blocked_task.id, None).unwrap();
assert!(
blocked_entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text.contains("chose no"))),
"{blocked_entries:?}"
);
assert!(
blocked_entries.iter().any(|e| matches!(
e,
AgentEntry::Permission { chosen: Some(c), .. } if c.contains("Blocked by policy")
)),
"{blocked_entries:?}"
);
}
+246 -16
Переглянути файл
@@ -14,8 +14,18 @@ pub(crate) struct Created {
pub base_commit: String,
}
/// The repository of `path`, free to execute: the host runs every agent command that
/// reaches a repository only in a trusted project.
fn discover(path: &Path) -> IdeResult<Option<Repo>> {
let repo = Repo::discover(path)?;
if let Some(repo) = &repo {
repo.set_execution_allowed(true);
}
Ok(repo)
}
fn repo_at(path: &Path) -> IdeResult<Repo> {
Repo::discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
discover(path)?.ok_or_else(|| IdeError::unavailable(format!("{} is not in a git repository", path.display())))
}
/// The current branch (or commit when detached) and its commit.
@@ -91,10 +101,11 @@ fn retrying(attempts: u32, pause: Duration, mut op: impl FnMut() -> std::io::Res
/// Files under `cwd` that differ from `base` (untracked files included), with line counts.
/// Outside a repository there is nothing to compare against: no changes.
pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(Vec::new()) };
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let stats = repo.numstat_with_worktree(base)?;
let stat = |rel: &str| stats.iter().find(|(p, ..)| p == rel).map_or((0, 0), |&(_, a, r)| (a, r));
let stats: std::collections::HashMap<String, (u32, u32)> =
repo.numstat_with_worktree(base)?.into_iter().map(|(p, a, r)| (p, (a, r))).collect();
let stat = |rel: &str| stats.get(rel).copied().unwrap_or((0, 0));
let mut out: Vec<AgentChange> = repo
.diff_with_worktree(base)?
.into_iter()
@@ -103,16 +114,139 @@ pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
AgentChange { path: IpcPath(root.join(&f.path)), rel_path: f.path, kind: f.kind, added, removed }
})
.collect();
let tracked: std::collections::HashSet<String> = out.iter().map(|c| c.rel_path.clone()).collect();
for s in repo.status()? {
if s.unstaged == Some(ChangeKind::Untracked) && !out.iter().any(|c| c.rel_path == s.rel_path) {
let added = std::fs::read_to_string(&*s.path).map_or(0, |t| t.lines().count() as u32);
out.push(AgentChange { path: s.path, rel_path: s.rel_path, kind: ChangeKind::Added, added, removed: 0 });
if s.unstaged == Some(ChangeKind::Untracked) && !tracked.contains(&s.rel_path) {
out.push(added(s.path, s.rel_path));
}
}
out.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
Ok(out)
}
fn added(path: IpcPath, rel_path: String) -> AgentChange {
let added = std::fs::read_to_string(&*path).map_or(0, |t| t.lines().count() as u32);
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0 }
}
/// Where a task works and what its changes are measured against.
pub(crate) enum TaskBase {
/// A worktree of its own: everything since the commit it was created from, the agent's
/// own commits included.
Worktree { cwd: PathBuf, base: String },
/// The project's checkout, where the user works and commits too. `start` is the commit it
/// started from (`None` in a repository without commits, or an imported session).
Local { cwd: PathBuf, start: Option<String> },
}
impl TaskBase {
pub(crate) fn cwd(&self) -> &Path {
match self {
Self::Worktree { cwd, .. } | Self::Local { cwd, .. } => cwd,
}
}
/// The files the task changed, less those changed before it started and untouched since.
pub(crate) fn changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<Vec<AgentChange>> {
let (cwd, start) = match self {
Self::Worktree { cwd, base } => return Ok(without_untouched(changes(cwd, base)?, before)),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
// What is uncommitted, as the Commit tool window lists it: commits made since the task
// started drop out.
let head = repo.branch_info()?.head_commit;
let listed = match &head {
Some(head) => changes(cwd, head)?,
None => {
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
all
}
};
let listed = without_untouched(listed, before);
let Some(start) = start.filter(|s| head.as_deref() != Some(*s)) else { return Ok(listed) };
// HEAD moved (a reset, a checkout, a pull): a file that is as it was when the task
// started is the user's doing, not the task's.
Ok(listed
.into_iter()
.filter(|c| {
before.contains_key(&c.rel_path)
|| !repo.file_bytes_at(&c.path.0, Some(start)).is_ok_and(|at| at == std::fs::read(&c.path.0).ok())
})
.collect())
}
/// The revision the task's changed files go back to on rollback, and are compared with:
/// the commit the task started from, or the current `HEAD` once the user committed on top
/// of it, so those commits stay. `None` while the repository has no commits.
pub(crate) fn rev(&self) -> IdeResult<Option<String>> {
let (cwd, start) = match self {
Self::Worktree { base, .. } => return Ok(Some(base.clone())),
Self::Local { cwd, start } => (cwd, start.as_deref()),
};
let Some(repo) = discover(cwd)? else { return Ok(None) };
let head = repo.branch_info()?.head_commit;
// A start commit git no longer knows (garbage-collected) can't be gone back to.
let start = start.filter(|s| repo.merge_base(s, s).ok().flatten().is_some());
Ok(match (start, head) {
(Some(start), Some(head)) if start != head && repo.merge_base(start, &head)?.as_deref() != Some(start) => {
Some(start.to_owned())
}
(start, head) => head.or_else(|| start.map(str::to_owned)),
})
}
}
/// Copies the task's `changes` (against `base`) from the worktree `cwd` into the
/// local checkout `root`, uncommitted: the user reviews and commits them with the
/// VCS tools. Files the checkout changed itself are left alone and reported, so
/// accepting never destroys the user's own edits. Returns accepted and skipped paths.
pub(crate) fn accept(
root: &Path,
cwd: &Path,
base: &str,
changes: &[AgentChange],
) -> IdeResult<crate::AgentAcceptOutcome> {
let repo = repo_at(cwd)?;
let mut accepted = Vec::with_capacity(changes.len());
let mut skipped = Vec::new();
for change in changes {
let local = root.join(&change.rel_path);
let base_bytes = repo.file_bytes_at(&change.path.0, Some(base))?;
let current = std::fs::read(&change.path.0).ok();
let local_bytes = std::fs::read(&local).ok();
if local_bytes == current {
accepted.push(change.rel_path.clone());
} else if local_bytes == base_bytes {
match current {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
}
std::fs::write(&local, bytes)?;
}
None => match std::fs::remove_file(&local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
},
}
accepted.push(change.rel_path.clone());
} else {
skipped.push(crate::AgentAcceptSkipped {
path: change.rel_path.clone(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
});
}
}
Ok(crate::AgentAcceptOutcome { accepted, skipped })
}
/// Size and modification time of a file, `None` once it is gone.
pub(crate) type Stamp = Option<(u64, u128)>;
@@ -124,7 +258,10 @@ pub(crate) fn stamp(path: &Path) -> Stamp {
/// The files already changed in `cwd` against `HEAD`, stamped so later edits to them show.
pub(crate) fn snapshot(cwd: &Path) -> IdeResult<BTreeMap<String, Stamp>> {
Ok(changes(cwd, "HEAD")?.into_iter().map(|c| (c.rel_path, stamp(&c.path))).collect())
// Paths and stamps only: no diff and no line counts, which read every untracked file
// and held up starting a task in a checkout with many of them.
let Some(repo) = Repo::discover(cwd)? else { return Ok(BTreeMap::new()) };
Ok(repo.status()?.into_iter().map(|s| (s.rel_path, stamp(&s.path))).collect())
}
/// Drops the changes that were there before the task and haven't been touched since.
@@ -161,11 +298,11 @@ pub(crate) fn save_before(cwd: &Path, before: &BTreeMap<String, Stamp>, dir: &Pa
}
/// Rolls `change` back: to the copy saved when the task started when the file had changes
/// then (`before`, `saved` holding the copies), otherwise to `base` (removed when `base`
/// lacks it).
/// then (`before`, `saved` holding the copies), otherwise to `rev` (removed when `rev` lacks
/// it, or when there is no `rev`: the repository had no commits).
pub(crate) fn restore(
cwd: &Path,
base: &str,
rev: Option<&str>,
change: &AgentChange,
before: &BTreeMap<String, Stamp>,
saved: Option<&Path>,
@@ -183,10 +320,13 @@ pub(crate) fn restore(
})?;
Some(std::fs::read(copy)?)
}
None => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(base))?
}
None => match rev {
Some(rev) => {
let repo = Repo::discover(cwd)?.ok_or_else(|| IdeError::invalid("not in a git repository"))?;
repo.file_bytes_at(path, Some(rev))?
}
None => None,
},
};
match content {
Some(bytes) => {
@@ -204,7 +344,7 @@ pub(crate) fn restore(
}
pub(crate) fn file_at(cwd: &Path, path: &Path, rev: &str) -> IdeResult<Option<String>> {
let Some(repo) = Repo::discover(cwd)? else { return Ok(None) };
let Some(repo) = discover(cwd)? else { return Ok(None) };
let encoding = match ide_vfs::load_text(path) {
Ok(loaded) => encoding_rs::Encoding::for_label(loaded.encoding.as_bytes()).unwrap_or(encoding_rs::UTF_8),
Err(_) => ide_vfs::detect_encoding(&repo.file_bytes_at(path, Some(rev))?.unwrap_or_default(), None),
@@ -391,4 +531,94 @@ mod tests {
assert!(remove(&repo, &tmp.path().join("wt"), &elsewhere).is_err());
assert!(elsewhere.join("keep.txt").exists());
}
fn accept_repo() -> (tempfile::TempDir, PathBuf, Created) {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
std::fs::write(repo.join("gone.txt"), "bye\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
(tmp, repo, created)
}
fn listed(cwd: &Path, base: &str) -> Vec<AgentChange> {
without_untouched(changes(cwd, base).unwrap(), &BTreeMap::new())
}
#[test]
fn accept_copies_new_modified_and_deleted_files_into_the_checkout() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("new.txt"), "x\n").unwrap();
std::fs::remove_file(created.path.join("gone.txt")).unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(outcome.skipped.is_empty());
assert_eq!(outcome.accepted, ["a.txt", "gone.txt", "new.txt"]);
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nTWO\n");
assert_eq!(std::fs::read_to_string(repo.join("new.txt")).unwrap(), "x\n");
assert!(!repo.join("gone.txt").exists());
// Accepting again changes nothing: the checkout already matches.
let again =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert!(again.skipped.is_empty());
assert_eq!(again.accepted, ["a.txt", "gone.txt", "new.txt"]);
}
#[test]
fn a_local_task_goes_back_to_its_start_unless_the_user_committed_on_top_of_it() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
let local = |start: Option<&str>| TaskBase::Local { cwd: repo.clone(), start: start.map(str::to_owned) };
let head = || head_of(&repo).unwrap().1;
let commit = |name: &str| {
std::fs::write(repo.join(name), "x\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", name]);
head_of(&repo).unwrap().1
};
assert_eq!(local(None).rev().unwrap(), None, "no commits yet");
let first = commit("a.txt");
assert_eq!(local(None).rev().unwrap(), Some(first.clone()), "an imported session: HEAD");
let start = commit("b.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "HEAD where it started");
let ahead = commit("c.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(ahead), "committed on top: HEAD");
git(&repo, &["reset", "-q", "--soft", &first]);
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "moved back: the start");
git(&repo, &["checkout", "-q", "-b", "side"]);
commit("d.txt");
assert_eq!(local(Some(&start)).rev().unwrap(), Some(start.clone()), "another branch: the start");
let gone = "0123456789abcdef0123456789abcdef01234567";
assert_eq!(local(Some(gone)).rev().unwrap(), Some(head()), "an unknown start: HEAD");
let worktree = TaskBase::Worktree { cwd: repo.clone(), base: start.clone() };
assert_eq!(worktree.rev().unwrap(), Some(start));
}
#[test]
fn accept_leaves_files_the_checkout_changed_itself() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
std::fs::write(created.path.join("gone.txt"), "bye\nbye\n").unwrap();
std::fs::write(repo.join("a.txt"), "one\nMINE\n").unwrap();
let outcome =
accept(&repo, &created.path, &created.base_commit, &listed(&created.path, &created.base_commit)).unwrap();
assert_eq!(outcome.accepted, ["gone.txt"]);
assert_eq!(outcome.skipped.len(), 1);
assert_eq!(outcome.skipped[0].path, "a.txt");
assert!(!outcome.skipped[0].reason.is_empty());
// The user's own edit survives; the other file comes over.
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\nMINE\n");
assert_eq!(std::fs::read_to_string(repo.join("gone.txt")).unwrap(), "bye\nbye\n");
}
}
+110 -22
Переглянути файл
@@ -79,23 +79,26 @@ fn scan(
Ok(())
}
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
/// One planned step as fingerprinted: resolved program, args, cwd and env.
type StepFingerprint = (PathBuf, Vec<String>, PathBuf, BTreeMap<String, String>);
/// Environment, planned steps and input paths shared by every Maven build
/// fingerprint: the filtered process environment plus the wrapper, settings
/// and JDK files the planned invocation reads.
struct Collected {
inputs: Vec<PathBuf>,
environment: BTreeMap<String, String>,
steps: Vec<StepFingerprint>,
inherited_hash: u64,
}
fn collect(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
projects: &[&maven::MavenProject],
plan: &BuildPlan,
local_repo: Option<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let Some(mut inputs) = maven::launch_inputs(root, &projects, local_repo) else { return Ok(None) };
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
) -> IdeResult<Option<Collected>> {
let Some(mut inputs) = maven::launch_inputs(root, projects, local_repo) else { return Ok(None) };
let mut environment: BTreeMap<String, String> = std::env::vars()
.map(|(key, value)| (key.to_ascii_uppercase(), value))
.filter(|(key, _)| {
@@ -165,26 +168,111 @@ pub fn launch_fingerprint(
Path::new(home).join(if cfg!(windows) { "bin/java.exe" } else { "bin/java" }),
]);
}
let steps: Vec<_> =
plan.steps.iter().map(|s| (process::program_path(&s.program, &s.env), &s.args, &s.cwd, &s.env)).collect();
let steps = plan
.steps
.iter()
.map(|s| (process::program_path(&s.program, &s.env), s.args.clone(), s.cwd.clone(), s.env.clone()))
.collect();
// Profiles and plugins can read arbitrary environment variables. Keep a signature
// of the inherited environment without persisting unrelated credentials.
let inherited: BTreeMap<_, _> = std::env::vars_os().collect();
let mut inherited_hash = std::hash::DefaultHasher::new();
inherited.hash(&mut inherited_hash);
let parameters =
serde_json::to_string(&(root, module, test, steps, environment, inherited_hash.finish(), projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(Collected { inputs, environment, steps, inherited_hash: inherited_hash.finish() }))
}
/// Scans every input, storing project inputs relative to the project.
fn scan_all(
root: &Path,
inputs: Vec<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<BTreeMap<PathBuf, Option<Stamp>>> {
let mut files = BTreeMap::new();
for path in inputs {
scan(root, &path, &mut files, &mut HashSet::new(), cancel)?;
}
// Store project inputs relative to the project, including each file's relative path.
let files = files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect();
Ok(Some(LaunchFingerprint { version: 2, parameters, files }))
Ok(files.into_iter().map(|(p, stamp)| (p.strip_prefix(root).unwrap_or(&p).to_path_buf(), stamp)).collect())
}
/// `None` means an input cannot be resolved safely: build normally without reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn launch_fingerprint(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let Some(m) = &model.maven else { return Ok(None) };
let projects = maven_launch_projects(root, m, module, test);
if projects.is_empty() {
return Ok(None);
}
let plan = launch_plan(root, model, module, test, BuildKind::Build)?;
let Some(collected) = collect(root, &projects, &plan, local_repo)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let parameters = serde_json::to_string(&(root, module, test, steps, environment, inherited_hash, projects))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
/// Inputs fingerprint of a menu build (`Build Project` / `Build Module`): the
/// built modules' sources, resources and POM chain plus the exact planned
/// steps, so an unchanged project skips Maven entirely (#2476). Only `Build`
/// callers reuse it, never `Rebuild` or `Artifacts`; `None` disables reuse.
/// All filesystem work belongs on the caller's blocking thread.
pub fn menu_fingerprint(
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> IdeResult<Option<LaunchFingerprint>> {
let plan = plan(root, model, BuildKind::Build, file, None)?;
if plan.system != BuildSystem::Maven {
return Ok(None);
}
let Some(m) = &model.maven else { return Ok(None) };
let projects: Vec<&maven::MavenProject> = match file {
Some(file) => {
// The innermost linked project owning the file, as in single-file compilation.
let rel = rel_str(root, file);
let module = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len());
match module {
Some(project) if project.packaging != "pom" => {
maven_launch_projects(root, m, &subdir(root, &project.dir), true)
}
_ => return Ok(None),
}
}
// A project build walks every linked project.
None => m.projects.iter().collect(),
};
if projects.is_empty() {
return Ok(None);
}
let Some(collected) = collect(root, &projects, &plan, None)? else { return Ok(None) };
let Collected { inputs, environment, steps, inherited_hash } = collected;
let scope = file.map(|f| rel_str(root, f));
let parameters = serde_json::to_string(&("menu-build", scope, steps, environment, inherited_hash))
.map_err(|e| IdeError::internal(e.to_string()))?;
Ok(Some(LaunchFingerprint { version: 2, parameters, files: scan_all(root, inputs, cancel)? }))
}
impl LaunchFingerprint {
/// Folders (relative to the project) of the Maven projects whose POM is an input.
pub(super) fn project_dirs(&self) -> impl Iterator<Item = &Path> {
self.files
.keys()
.filter(|p| p.is_relative() && p.file_name().is_some_and(|n| n == "pom.xml"))
.map(|pom| pom.parent().unwrap_or(Path::new("")))
}
/// Relative paths whose content changed while `(length, mtime)` stayed the same
/// compared to `previous` (timestamp-preserving external edits, coarse-granularity
/// filesystems). Timestamp-incremental builders (Maven, the Kotlin plugin) would
+253
Переглянути файл
@@ -0,0 +1,253 @@
//! Resources a Gradle build changes while copying them (filters, `expand`, renames, extra
//! sources). The Java language server copies them as they are into Buildship's
//! `bin/<source set>` folders, which launches run from; IntelliJ IDEA runs Gradle's copy.
//! Before a launch, Gradle processes them and its output replaces those plain copies, so
//! each resource is on the classpath once, as the build produces it.
use super::{BuildPlan, BuildStep, BuildSystem, Format, run, subdir};
use crate::BuildModel;
use crate::gradle::{self, GradleModel, ReportedProject};
use ide_core::{CancellationToken, IdeError, IdeResult, Severity};
use std::fs;
use std::path::{Path, PathBuf};
/// A source set whose resources Gradle processes, with the folder the launch reads them from.
#[derive(Debug, PartialEq)]
struct Target {
/// Task path from the root build (`:app:processResources`).
task: String,
/// Where the task writes the resources.
output: PathBuf,
/// The Java language server's output folder of the source set, on the launch classpath.
jdtls: PathBuf,
}
fn targets(reported: &[ReportedProject], class_paths: &[String]) -> Vec<Target> {
// buildSrc is built before the build, never addressable as `:buildSrc:<task>`.
let addressable = |p: &&ReportedProject| !p.build.as_deref().is_some_and(|b| b.ends_with(":buildSrc"));
let sets: Vec<_> = reported
.iter()
.filter(addressable)
.flat_map(|p| p.source_sets.iter().filter_map(move |s| Some((p, s, s.processed.as_ref()?))))
.collect();
if sets.is_empty() {
return Vec::new();
}
// Canonical, as Gradle reports its folders: jdtls may spell them differently.
let entries: Vec<PathBuf> = class_paths.iter().filter_map(|p| dunce::canonicalize(p).ok()).collect();
sets.into_iter()
.filter_map(|(p, set, resources)| {
// Buildship's output folder of the source set.
let jdtls = dunce::canonicalize(Path::new(&p.dir).join("bin").join(&set.name)).ok()?;
let project = if p.path == ":" { "" } else { p.path.as_str() };
entries.contains(&jdtls).then(|| Target {
task: format!("{project}:{}", resources.task),
output: PathBuf::from(&resources.dir),
jdtls,
})
})
.collect()
}
/// Before a JVM launch from the Java language server's output folders on `class_paths`:
/// runs the Gradle tasks that process resources of those folders' source sets (see the
/// module docs) and copies their output over the plain copies. `cache_dir` holds the
/// linked builds' last syncs. Returns the tasks run, none for builds that copy their
/// resources as they are.
pub fn process_launch_resources(
root: &Path,
model: &BuildModel,
cache_dir: &Path,
class_paths: &[String],
cancel: &CancellationToken,
) -> IdeResult<Vec<String>> {
// A composite also reports the builds it includes: the deepest build owning a folder runs its task.
let mut builds: Vec<&GradleModel> = model.gradle.iter().collect();
builds.sort_by_key(|g| std::cmp::Reverse(subdir(root, &g.dir).components().count()));
let mut done: Vec<PathBuf> = Vec::new();
let mut tasks = Vec::new();
for g in builds {
let reported = crate::sync::reported_gradle_projects(&gradle::cache_dir(cache_dir, &g.dir));
let targets: Vec<Target> =
targets(&reported, class_paths).into_iter().filter(|t| !done.contains(&t.jdtls)).collect();
if targets.is_empty() {
continue;
}
let mut step = BuildStep::new(&g.command, &["--console=plain"], &subdir(root, &g.dir), Format::Gradle);
step.args.extend(targets.iter().map(|t| t.task.clone()));
step.env = g.env.clone();
let plan = BuildPlan { system: BuildSystem::Gradle, title: "Process resources".into(), steps: vec![step] };
let result = run(root, &plan, cancel, &|_| {})?;
if !result.success {
let errors: Vec<&str> = result
.messages
.iter()
.filter(|m| m.severity == Severity::Error)
.take(5)
.map(|m| m.text.as_str())
.collect();
return Err(IdeError::internal(errors.join("\n")));
}
for t in targets {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
overlay(&t.output, &t.output, &t.jdtls)?;
done.push(t.jdtls);
tasks.push(t.task);
}
}
Ok(tasks)
}
/// Copies the files under `dir` to the same place under `to` (relative to `base`) where
/// they differ, so unchanged resources are not rewritten on every launch.
fn overlay(base: &Path, dir: &Path, to: &Path) -> IdeResult<()> {
let entries = match fs::read_dir(dir) {
// No resources: the task had nothing to copy.
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(()),
entries => entries?,
};
for entry in entries {
let entry = entry?;
let path = entry.path();
let kind = entry.file_type()?;
if kind.is_dir() {
overlay(base, &path, to)?;
} else if kind.is_file() {
let dest = to.join(path.strip_prefix(base).map_err(|e| IdeError::internal(e.to_string()))?);
if !same_content(&path, &dest) {
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent)?;
}
fs::copy(&path, &dest)?;
}
}
}
Ok(())
}
fn same_content(a: &Path, b: &Path) -> bool {
match (fs::metadata(a), fs::metadata(b)) {
(Ok(m), Ok(n)) if m.len() == n.len() => matches!((fs::read(a), fs::read(b)), (Ok(x), Ok(y)) if x == y),
_ => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gradle::{ProcessedResources, ReportedSourceSet};
fn write(path: &Path, text: &str) {
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, text).unwrap();
}
fn set(name: &str, processed: Option<&Path>) -> ReportedSourceSet {
ReportedSourceSet {
name: name.into(),
test: name == "test",
processed: processed.map(|dir| ProcessedResources {
task: if name == "main" { "processResources".into() } else { "processTestResources".into() },
dir: dir.to_string_lossy().into_owned(),
}),
..Default::default()
}
}
fn project(path: &str, dir: &Path, sets: Vec<ReportedSourceSet>) -> ReportedProject {
ReportedProject {
path: path.into(),
dir: dir.to_string_lossy().into_owned(),
source_sets: sets,
..Default::default()
}
}
/// Only processed source sets whose jdtls output is on the classpath: the launched
/// module, upstream modules of the build, tests only for a test classpath; never buildSrc.
#[test]
fn targets_are_processed_sets_on_the_classpath() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
let (app, lib, plain, src) = (root.join("app"), root.join("lib"), root.join("plain"), root.join("buildSrc"));
for dir in [&root, &app, &lib, &plain, &src] {
for set in ["main", "test"] {
fs::create_dir_all(dir.join("bin").join(set)).unwrap();
}
}
let out = |dir: &Path, set: &str| dir.join("build/resources").join(set);
let mut build_src = project(":buildSrc", &src, vec![set("main", Some(&out(&src, "main")))]);
build_src.build = Some(":buildSrc".into());
let reported = [
project(":", &root, vec![set("main", Some(&out(&root, "main")))]),
project(":app", &app, vec![set("main", Some(&out(&app, "main"))), set("test", Some(&out(&app, "test")))]),
project(":lib", &lib, vec![set("main", Some(&out(&lib, "main")))]),
project(":plain", &plain, vec![set("main", None)]),
build_src,
];
let entry = |dir: &Path, set: &str| dir.join("bin").join(set).to_string_lossy().into_owned();
let runtime = [
entry(&app, "main"),
// Spelled differently from Gradle's canonical folders.
format!("{}/", entry(&lib, "main")),
entry(&plain, "main"),
entry(&src, "main"),
"/m2/spring-core.jar".into(),
];
let tasks = |cp: &[String]| targets(&reported, cp).into_iter().map(|t| t.task).collect::<Vec<_>>();
assert_eq!(tasks(&runtime), [":app:processResources", ":lib:processResources"]);
let mut test = runtime.to_vec();
test.insert(0, entry(&app, "test"));
assert_eq!(tasks(&test), [":app:processResources", ":app:processTestResources", ":lib:processResources"]);
assert_eq!(tasks(&[entry(&root, "main")]), [":processResources"], "the root project");
let app_targets = targets(&reported, &runtime);
let t = &app_targets[0];
assert_eq!(
(t.output.as_path(), t.jdtls.as_path()),
(out(&app, "main").as_path(), app.join("bin/main").as_path())
);
let unprocessed = [project(":app", &app, vec![set("main", None)])];
assert!(targets(&unprocessed, &runtime).is_empty());
}
/// The processed copy replaces jdtls's plain one (the reported `@version@` case), new
/// files are added, identical ones are left alone and files Gradle doesn't produce stay.
#[test]
fn overlay_replaces_plain_copies_with_gradle_output() {
let tmp = tempfile::tempdir().unwrap();
let (output, jdtls) = (tmp.path().join("build/resources/main"), tmp.path().join("bin/main"));
write(&output.join("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
write(&output.join("static/extra.txt"), "from an extra source");
write(&output.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("application.yaml"), "spring:\n application:\n version: @version@\n");
write(&jdtls.join("migrations/V1__init.sql"), "create table t (id int);");
write(&jdtls.join("com/example/App.class"), "bytecode");
let unchanged = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
std::thread::sleep(std::time::Duration::from_millis(20));
overlay(&output, &output, &jdtls).unwrap();
let read = |rel: &str| fs::read_to_string(jdtls.join(rel)).unwrap();
assert_eq!(read("application.yaml"), "spring:\n application:\n version: 0.0.1\n");
assert_eq!(read("static/extra.txt"), "from an extra source");
assert_eq!(read("com/example/App.class"), "bytecode");
let modified = fs::metadata(jdtls.join("migrations/V1__init.sql")).unwrap().modified().unwrap();
assert_eq!(modified, unchanged, "identical files are not rewritten");
overlay(&tmp.path().join("missing"), &tmp.path().join("missing"), &jdtls).unwrap();
}
#[test]
fn builds_without_processed_resources_run_nothing() {
let tmp = tempfile::tempdir().unwrap();
// Running this command would fail the call.
let missing = tmp.path().join("missing-gradle").to_string_lossy().into_owned();
let model =
BuildModel { gradle: vec![GradleModel { command: missing, ..Default::default() }], ..Default::default() };
let class_paths = [tmp.path().join("bin/main").to_string_lossy().into_owned()];
let tasks =
process_launch_resources(tmp.path(), &model, tmp.path(), &class_paths, &CancellationToken::new()).unwrap();
assert!(tasks.is_empty());
}
}
+40 -2
Переглянути файл
@@ -4,9 +4,17 @@
//! for the Build tool window, the Problems view and the editor.
mod fingerprint;
mod launch_resources;
mod parse;
mod single;
mod up_to_date;
pub use fingerprint::{LaunchFingerprint, launch_fingerprint};
pub use fingerprint::{LaunchFingerprint, launch_fingerprint, menu_fingerprint};
pub use launch_resources::process_launch_resources;
pub use single::{
SingleFileTarget, compile_file_plan, find_javac, javac_step, single_file_classpath, single_file_target,
};
pub use up_to_date::{menu_up_to_date, record_menu_build};
use crate::process::{self, Stream};
use crate::{BuildModel, maven};
@@ -258,6 +266,11 @@ pub(super) fn maven_steps(
BuildKind::Rebuild => &["clean", "test-compile"],
BuildKind::Artifacts => &["package", "-DskipTests"],
};
// The IDE's compile builds are not coverage runs. JaCoCo's offline `instrument`
// rewrites target/classes in `process-classes`; `jacoco.skip` prevents that in this
// build. It does not restore classes instrumented by an earlier build: Rebuild's
// `clean` removes those, while an incremental Build can still reuse them.
let skip_jacoco = matches!(kind, BuildKind::Build | BuildKind::Rebuild);
let tops: Vec<&maven::MavenProject> = m.projects.iter().filter(|p| !p.aggregated).collect();
if tops.is_empty() {
return Err(IdeError::not_found("no Maven project found"));
@@ -270,6 +283,9 @@ pub(super) fn maven_steps(
Format::Maven,
);
s.args.extend(p.options.iter().cloned());
if skip_jacoco {
s.args.push("-Djacoco.skip=true".into());
}
s.args.extend(extra.iter().cloned());
s.args.extend(goals.iter().map(|g| (*g).to_owned()));
s.env = p.env.clone();
@@ -1033,6 +1049,22 @@ mod tests {
std::fs::write(p, text).unwrap();
}
#[test]
fn maven_compile_steps_skip_jacoco_but_artifacts_do_not() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path().join("demo");
write(&root, "pom.xml", "<project><artifactId>demo</artifactId></project>");
let model = crate::load(&root, &tmp.path().join("cache"), None);
for (kind, expected) in [(BuildKind::Build, true), (BuildKind::Rebuild, true), (BuildKind::Artifacts, false)] {
let plan = plan(&root, &model, kind, None, None).unwrap();
assert_eq!(
plan.steps[0].args.iter().any(|arg| arg == "-Djacoco.skip=true"),
expected,
"{kind:?}: {plan:?}"
);
}
}
#[test]
fn plans_per_build_system() {
let tmp = tempfile::tempdir().unwrap();
@@ -1049,7 +1081,10 @@ mod tests {
assert_eq!(p.system, BuildSystem::Maven);
assert_eq!(p.title, "demo");
assert_eq!(p.steps.len(), 1);
assert_eq!(p.steps[0].args, ["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "test-compile"]);
assert_eq!(
p.steps[0].args,
["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG, "-Djacoco.skip=true", "test-compile"]
);
let p = plan(&root, &model, BuildKind::Rebuild, Some(&root.join("core/src/main/java/A.java")), None).unwrap();
assert_eq!(
p.steps[0].args,
@@ -1057,6 +1092,7 @@ mod tests {
"-B",
"-Dstyle.color=never",
maven::LOCAL_REPO_TRACKING_ARG,
"-Djacoco.skip=true",
"-pl",
"core",
"-am",
@@ -1345,6 +1381,7 @@ mod tests {
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
@@ -1361,6 +1398,7 @@ mod tests {
maven::LOCAL_REPO_TRACKING_ARG,
"-s",
"settings.xml",
"-Djacoco.skip=true",
"-pl",
"app",
"-am",
+1 -26
Переглянути файл
@@ -7,6 +7,7 @@
//! [`super::resolve`] turns them into IDE coordinates.
use ide_core::Severity;
pub(crate) use ide_core::output_encoding::strip_ansi;
use serde_json::Value;
/// Output dialect of a build step.
@@ -98,32 +99,6 @@ pub(crate) struct OutputParser {
pending: Option<Pending>,
}
/// Removes ANSI escape sequences (colour, cursor movement).
pub(crate) fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
fn num(s: &str) -> Option<u32> {
s.trim().parse().ok().filter(|n| *n > 0)
}
+331
Переглянути файл
@@ -0,0 +1,331 @@
//! Single-file Java compilation with `javac` for the Compile File action (#2476).
//!
//! A module build (`-pl … -am`) still walks every upstream module, and packaging
//! walks the whole reactor; compiling one file only needs that file's module
//! outputs plus its dependency classpath, which is much faster to assemble.
use super::*;
use std::fs;
use std::process::Command as ProcCommand;
/// Output file of the `dependency:build-classpath` probe, relative to each module.
const CP_FILE: &str = "target/ide-compile-cp.txt";
/// A `.java` source and the module build that compiles it.
pub struct SingleFileTarget {
/// Absolute module directory.
pub module: PathBuf,
/// The source is under `src/test`: compile to test classes with the test classpath.
pub test: bool,
/// Absolute `target/classes` (or `target/test-classes`).
pub out_dir: PathBuf,
/// Reactor root building the module (owns the Maven command and options).
pub top: maven::MavenProject,
/// `-pl` value of the module relative to the reactor root; empty when the
/// module is the root itself.
pub pl: String,
/// `maven.compiler.release` (or equivalent) of the module's project.
pub java_version: Option<u32>,
}
/// Locates the linked Maven module owning a `.java` source file.
pub fn single_file_target(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<SingleFileTarget> {
if file.extension().is_none_or(|ext| ext != "java") {
return Err(IdeError::invalid("only Java sources can be compiled as a single file"));
}
if !file.is_file() {
return Err(IdeError::not_found("no Java source found for single-file compilation"));
}
let Some(m) = &model.maven else {
return Err(IdeError::not_found("no linked Maven project found for module"));
};
// The innermost project whose directory contains the file owns it; the
// aggregator root (`""`) is a prefix of everything, so take the longest match.
let rel = rel_str(root, file);
let project = m
.projects
.iter()
.filter(|p| rel == p.dir || rel.starts_with(&format!("{}/", p.dir)) || p.dir.is_empty())
.max_by_key(|p| p.dir.len())
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
if project.packaging == "pom" {
// An aggregator owns no sources itself: a file that only matches it
// belongs to no linked module.
return Err(IdeError::not_found("no linked Maven project found for module"));
}
let module = subdir(root, &project.dir);
let top = maven_reactor(m, &project.dir)
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
let top_dir = subdir(root, &top.dir);
let pl = module.strip_prefix(&top_dir).map(|r| r.to_string_lossy().replace('\\', "/")).unwrap_or_default();
// Test sources see the test classpath and compile into test classes.
let parts: Vec<&str> = rel.split('/').collect();
let test = parts.windows(2).any(|w| w == ["src", "test"]);
let out_dir = module.join("target").join(if test { "test-classes" } else { "classes" });
Ok(SingleFileTarget { module, test, out_dir, top: top.clone(), pl, java_version: project.java_version })
}
/// `javac` for the module: its `JAVA_HOME`, else `PATH`.
pub fn find_javac(env: &BTreeMap<String, String>) -> String {
let home = env.get("JAVA_HOME").map(Path::new);
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
if let Some(javac) = home.map(|h| h.join("bin").join(exe)).filter(|p| p.is_file()) {
return javac.to_string_lossy().into_owned();
}
exe.to_owned()
}
/// Dependency classpath of the module: existing reactor class directories plus
/// `mvn dependency:build-classpath` (which resolves without compiling anything,
/// so `-am` stays cheap). Entries are platform-separated (`;` on Windows).
pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFileTarget) -> IdeResult<String> {
let mut entries: Vec<String> = Vec::new();
// The module's own classes first: the file almost always uses classes next to it.
let own: &[&str] = if target.test { &["target/test-classes", "target/classes"] } else { &["target/classes"] };
for out in own {
let path = target.module.join(out);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
if let Some(m) = &model.maven {
// Upstream reactor modules first: their freshly built classes shadow
// installed snapshots of the same artifacts.
for project in maven_launch_projects(root, m, &target.module, target.test) {
let dir = subdir(root, &project.dir);
if dir == target.module {
continue;
}
for classes in ["target/classes", "target/test-classes"] {
let path = dir.join(classes);
if path.is_dir() {
entries.push(path.to_string_lossy().into_owned());
}
}
}
}
let probe = CP_FILE;
let mut args = vec!["-B", "-Dstyle.color=never", maven::LOCAL_REPO_TRACKING_ARG];
args.extend(target.top.options.iter().map(String::as_str));
if !target.pl.is_empty() {
args.extend(["-pl", target.pl.as_str(), "-am"]);
}
let output_arg = format!("-Dmdep.outputFile={probe}");
args.extend(["dependency:build-classpath", output_arg.as_str(), "-Dmdep.includeScope=test", "-q"]);
let top_dir = subdir(root, &target.top.dir);
let status = ProcCommand::new(&target.top.command)
.args(&args)
.envs(&target.top.env)
.current_dir(&top_dir)
.output()
.map_err(|e| IdeError::internal(format!("cannot resolve the compile classpath: {e}")))?;
if !status.status.success() {
let stderr = String::from_utf8_lossy(&status.stderr);
let tail: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
let tail = tail.iter().rev().take(5).rev().cloned().collect::<Vec<_>>().join("\n");
return Err(IdeError::internal(format!("cannot resolve the compile classpath:\n{tail}")));
}
// With `-am` every visited module writes its own file; only this module's counts.
let cp_file = target.module.join(probe);
let content = fs::read_to_string(&cp_file).map_err(|e| {
IdeError::internal(format!("cannot resolve the compile classpath: {} ({e})", cp_file.display()))
})?;
let separator = if cfg!(windows) { ';' } else { ':' };
entries.extend(content.split([separator, '\n', '\r']).map(str::trim).filter(|e| !e.is_empty()).map(str::to_owned));
entries.dedup();
Ok(entries.join(&separator.to_string()))
}
/// The `javac` step compiling `file` into `target`'s output directory.
pub fn javac_step(target: &SingleFileTarget, file: &Path, classpath: &str, javac: &str) -> IdeResult<BuildStep> {
fs::create_dir_all(&target.out_dir)
.map_err(|e| IdeError::internal(format!("cannot create {} ({e})", target.out_dir.display())))?;
let mut step = BuildStep::new(javac, &[], &target.module, Format::Gradle);
// Raw `javac` prints `path:line: error: …`, the same form Gradle forwards.
step.args.push("-encoding".into());
step.args.push("UTF-8".into());
step.args.push("-d".into());
step.args.push(target.out_dir.to_string_lossy().into_owned());
step.args.push("-cp".into());
step.args.push(classpath.to_owned());
// Classes of the module that were never built (or are older than their source) are read
// from its sources; only `file` is written, so a test compile never puts main classes
// into test-classes.
let sources = source_roots(target);
if !sources.is_empty() {
let separator = if cfg!(windows) { ";" } else { ":" };
step.args.push("-sourcepath".into());
step.args.push(sources.iter().map(|p| p.to_string_lossy()).collect::<Vec<_>>().join(separator));
step.args.push("-implicit:none".into());
}
if let Some(version) = target.java_version {
step.args.push("--release".into());
step.args.push(version.to_string());
}
step.args.push(file.to_string_lossy().into_owned());
step.env = target.top.env.clone();
Ok(step)
}
/// The module's Java source folders that exist, generated ones included; test sources see
/// the main ones too.
fn source_roots(target: &SingleFileTarget) -> Vec<PathBuf> {
let scopes: &[(&str, &str)] = if target.test {
&[("src/test/java", "target/generated-test-sources"), ("src/main/java", "target/generated-sources")]
} else {
&[("src/main/java", "target/generated-sources")]
};
let mut roots = Vec::new();
for (src, generated) in scopes {
roots.push(target.module.join(src));
let generated = target.module.join(generated);
let mut dirs: Vec<PathBuf> =
fs::read_dir(&generated).into_iter().flatten().flatten().map(|d| d.path()).filter(|p| p.is_dir()).collect();
dirs.sort();
roots.extend(dirs);
}
roots.retain(|p| p.is_dir());
roots
}
/// Resolves the classpath and plans the `javac` invocation for one Java source.
pub fn compile_file_plan(root: &Path, model: &BuildModel, file: &Path) -> IdeResult<BuildPlan> {
let target = single_file_target(root, model, file)?;
let classpath = single_file_classpath(root, model, &target)?;
let javac = find_javac(&target.top.env);
let name = file.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default();
Ok(BuildPlan {
system: BuildSystem::Maven,
title: format!("Compile {name}"),
steps: vec![javac_step(&target, file, &classpath, &javac)?],
})
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version>
<packaging>pom</packaging><modules><module>lib</module><module>app</module></modules></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>lib</artifactId></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>app</artifactId><dependencies><dependency><groupId>demo</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(root, "app/src/main/java/demo/Main.java", "class Main {}");
write(root, "app/src/test/java/demo/MainTest.java", "class MainTest {}");
write(root, "lib/target/classes/Lib.class", "built");
crate::load(root, &root.join("cache"), None)
}
#[test]
fn target_finds_the_owning_module_and_the_test_scope() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let main = single_file_target(root, &model, &root.join("app/src/main/java/demo/Main.java")).unwrap();
assert_eq!(main.module, root.join("app"));
assert!(!main.test);
assert_eq!(main.out_dir, root.join("app/target/classes"));
let test = single_file_target(root, &model, &root.join("app/src/test/java/demo/MainTest.java")).unwrap();
assert!(test.test);
assert_eq!(test.out_dir, root.join("app/target/test-classes"));
}
#[test]
fn target_rejects_non_java_missing_and_unlinked_files() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "app/pom.xml", "<project/>");
assert!(single_file_target(root, &model, &root.join("app/pom.xml")).is_err());
assert!(single_file_target(root, &model, &root.join("app/src/main/java/demo/Missing.java")).is_err());
write(root, "other/Main.java", "class Main {}");
assert!(single_file_target(root, &model, &root.join("other/Main.java")).is_err());
}
#[test]
fn javac_prefers_the_module_jdk_and_scopes_test_sources() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let file = root.join("app/src/test/java/demo/MainTest.java");
let target = single_file_target(root, &model, &file).unwrap();
let jdk = root.join("jdk");
fs::create_dir_all(jdk.join("bin")).unwrap();
let exe = if cfg!(windows) { "javac.exe" } else { "javac" };
fs::write(jdk.join("bin").join(exe), "fake").unwrap();
let mut target = target;
target.top.env.insert("JAVA_HOME".into(), jdk.to_string_lossy().into_owned());
let step = javac_step(&target, &file, "/repo/a.jar", &find_javac(&target.top.env)).unwrap();
assert!(step.program.ends_with(exe));
assert!(target.out_dir.is_dir());
let args = step.args.join(" ");
assert!(args.contains("-encoding UTF-8"), "{args}");
assert!(args.contains("-d ") && args.contains("test-classes"), "{args}");
assert!(args.contains("-cp /repo/a.jar"), "{args}");
assert!(args.contains("MainTest.java"), "{args}");
// Test sources resolve their own and the main sources, without writing them.
let sourcepath = step.args.iter().position(|a| a == "-sourcepath").map(|i| step.args[i + 1].clone());
let roots = sourcepath.expect("a -sourcepath");
assert!(roots.contains("src/test/java") && roots.contains("src/main/java"), "{roots}");
assert!(step.args.iter().any(|a| a == "-implicit:none"), "{args}");
// No `JAVA_HOME`: plain `javac` from `PATH`.
assert_eq!(find_javac(&BTreeMap::new()), exe);
}
/// The classpath probe runs the project's Maven command with `-pl … -am`
/// `dependency:build-classpath`; the fake `mvnw` below only records its
/// arguments and writes a canned classpath file for the `-pl` module.
#[cfg(unix)]
#[test]
fn classpath_probe_writes_one_file_per_module_and_reads_the_compiled_one() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
fixture(root);
let script = root.join("mvnw");
fs::write(
&script,
"#!/bin/sh\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/a.jar:/repo/b.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
let model = crate::load(root, &root.join("cache"), None);
let file = root.join("app/src/main/java/demo/Main.java");
let target = single_file_target(root, &model, &file).unwrap();
assert!(target.top.command.ends_with("mvnw"));
assert_eq!(target.pl, "app");
let cp = single_file_classpath(root, &model, &target).unwrap();
let separator = ':';
// Reactor class directories first, so fresh outputs shadow installed snapshots.
let lib = root.join("lib/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&lib), "{cp}");
// The module's own classes come first once they exist.
fs::create_dir_all(root.join("app/target/classes")).unwrap();
let cp = single_file_classpath(root, &model, &target).unwrap();
let own = root.join("app/target/classes").to_string_lossy().into_owned();
assert!(cp.starts_with(&format!("{own}:{lib}")), "{cp}");
assert!(cp.contains("/repo/a.jar"), "{cp}");
assert!(cp.contains("/repo/b.jar"), "{cp}");
assert!(cp.split(separator).count() >= 3, "{cp}");
}
}
+253
Переглянути файл
@@ -0,0 +1,253 @@
//! Up-to-date checks for menu builds (`Build Project` / `Build Module`, #2476).
//!
//! Menu builds have no fingerprint record on the launch path, so a second
//! click always pays a full offline Maven run. This module stores one JSON
//! record per scope (project, or one module file's build) after a successful
//! build and compares it before the next build; on a match the build is
//! skipped entirely. Like launch reuse, timestamp-preserving external edits
//! are re-stamped first so Maven actually recompiles them on the next run.
use super::*;
use std::fs;
/// Record of the last successful menu build of one scope.
#[derive(serde::Serialize, serde::Deserialize)]
struct Record {
fingerprint: LaunchFingerprint,
/// Output folders of the built projects that existed after the build, relative to the
/// project. Deleting one outside the IDE (`mvn clean` in a terminal) leaves every input
/// unchanged, so a build is skipped only while they are all still there.
#[serde(default)]
outputs: Vec<PathBuf>,
}
/// The build's output folders that exist now, relative to `root`.
fn existing_outputs(root: &Path, fingerprint: &LaunchFingerprint) -> Vec<PathBuf> {
fingerprint
.project_dirs()
.flat_map(|dir| ["target/classes", "target/test-classes"].map(|out| dir.join(out)))
.filter(|rel| root.join(rel).is_dir())
.collect()
}
fn record_path(cache: &Path, root: &Path, file: Option<&Path>) -> PathBuf {
let base = cache.join("menu-build").join("records");
match file {
Some(f) => base.join(f.strip_prefix(root).unwrap_or(f)).with_extension("json"),
None => base.join("project.json"),
}
}
/// Re-stamps files whose content changed without advancing mtime (git
/// checkout, external sync): timestamp-incremental Maven would otherwise skip
/// them as "unchanged" and the rebuild would compile nothing.
fn restamp(root: &Path, edits: &[PathBuf]) {
let now = std::time::SystemTime::now();
for rel in edits {
let path = root.join(rel);
let _ = fs::File::options().write(true).open(&path).and_then(|f| f.set_modified(now)).or_else(|_| {
fs::OpenOptions::new()
.create(true)
.truncate(false)
.write(true)
.open(&path)
.and_then(|f| f.set_modified(now))
});
}
}
/// Whether the last successful menu build of this scope is still fresh: its
/// record exists and the fingerprint matches. On a mismatch caused by
/// timestamp-preserving external edits the files are re-stamped first, so the
/// rebuild Maven runs actually recompiles them.
pub fn menu_up_to_date(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) -> bool {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return false,
};
let bytes = match fs::read(record_path(cache, root, file)) {
Ok(bytes) => bytes,
_ => return false,
};
let record: Record = match serde_json::from_slice(&bytes) {
Ok(record) => record,
_ => return false,
};
if record.fingerprint != current {
restamp(root, &record.fingerprint.mtime_preserved_edits(&current));
return false;
}
record.outputs.iter().all(|out| root.join(out).is_dir())
}
/// Records a successful menu build as the baseline for future up-to-date
/// checks. Failures are swallowed: a missing record only costs one Maven run.
pub fn record_menu_build(
cache: &Path,
root: &Path,
model: &BuildModel,
file: Option<&Path>,
cancel: &CancellationToken,
) {
let current = match menu_fingerprint(root, model, file, cancel) {
Ok(Some(current)) => current,
_ => return,
};
// A torn write fails to parse later and is treated as "no record", so a
// plain write is enough.
let path = record_path(cache, root, file);
if path.parent().is_some_and(|p| fs::create_dir_all(p).is_err()) {
return;
}
let outputs = existing_outputs(root, &current);
let bytes = match serde_json::to_vec(&Record { fingerprint: current, outputs }) {
Ok(bytes) => bytes,
_ => return,
};
let _ = fs::write(&path, bytes);
}
#[cfg(test)]
mod tests {
use super::*;
fn write(root: &Path, rel: &str, value: &str) {
let path = root.join(rel);
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, value).unwrap();
}
fn fixture(root: &Path) -> BuildModel {
maven_two_module(root, "int x;");
BuildModel { maven: Some(maven::load(root)), ..Default::default() }
}
fn maven_two_module(root: &Path, extra_app: &str) {
write(
root,
"pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><groupId>t</groupId><artifactId>root</artifactId><version>1</version><packaging>pom</packaging><modules><module>app</module><module>lib</module></modules></project>"#,
);
write(
root,
"app/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>app</artifactId><dependencies><dependency><groupId>t</groupId><artifactId>lib</artifactId><version>1</version></dependency></dependencies></project>"#,
);
write(
root,
"lib/pom.xml",
r#"<project><modelVersion>4.0.0</modelVersion><parent><groupId>t</groupId><artifactId>root</artifactId><version>1</version></parent><artifactId>lib</artifactId></project>"#,
);
write(root, "app/src/main/java/t/A.java", &format!("package t; public class A {{ {extra_app} }}"));
write(root, "lib/src/main/java/t/L.java", "package t; public class L {}");
}
#[test]
fn record_then_skip_without_changes() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
// The project scope is independent of the module scope.
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn deleted_outputs_invalidate_the_record() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let cancel = CancellationToken::new();
for out in ["app/target/classes/t", "lib/target/classes/t", "app/target/test-classes"] {
fs::create_dir_all(root.join(out)).unwrap();
}
record_menu_build(&cache, root, &model, None, &cancel);
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
// `mvn clean` outside the IDE: no input changed, but the classes are gone.
fs::remove_dir_all(root.join("lib/target")).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, None, &cancel));
fs::create_dir_all(root.join("lib/target/classes")).unwrap();
assert!(menu_up_to_date(&cache, root, &model, None, &cancel));
}
#[test]
fn edit_invalidates_and_rerecord_restores_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
write(root, "app/src/main/java/t/A.java", "package t; public class A { int y; }");
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
record_menu_build(&cache, root, &model, Some(&file), &cancel);
assert!(menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn mtime_preserved_edit_forces_rebuild_and_restamps() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let rel = "app/src/main/java/t/A.java";
let file = root.join(rel);
let cancel = CancellationToken::new();
record_menu_build(&cache, root, &model, Some(&file), &cancel);
let before = fs::metadata(&file).unwrap().modified().unwrap();
// Same length, same mtime, different content: an external edit that
// preserved the timestamp.
write(root, rel, "package t; public class A { int z; }");
fs::File::options().write(true).open(&file).and_then(|f| f.set_modified(before)).unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
assert!(fs::metadata(&file).unwrap().modified().unwrap() > before);
}
#[test]
fn corrupt_record_never_skips() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cache = root.join("cache");
let file = root.join("app/src/main/java/t/A.java");
let cancel = CancellationToken::new();
fs::create_dir_all(cache.join("menu-build").join("records").join("app/src/main/java/t")).unwrap();
fs::write(cache.join("menu-build").join("records").join("app/src/main/java/t/A.json"), b"{not json").unwrap();
assert!(!menu_up_to_date(&cache, root, &model, Some(&file), &cancel));
}
#[test]
fn unsupported_builds_never_skip() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
write(root, "other/Main.java", "public class Main {}");
let cache = root.join("cache");
let cancel = CancellationToken::new();
// A file outside any non-aggregator module has no fingerprint.
let stray = root.join("other/Main.java");
record_menu_build(&cache, root, &model, Some(&stray), &cancel);
assert!(!menu_up_to_date(&cache, root, &model, Some(&stray), &cancel));
}
}
+22 -1
Переглянути файл
@@ -8,6 +8,24 @@ def canon = { File f -> f.canonicalPath }
// the identity path (`:lib:core`, `:buildSrc`) tells them apart and addresses their tasks
// from the root build. Internal API, hence the fallback.
def identity = { Project p -> try { p.identityPath.toString() } catch (ignored) { p.path } }
// The task copying a source set's resources and its output folder, when it changes them on the
// way (filters, `expand`, renames, includes / excludes, extra sources): jdtls copies resources as
// they are, so launches take them from Gradle. Internal API, hence the fallback.
def processedResources = { Project p, s ->
try {
def task = p.tasks.findByName(s.processResourcesTaskName)
if (task == null) return null
def specs = 0
def changes = task.mainSpec.sourcePaths.size() > 1
task.rootSpec.buildRootResolver().walk { r ->
specs++
changes |= !r.allCopyActions.isEmpty() || !r.allIncludes.isEmpty() || !r.allExcludes.isEmpty() ||
!r.allIncludeSpecs.isEmpty() || !r.allExcludeSpecs.isEmpty() || r.destPath.segments.length > 0
}
// The root and main specs exist for every task; more come from `from(…) { … }`.
(changes || specs > 2) ? [task: task.name, dir: canon(task.destinationDir)] : null
} catch (ignored) { null }
}
def describe = { Project p ->
def m = [path: identity(p), dir: canon(p.projectDir), buildDir: canon(p.layout.buildDirectory.get().asFile)]
@@ -26,7 +44,10 @@ def describe = { Project p ->
def test = s.name == 'test' || s.name == 'testFixtures' || suites.contains(s.name) ||
s.name.endsWith('Test') || s.name.endsWith('Tests') ||
(!src.isEmpty() && src.every { ideaTests.contains(it) })
[name: s.name, test: test, src: src, res: res]
def set = [name: s.name, test: test, src: src, res: res]
def processed = processedResources(p, s)
if (processed != null) set.processed = processed
set
}
}
try {
+13
Переглянути файл
@@ -335,6 +335,19 @@ pub struct ReportedSourceSet {
/// Absolute resource folders.
#[serde(default)]
pub res: Vec<String>,
/// Set when Gradle changes the resources while copying them (filters, `expand`, renames,
/// includes / excludes, extra sources), which the Java language server's copy does not.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub processed: Option<ProcessedResources>,
}
/// The task that copies a source set's resources, and where it writes them.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ProcessedResources {
/// Task name in its project (`processResources`).
pub task: String,
/// Absolute output folder.
pub dir: String,
}
const MODEL_LINE: &str = "@openidea-model\t";
+1
Переглянути файл
@@ -536,6 +536,7 @@ mod tests {
crate::maven::LOCAL_REPO_TRACKING_ARG,
"-o",
"-Peditor",
"-Djacoco.skip=true",
"test-compile"
]
);
+25 -3
Переглянути файл
@@ -73,6 +73,9 @@ struct GradleCache {
/// The projects as Gradle evaluated them; empty in caches from before they were read.
#[serde(default)]
projects: Vec<ReportedProject>,
/// The projects report how their resources are processed; older caches are stale.
#[serde(default)]
resources: bool,
}
/// Init script printing every project, its source sets and Java version once configured.
@@ -244,11 +247,12 @@ pub fn apply_gradle_cache(root: &Path, cache_dir: &Path, model: &mut GradleModel
}
}
}
// A successful sync from before projects were cached has not seen computed includes.
let without_projects = cache.error.is_none() && cache.projects.is_empty();
// A successful sync from before projects were cached has not seen computed includes,
// nor, before resources were read, which launches need Gradle to process them.
let outdated = cache.error.is_none() && (cache.projects.is_empty() || !cache.resources);
model.synced_at = Some(cache.at_ms);
model.sync_error = cache.error;
model.stale = without_projects || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
model.stale = outdated || cache.hash != hash_gradle_files(&gradle::build_files(root, model));
}
// ---------------------------------------------------------------- processes
@@ -462,6 +466,7 @@ pub fn sync_gradle_model(
error: (!out.success).then(|| error_summary(&out.text)),
tasks: if out.success { gradle::parse_tasks(&out.text) } else { Vec::new() },
projects,
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache)
}
@@ -713,6 +718,7 @@ mod tests {
test,
src: src.iter().map(|p| core_src(p)).collect(),
res: res.iter().map(|p| core_src(p)).collect(),
processed: None,
};
// A settings script's own println comes first; task listing lines follow.
let out = format!(
@@ -737,6 +743,15 @@ mod tests {
);
let projects = parse_reported_projects(&out);
assert_eq!(projects.len(), 3, "the malformed line is skipped");
// As the init script prints a set whose resources Gradle filters or renames.
let processed = parse_reported_projects(&format!(
"@openidea-model\t{}",
r#"{"path":":","dir":"/p","sourceSets":[{"name":"main","test":false,"src":[],"res":[],"processed":{"task":"processResources","dir":"/p/build/resources/main"}}]}"#
));
assert_eq!(
processed[0].source_sets[0].processed,
Some(gradle::ProcessedResources { task: "processResources".into(), dir: "/p/build/resources/main".into() })
);
let src = |p: &str| PathBuf::from(core_src(p));
assert_eq!(
gradle::reported_source_roots(&projects),
@@ -802,12 +817,18 @@ mod tests {
..Default::default()
})
.collect(),
resources: true,
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut model);
assert_eq!(model.projects[1].tasks.len(), 1);
assert!(model.projects[0].tasks.iter().any(|t| t.name == "wrapper"), "root keeps its defaults");
assert!(!model.stale);
write_cache(&cache_dir, GRADLE_CACHE, &GradleCache { resources: false, ..cache.clone() }).unwrap();
let mut m = gradle::load(&root).unwrap();
apply_gradle_cache(&root, &cache_dir, &mut m);
assert!(m.stale, "synced before resource processing was read");
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
let stale_after = |script: &str| {
std::fs::write(root.join("app/build.gradle"), script).unwrap();
@@ -841,6 +862,7 @@ mod tests {
dir: root.to_string_lossy().into_owned(),
..Default::default()
}],
resources: true,
..Default::default()
};
write_cache(&cache_dir, GRADLE_CACHE, &cache).unwrap();
+20
Переглянути файл
@@ -333,6 +333,23 @@ pub struct InlayHintsSettings {
pub code_vision_usages: bool,
/// Code vision: last author of a declaration (VCS blame) beside its usages.
pub code_vision_author: bool,
/// Code vision: "no usages" on declarations nothing uses (IDEA shows it; clicking searches everywhere).
pub code_vision_no_usages: bool,
/// Code vision: where the usages go, above the declaration or at the end of its line.
pub code_vision_usages_position: CodeVisionPosition,
/// Code vision: where the author goes, above the declaration or at the end of its line.
pub code_vision_author_position: CodeVisionPosition,
}
/// Editor › Inlay Hints › Code Vision position (IDEA's "Default position for metrics").
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum CodeVisionPosition {
/// A row of its own above the declaration.
#[default]
Top,
/// After the end of the declaration's first line.
Right,
}
impl Default for InlayHintsSettings {
@@ -344,6 +361,9 @@ impl Default for InlayHintsSettings {
disabled_languages: Vec::new(),
code_vision_usages: true,
code_vision_author: true,
code_vision_no_usages: true,
code_vision_usages_position: CodeVisionPosition::Top,
code_vision_author_position: CodeVisionPosition::Top,
}
}
}
+50
Переглянути файл
@@ -273,6 +273,32 @@ pub enum DebugEvent {
change_type: String,
message: String,
},
/// The session's capabilities changed after it started: fetch them again.
Capabilities,
/// The adapter's `invalidated` event: what it shows for `areas` (`all`, `stacks`, `threads`,
/// `variables`; empty means all) is stale and has to be fetched again.
Invalidated {
areas: Vec<String>,
},
JvmBackend {
state: JvmBackendState,
},
}
/// The backend that reads a debugged JVM's objects (instances, fields, referrers).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmBackendState {
Ready,
/// Turned off for the rest of the session: the debug adapter lacks what it needs.
Unsupported {
detail: String,
},
/// The Kotlin JDWP link only forwards now; inspecting stops.
Degraded {
detail: String,
},
Closed,
}
/// Where backend crates publish [`IdeEvent`]s. Cheap to clone via `Arc`.
@@ -294,3 +320,27 @@ pub struct NullSink;
impl EventSink for NullSink {
fn emit(&self, _: IdeEvent) {}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn jvm_backend_events_json_shape() {
let degraded = DebugEvent::JvmBackend { state: JvmBackendState::Degraded { detail: "x".into() } };
assert_eq!(
serde_json::to_value(&degraded).unwrap(),
json!({"kind": "jvmBackend", "state": {"kind": "degraded", "detail": "x"}})
);
assert_eq!(serde_json::to_value(DebugEvent::Capabilities).unwrap(), json!({"kind": "capabilities"}));
}
#[test]
fn invalidated_event_json_shape() {
assert_eq!(
serde_json::to_value(DebugEvent::Invalidated { areas: vec!["variables".into()] }).unwrap(),
json!({"kind": "invalidated", "areas": ["variables"]})
);
}
}
+10 -2
Переглянути файл
@@ -12,11 +12,16 @@ use std::path::{Path, PathBuf};
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// `p` as the UI sees it: `C:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
/// `p` as the UI sees it: `C:\a\b`, `c:\a\b` and `\\?\C:\a\b` become `C:/a/b`.
pub fn to_ipc(p: &Path) -> String {
#[cfg(windows)]
{
dunce::simplified(p).to_string_lossy().replace('\\', "/")
let mut s = dunce::simplified(p).to_string_lossy().replace('\\', "/");
// The UI compares path strings; a drive letter is case-insensitive.
if s.as_bytes().get(1) == Some(&b':') {
s[..1].make_ascii_uppercase();
}
s
}
#[cfg(not(windows))]
{
@@ -144,5 +149,8 @@ mod tests {
assert_eq!(to_ipc(Path::new(r"\\?\C:\a\b")), "C:/a/b");
assert_eq!(to_ipc(&Path::new("C:/a").join("b")), "C:/a/b");
assert_eq!(serde_json::to_string(&IpcPath::from(r"C:\a\b")).unwrap(), r#""C:/a/b""#);
assert_eq!(to_ipc(Path::new(r"c:\a\b")), "C:/a/b");
assert_eq!(to_ipc(Path::new(r"\\?\d:\a")), "D:/a");
assert_eq!(to_ipc(Path::new(r"\\server\share\a")), "//server/share/a");
}
}
+26
Переглянути файл
@@ -21,6 +21,32 @@
use std::collections::BTreeMap;
/// Removes ANSI escape sequences (colour, cursor movement).
pub fn strip_ansi(s: &str) -> String {
if !s.contains('\u{1b}') {
return s.to_owned();
}
let mut out = String::with_capacity(s.len());
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.peek() == Some(&'[') {
chars.next();
for c in chars.by_ref() {
if c.is_ascii_alphabetic() || c == '~' {
break;
}
}
} else {
chars.next();
}
}
out
}
/// Decodes complete lines or collected output from IDE-managed tools, preferring UTF-8.
/// Windows launcher/native diagnostics fall back to the system ANSI code page.
/// Callers must buffer incomplete lines so multibyte characters are never split.
+112
Переглянути файл
@@ -72,6 +72,75 @@ pub fn kill_tree_of(pid: u32) {
let _ = pid;
}
/// Whether a process with this id exists. PID reuse makes this conservative: a reused id keeps
/// whatever was tied to the old process alive, a live process is never reported dead.
pub fn process_alive(pid: u32) -> bool {
// `kill(0, 0)` would signal the caller's own process group.
if pid == 0 {
return false;
}
#[cfg(unix)]
{
// Above `i32::MAX` the id would wrap to a negative value, which `kill` reads as a group.
let Ok(pid) = libc::pid_t::try_from(pid) else { return false };
// SAFETY: signal 0 delivers nothing; it only checks that the process exists and is reachable.
unsafe { libc::kill(pid, 0) == 0 || std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM) }
}
#[cfg(windows)]
{
use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle};
use windows_sys::Win32::Foundation::{ERROR_ACCESS_DENIED, GetLastError, STILL_ACTIVE};
use windows_sys::Win32::System::Threading::{
GetExitCodeProcess, OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION,
};
// SAFETY: plain call; a non-null result is a process handle owned by this function.
let handle = unsafe { OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, 0, pid) };
if handle.is_null() {
// A process we may not open still exists.
// SAFETY: reads the thread's last error, set by the call above.
return unsafe { GetLastError() } == ERROR_ACCESS_DENIED;
}
// SAFETY: `handle` is valid and nobody else owns it.
let handle = unsafe { OwnedHandle::from_raw_handle(handle) };
let mut code = 0u32;
// SAFETY: a live process handle opened with query access; one out-parameter.
let ok = unsafe { GetExitCodeProcess(handle.as_raw_handle(), &mut code) };
// A handle that cannot be queried is not evidence that the process is gone.
ok == 0 || code == STILL_ACTIVE as u32
}
#[cfg(not(any(unix, windows)))]
{
true
}
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
pub fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// [`kill_tree`] that first asks the group to exit: `SIGTERM`, then `grace`, then `SIGKILL`.
/// Git removes its `index.lock` on `SIGTERM`, where `SIGKILL` leaves it behind and the next
/// git command fails. Windows has no such signal to send from outside the child's console,
@@ -730,6 +799,30 @@ fn append_path_entries(path: &OsStr, extra: &OsStr) -> OsString {
mod tests {
use super::*;
#[test]
fn process_alive_tells_live_from_dead_pids() {
assert!(process_alive(std::process::id()));
assert!(!process_alive(0));
let mut cmd = if cfg!(windows) {
let mut cmd = Command::new("cmd");
cmd.args(["/C", "exit 0"]);
cmd
} else {
Command::new("true")
};
let mut child = cmd.spawn().unwrap();
let pid = child.id();
child.wait().unwrap();
drop(child);
assert!(!process_alive(pid));
}
#[cfg(unix)]
#[test]
fn process_alive_rejects_ids_that_are_not_a_single_process() {
assert!(!process_alive(u32::MAX));
}
#[cfg(unix)]
#[test]
fn kill_tree_of_reaches_what_the_child_started() {
@@ -1050,6 +1143,25 @@ mod tests {
}
assert_eq!(find_executable("nope", &search), None);
}
#[cfg(unix)]
#[test]
fn process_tree_lists_the_child_of_a_shell() {
let mut cmd = Command::new("sh");
own_group(&mut cmd).args(["-c", "sleep 5 & wait"]);
let mut child = cmd.spawn().unwrap();
let sh_pid = child.id() as i32;
// The shell forks `sleep` after it starts, so the tree grows a moment later.
let deadline = std::time::Instant::now() + Duration::from_secs(2);
let mut tree = process_tree(sh_pid);
while tree.len() < 2 && std::time::Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(20));
tree = process_tree(sh_pid);
}
kill_tree(&mut child);
assert_eq!(tree[0], sh_pid, "the pid itself comes first");
assert!(tree.len() >= 2 && tree[1..].iter().all(|p| *p != sh_pid), "the shell's child follows: {tree:?}");
}
}
#[cfg(all(test, unix))]
+13 -1
Переглянути файл
@@ -63,6 +63,8 @@ pub struct Settings {
pub agents: AgentSettings,
/// Agents › MCP Server (global only).
pub mcp_server: McpServerSettings,
/// Tools › Agents: Command Execution Policy.
pub agent_security: AgentSecuritySettings,
/// Plugin settings (`[contributes.settings]`): plugin id → its values by key.
#[specta(type = BTreeMap<String, BTreeMap<String, specta_typescript::Unknown>>)]
pub plugins: BTreeMap<String, serde_json::Map<String, serde_json::Value>>,
@@ -261,6 +263,9 @@ pub enum WhileRunning {
pub struct McpServerSettings {
/// Serve the tools on 127.0.0.1 while the IDE runs.
pub enabled: bool,
/// Run the tools that change the project or start processes without asking first
/// (IDEA's brave mode). Read from the IDE's own settings only, never a project's.
pub run_without_confirmation: bool,
}
/// Absent profiles retain automatic discovery for existing projects.
@@ -408,6 +413,7 @@ impl Default for Settings {
cmake: CMakeSettings::default(),
agents: AgentSettings::default(),
mcp_server: McpServerSettings::default(),
agent_security: AgentSecuritySettings::default(),
plugins: BTreeMap::new(),
}
}
@@ -453,6 +459,10 @@ pub struct GeneralSettings {
/// Where updates come from: `auto` (the CNB mirror for the Chinese UI, else GitHub) | `github` | `cnb`.
pub update_source: String,
pub auto_check_updates: bool,
/// System Settings › Updates: open the What's New page after the app starts a new version.
pub show_whats_new_after_update: bool,
/// The version the app last started with, so the first start after an update is recognisable.
pub last_run_version: String,
}
impl Default for GeneralSettings {
@@ -471,6 +481,8 @@ impl Default for GeneralSettings {
project_name: String::new(),
update_source: "auto".into(),
auto_check_updates: true,
show_whats_new_after_update: true,
last_run_version: String::new(),
}
}
}
@@ -574,7 +586,7 @@ pub struct AppearanceSettings {
pub project_colors_in_toolbar: bool,
pub dnd_with_alt_only: bool,
pub menu_icons: bool,
/// `hamburger` | `merged`: main menu presentation in the frameless title bar.
/// `hamburger` | `merged` | `toolbar` (a separate row): main menu presentation in the frameless title bar.
pub main_menu_style: String,
/// A lone Alt press focuses or opens the main menu (IDEA `ide.windows.main.menu.focus.on.alt`).
pub main_menu_alt: bool,
+35 -2
Переглянути файл
@@ -138,8 +138,11 @@ impl WorkspaceEdit {
continue;
}
*expected = expected.or(version);
// Repeats within one entry are intended (LSP inserts at one position
// apply in order); only an alias entry repeats what is already merged.
let merged = own.len();
for edit in edits {
if !own.contains(&edit) {
if !own[..merged].contains(&edit) {
own.push(edit);
}
}
@@ -277,10 +280,40 @@ pub struct OutlineItem {
/// Fuzzy-match highlight: indices (in chars of the displayed string) that matched.
pub type MatchIndices = Vec<u32>;
#[cfg(all(test, unix))]
#[cfg(test)]
mod workspace_edit_tests {
use super::*;
#[test]
fn repeated_edits_within_one_entry_survive_but_repeats_across_entries_do_not() {
let path = PathBuf::from("missing-dir/main.ts");
let at = |line, text: &str| TextEdit {
range: Range::new(Pos::new(line, 0), Pos::new(line, 0)),
new_text: text.into(),
};
let entry = |edits: Vec<TextEdit>| FileEdit::Edit { path: path.clone().into(), edits, version: None };
let (nl, brace, other) = (at(0, "\n"), at(1, "}"), at(2, "x"));
for (entries, expected) in [
(vec![vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![brace.clone(), nl.clone(), brace.clone()]], vec![brace.clone(), nl.clone(), brace.clone()]),
(vec![vec![nl.clone(), nl.clone()], vec![nl.clone(), nl.clone()]], vec![nl.clone(), nl.clone()]),
(vec![vec![nl.clone()], vec![nl.clone(), other.clone()]], vec![nl.clone(), other.clone()]),
(
vec![vec![nl.clone()], vec![other.clone(), other.clone()]],
vec![nl.clone(), other.clone(), other.clone()],
),
] {
let mut workspace =
WorkspaceEdit { files: entries.clone().into_iter().map(entry).collect(), ..Default::default() };
workspace.deduplicate();
assert_eq!(workspace.files, [entry(expected.clone())], "{entries:?}");
let once = workspace.clone();
workspace.deduplicate();
assert_eq!(workspace, once, "{entries:?}");
}
}
#[cfg(unix)]
#[test]
fn aliases_are_one_edit_but_resource_operations_preserve_order() {
let dir = tempfile::tempdir().unwrap();
+29
Переглянути файл
@@ -322,6 +322,7 @@ pub struct TerminalSettings {
pub override_ide_shortcuts: bool,
pub shell_integration: bool,
pub highlight_hyperlinks: bool,
pub highlight_selection: bool,
pub option_as_meta: bool,
/// Escape moves focus to the editor instead of reaching the shell.
pub escape_to_editor: bool,
@@ -354,6 +355,7 @@ impl Default for TerminalSettings {
override_ide_shortcuts: true,
shell_integration: true,
highlight_hyperlinks: true,
highlight_selection: true,
option_as_meta: false,
escape_to_editor: false,
cursor_shape: "block".into(),
@@ -410,3 +412,30 @@ impl Default for ExternalTool {
}
}
}
/// Tools › Agents: Command Execution Policy (whitelist and blacklist patterns).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct AgentSecuritySettings {
/// Commands or patterns (`git status*`, `pwd`) automatically approved without prompting.
pub allowed_commands: Vec<String>,
/// Commands or patterns (`rm -rf *`, `mkfs*`) automatically rejected.
pub blocked_commands: Vec<String>,
}
impl Default for AgentSecuritySettings {
fn default() -> Self {
Self {
allowed_commands: Vec::new(),
blocked_commands: vec![
"rm -rf *".into(),
"rm -rf /".into(),
"mkfs*".into(),
"dd *".into(),
":(){ :|:& };:".into(),
"shutdown*".into(),
"reboot*".into(),
],
}
}
}
+5
Переглянути файл
@@ -11,12 +11,17 @@ tempfile.workspace = true
ide-core.workspace = true
ide-language.workspace = true
ide-jsonrpc.workspace = true
ide-jvm-attach.workspace = true
ide-jvm-inspect.workspace = true
ide-jdwp.workspace = true
ide-heap.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
tokio = { workspace = true, features = ["net"] }
parking_lot.workspace = true
memchr = "2"
percent-encoding = "2"
portable-pty.workspace = true
tracing.workspace = true
+7
Переглянути файл
@@ -72,6 +72,13 @@ fn builtin(adapter: &str) -> Option<DebuggerConfig> {
"stdio",
"Install Kotlin Debug Adapter in Settings › Languages & Frameworks › Language Servers",
),
// Inside the language server: reached through `debugServer`, never started.
"kotlin-lsp" => cfg(
"kotlin-lsp",
&[],
"tcp",
"Install JetBrains Kotlin LSP in Settings › Languages & Frameworks › Language Servers",
),
// Windows installs `python.exe`; `python3.exe` there is at most a Store alias.
"debugpy" => cfg(
if cfg!(windows) { "python" } else { "python3" },
+12
Переглянути файл
@@ -342,4 +342,16 @@ mod tests {
assert_eq!(error_message(&msg), "cannot find a.py");
assert_eq!(error_message(&json!({"message": "nope"})), "nope");
}
#[test]
fn a_failed_launch_response_keeps_delves_too_old_error_verbatim() {
// The UI's repair offer keys off this exact wording (`missingAdapter`), so the
// conversion to the IPC error must not fold it into a generic message.
let message = "Failed to launch /p/api: Version of Delve is too old for Go version go1.27.0 \
(maximum supported version 1.26, suppress this error with --check-go-version=false)";
let e = ide_core::IdeError::from(DapError::Failed { command: "launch".into(), message: message.into() });
assert_eq!(e.kind, ide_core::error::ErrorKind::InvalidInput);
assert_eq!(e.message, format!("launch failed: {message}"));
assert!(e.message.contains("Version of Delve is too old for Go version go1.27.0"));
}
}
+100 -13
Переглянути файл
@@ -12,6 +12,35 @@ use tokio::net::{TcpListener, TcpStream};
/// How long the relay waits for java-debug to attach once the JVM is connected.
const ATTACH_TIMEOUT: Duration = Duration::from_secs(60);
/// Loopback ports relaying a connected JVM, while their relay runs.
static RELAY_PORTS: std::sync::Mutex<Vec<u16>> = std::sync::Mutex::new(Vec::new());
fn relay_ports() -> std::sync::MutexGuard<'static, Vec<u16>> {
RELAY_PORTS.lock().unwrap_or_else(std::sync::PoisonError::into_inner)
}
/// Whether `port` is a relay of [`listen_for_jvm`]: a session attaching to it debugs a JVM that
/// connected to the IDE, so the JVM has no JDWP listener the memory view could find it by.
pub(crate) fn is_listen_relay(port: u16) -> bool {
relay_ports().contains(&port)
}
/// Registers a relay port for as long as it lives.
struct RelayPort(u16);
impl RelayPort {
fn new(port: u16) -> Self {
relay_ports().push(port);
Self(port)
}
}
impl Drop for RelayPort {
fn drop(&mut self) {
relay_ports().retain(|&p| p != self.0);
}
}
/// The address to listen on for `host`: the interface it names, or every interface when
/// it is not one of this machine's (a DNS name or a NAT address the JVM connects to).
async fn bind(host: &str, port: u16) -> IdeResult<TcpListener> {
@@ -40,11 +69,11 @@ pub async fn listen_for_jvm(host: &str, port: u16, cancel: CancellationToken) ->
let _ = jvm.set_nodelay(true);
let relay = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = relay.local_addr()?.port();
tokio::spawn(relay_one(relay, jvm));
tokio::spawn(relay_one(relay, jvm, RelayPort::new(local)));
Ok(local)
}
async fn relay_one(relay: TcpListener, mut jvm: TcpStream) {
async fn relay_one(relay: TcpListener, mut jvm: TcpStream, _registered: RelayPort) {
let debugger = match tokio::time::timeout(ATTACH_TIMEOUT, relay.accept()).await {
Ok(Ok((s, _))) => s,
_ => return,
@@ -58,22 +87,64 @@ async fn relay_one(relay: TcpListener, mut jvm: TcpStream) {
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use super::*;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
#[tokio::test]
async fn relays_the_jdwp_handshake_both_ways() {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
let waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
// The remote JVM (`server=n`) connects to the IDE.
let mut jvm = loop {
if let Ok(s) = TcpStream::connect(("127.0.0.1", port)).await {
break s;
/// A remote JVM (`server=n`) connected to [`listen_for_jvm`], and the relay port. A port
/// picked by binding to 0 can be taken by a parallel test before `listen_for_jvm` binds it,
/// so a failed listen is retried on a fresh one.
pub(crate) async fn connected_jvm() -> (TcpStream, u16) {
for _ in 0..20 {
let port = TcpListener::bind(("127.0.0.1", 0)).await.unwrap().local_addr().unwrap().port();
if let Some(connected) = jvm_on(port).await {
return connected;
}
}
panic!("listen_for_jvm never accepted a JVM");
}
/// One attempt of [`connected_jvm`]; `None` when the port turned out to be taken.
async fn jvm_on(port: u16) -> Option<(TcpStream, u16)> {
let mut waiting = tokio::spawn(listen_for_jvm("127.0.0.1", port, CancellationToken::new()));
let mut jvm = None;
for _ in 0..100 {
// A listen that failed to bind leaves nothing to connect to.
if waiting.is_finished() {
break;
}
// macOS does not refuse a connect to a port a socket holds without listening: the
// SYN goes unanswered until the connect times out.
let connect = TcpStream::connect(("127.0.0.1", port));
if let Ok(Ok(s)) = tokio::time::timeout(Duration::from_secs(1), connect).await {
jvm = Some(s);
break;
}
tokio::time::sleep(Duration::from_millis(20)).await;
};
let local = waiting.await.unwrap().unwrap();
}
// A connection to another test's listener never reaches `listen_for_jvm`.
let relay = tokio::time::timeout(Duration::from_secs(2), &mut waiting).await;
if let (Some(jvm), Ok(Ok(Ok(relay)))) = (jvm, relay) {
return Some((jvm, relay));
}
waiting.abort();
None
}
#[tokio::test]
async fn a_port_another_socket_holds_is_given_up() {
// Bound but not listening, as a parallel test's outgoing connection holds its local port.
let held = tokio::net::TcpSocket::new_v4().unwrap();
held.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = held.local_addr().unwrap().port();
let attempt = tokio::time::timeout(Duration::from_secs(5), jvm_on(port)).await;
assert!(matches!(attempt, Ok(None)), "an attempt on a taken port gives up instead of waiting forever");
}
#[tokio::test]
async fn relays_the_jdwp_handshake_both_ways() {
// The remote JVM (`server=n`) connects to the IDE.
let (mut jvm, local) = connected_jvm().await;
// java-debug attaches to the relay and starts the handshake.
let mut debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
debugger.write_all(b"JDWP-Handshake").await.unwrap();
@@ -85,6 +156,22 @@ mod tests {
assert_eq!(&buf, b"JDWP-Handshake");
}
#[tokio::test]
async fn relay_port_is_known_while_the_relay_runs() {
let (jvm, local) = connected_jvm().await;
assert!(is_listen_relay(local));
let debugger = TcpStream::connect(("127.0.0.1", local)).await.unwrap();
drop((jvm, debugger));
for _ in 0..200 {
if !is_listen_relay(local) {
return;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("relay port {local} still registered after the relay ended");
}
#[tokio::test]
async fn waiting_is_cancellable() {
let cancel = CancellationToken::new();
+92
Переглянути файл
@@ -0,0 +1,92 @@
//! The class table counted by the debugger (JDWP `InstanceCounts`) where no histogram can be
//! taken: JDK 8-16 while suspended, Windows, a remote JVM. Only while suspended, and slow on a
//! large heap (seconds on JDK 17+).
use super::inspect::Call;
use super::snapshot::diff_rows;
use super::{ClassCounts, JvmMemoryResult, JvmMemorySnapshot, JvmMemorySource, JvmMemoryState, JvmMemoryUnavailable};
use crate::session::Session;
use ide_core::IdeResult;
use ide_jvm_attach::HistogramRow;
use ide_jvm_inspect::ClassCount;
/// Reasons the debugger may be able to work around; the others (the program exited, the JVM
/// did not answer) would fail for it too.
pub(super) fn falls_back(reason: &JvmMemoryUnavailable) -> bool {
use JvmMemoryUnavailable as U;
matches!(
reason,
U::UnsafeWhileSuspended { .. }
| U::UnsupportedPlatform
| U::AttachedJvmNotFound
| U::ListenerNotRunning
| U::ListenModeNotSupported
| U::NoJvmProcess
)
}
/// `histogram` unless it is unavailable for a reason `session`'s inspector can work around.
pub(super) async fn or_debugger_counts(
state: &mut JvmMemoryState,
histogram: Result<JvmMemorySnapshot, JvmMemoryUnavailable>,
session: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let reason = match histogram {
Ok(snapshot) => return Ok(JvmMemoryResult::Ready { snapshot }),
Err(reason) => reason,
};
let unavailable = |can_count_when_paused| Ok(JvmMemoryResult::Unavailable { reason, can_count_when_paused });
let Some(session) = session.filter(|_| falls_back(&reason)) else {
return unavailable(false);
};
let call = match Call::begin(session, None).await {
Ok(call) => call,
Err(super::JvmInspectUnavailable::NotSuspended) => return unavailable(true),
Err(_) => return unavailable(false),
};
let started = std::time::Instant::now();
let result = call.run(call.inspector.class_counts(&call.target)).await;
match call.finish(result) {
Ok(counts) => {
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let previous = state.baseline(JvmMemorySource::Debugger);
let (snapshot, counts) = debugger_snapshot(counts, previous.as_deref(), elapsed_ms);
state.remember(JvmMemorySource::Debugger, counts);
Ok(JvmMemoryResult::Ready { snapshot })
}
Err(e) => {
tracing::debug!("debugger class counts: {e}");
let running = !session.suspended.get();
unavailable(running && session.jvm_inspector.lock().is_some())
}
}
}
/// Classes without instances are left out; most instances first.
pub(super) fn debugger_snapshot(
counts: Vec<ClassCount>,
previous: Option<&ClassCounts>,
elapsed_ms: u32,
) -> (JvmMemorySnapshot, ClassCounts) {
let rows = counts
.into_iter()
.filter(|c| c.instances > 0)
.map(|c| HistogramRow { name: c.name, module: None, instances: c.instances, bytes: 0 })
.collect();
let (mut rows, counts) = diff_rows(rows, previous);
for row in &mut rows {
row.delta_bytes = None;
}
rows.sort_by(|a, b| b.instances.cmp(&a.instances).then_with(|| a.name.cmp(&b.name)));
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: 0,
rows,
java_version: String::new(),
taken_while_suspended: true,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Debugger,
};
(snapshot, counts)
}
+293
Переглянути файл
@@ -0,0 +1,293 @@
//! A heap dump of a debugged JVM, written over its attach listener into a snapshot directory.
use super::snapshot::{AttachProbe, JdkProbe, attach_failure, gone, pick_jvm_pid};
use super::{
JvmMemoryState, JvmMemoryUnavailable, JvmTarget, histogram_pid, keep_found, lookup, resolve_jvm,
safe_while_suspended, session_target,
};
use crate::session::Session;
use ide_core::{CancellationToken, IdeError, IdeResult};
use ide_jvm_attach::{AttachError, HeapDumpReply};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::sync::Arc;
const DUMP_FILE: &str = "dump.hprof";
/// For a refusal whose reply holds no reason: empty, or only the line announcing the dump.
const NO_REASON: &str = "the JVM wrote no heap dump and gave no reason";
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmHeapUnavailable {
/// The reason a class table could not be read applies to the dump too.
Memory {
reason: JvmMemoryUnavailable,
},
/// Bytes.
NotEnoughDiskSpace {
needed: u64,
available: u64,
},
DumpFailed {
message: String,
},
DumpTimedOut {
seconds: u32,
},
}
#[derive(Clone, Debug, PartialEq)]
pub struct JvmHeapDump {
pub path: PathBuf,
/// The class histogram taken just before the dump.
pub histogram: Vec<ide_heap::HistogramEntry>,
/// Raw `VM.flags`; empty when the JVM did not answer.
pub vm_flags: String,
/// 0 when unknown.
pub java_major: u32,
/// Empty when unknown.
pub java_version: String,
pub pid: u32,
pub taken_while_suspended: bool,
pub dump_ms: u32,
}
#[derive(Clone, Debug, PartialEq)]
pub enum JvmHeapOutcome {
Ready(JvmHeapDump),
/// The snapshot directory has been deleted.
Unavailable(JvmHeapUnavailable),
/// The snapshot directory has been deleted.
Cancelled,
}
pub(crate) async fn session_heap_dump(
session: Arc<Session>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let launched = histogram_pid(&session);
let capture = {
let (dir, cancel) = (dir.clone(), cancel.clone());
async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
let stopped = session.clone();
let suspended = move || capture_suspended(&stopped);
heap_dump_with(&session.jvm_memory, suspended, probe, target, dir, cancel).await
}
};
detached(capture, dir, cancel).await
}
/// The JVM can be suspended before the adapter reports the stop, so a session with a JDWP link
/// also counts the link's view.
pub(super) fn capture_suspended(session: &Session) -> bool {
session.suspended.get()
|| session
.jdwp_link
.lock()
.as_ref()
.is_some_and(|link| matches!(link.state(), ide_jdwp::LinkState::Suspended { .. }))
}
/// Runs `capture` in a task of its own, so a caller that stops waiting cannot release the
/// session's lock while the JVM is still writing the dump and let a class-table request run
/// next to it. On `cancel` returns at once and deletes `dir`.
pub(super) async fn detached(
capture: impl Future<Output = IdeResult<JvmHeapOutcome>> + Send + 'static,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let task = tokio::spawn(capture);
tokio::select! {
joined = task => joined.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?,
() = cancel.cancelled() => {
discard_later(dir).await;
Ok(JvmHeapOutcome::Cancelled)
}
}
}
/// Holds `state`'s lock throughout, as a class-table request does; `suspended` and then `probe`
/// are read once it is held, and `suspended` again by [`take_heap_dump`]. Anything but `Ready`
/// deletes `dir`.
pub(super) async fn heap_dump_with(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
// The capture's histogram never becomes the Memory view's baseline, so the state is unused.
let _turn = state.lock().await;
let outcome = capture(suspended, probe, target, dir.clone(), cancel).await;
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard_later(dir).await;
}
outcome
}
async fn capture(
suspended_now: impl Fn() -> bool + Send + Sync + 'static,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
dir: PathBuf,
cancel: CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let suspended = suspended_now();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(memory(reason)),
};
let probe = probe(suspended);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_heap_dump(&*probe, &tree, suspended, &suspended_now, &dir, &cancel)),
Err(reason) => (None, Ok(memory(reason))),
})
.await
.map_err(|e| IdeError::internal(format!("JVM heap dump failed: {e}")))?;
keep_found(target, found);
outcome
}
/// `tree` starts with the session's reported pid, followed by its descendants. `suspended` is
/// the stop read when the capture began; `suspended_now` reads it again. Anything but `Ready`
/// deletes `dir`.
pub(crate) fn take_heap_dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let outcome = dump(probe, tree, suspended, suspended_now, dir, cancel);
if !matches!(outcome, Ok(JvmHeapOutcome::Ready(_))) {
discard(dir);
}
outcome
}
fn dump(
probe: &dyn AttachProbe,
tree: &[i32],
suspended: bool,
suspended_now: &dyn Fn() -> bool,
dir: &Path,
cancel: &CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
use JvmHeapUnavailable as H;
use JvmMemoryUnavailable as U;
let Some(root) = tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(memory(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(tree, |p| probe.has_listener(p)) else {
return Ok(memory(gone(probe, root)));
};
let version = probe.version(jvm);
let java_major = version.as_ref().map_or(0, |v| v.major);
let safe = safe_while_suspended(version.as_ref());
let unsafe_stop = || Ok(memory(U::UnsafeWhileSuspended { java_major }));
if suspended && !safe {
return unsafe_stop();
}
// The program may stop during the histogram, and a stopped JDK below 17 hangs on the next
// request until it resumes.
let stopped_since = || !safe && suspended_now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(memory),
};
let histogram: Vec<ide_heap::HistogramEntry> = ide_jvm_attach::parse_class_histogram(&text)
.rows
.into_iter()
.map(|r| ide_heap::HistogramEntry { name: r.name, instances: r.instances, bytes: r.bytes })
.collect();
let (instances, bytes) =
histogram.iter().fold((0u64, 0u64), |(i, b), e| (i.saturating_add(e.instances), b.saturating_add(e.bytes)));
let needed = ide_heap::capture_need(bytes, instances);
if let Some(available) = probe.free_bytes(dir)
&& available < needed
{
return Ok(JvmHeapOutcome::Unavailable(H::NotEnoughDiskSpace { needed, available }));
}
if stopped_since() {
return unsafe_stop();
}
// Without flags the index assumes the default object layout.
let vm_flags = probe.vm_flags(jvm).unwrap_or_else(|e| {
tracing::debug!("VM.flags of JVM {jvm}: {e}");
String::new()
});
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
if stopped_since() {
return unsafe_stop();
}
let path = dir.join(DUMP_FILE);
let timeout = ide_jvm_attach::dump_timeout(bytes);
let started = std::time::Instant::now();
let reply = probe.heap_dump(jvm, &path, timeout);
let dump_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
if cancel.is_cancelled() {
return Ok(JvmHeapOutcome::Cancelled);
}
let failed = |message: String| Ok(JvmHeapOutcome::Unavailable(H::DumpFailed { message }));
match reply {
Ok(HeapDumpReply::Created { .. }) => {
if !std::fs::metadata(&path).is_ok_and(|m| m.is_file() && m.len() > 0) {
return failed(format!("the JVM reported a heap dump, but {} is missing or empty", path.display()));
}
Ok(JvmHeapOutcome::Ready(JvmHeapDump {
path,
histogram,
vm_flags,
java_major,
java_version: version.map(|v| v.text).unwrap_or_default(),
pid: jvm,
taken_while_suspended: suspended,
dump_ms,
}))
}
Ok(HeapDumpReply::Refused { message }) if message.is_empty() || message.starts_with("Dumping heap to") => {
failed(NO_REASON.into())
}
Ok(HeapDumpReply::Refused { message }) => failed(message),
Err(AttachError::Timeout) => {
let seconds = u32::try_from(timeout.as_secs()).unwrap_or(u32::MAX);
Ok(JvmHeapOutcome::Unavailable(H::DumpTimedOut { seconds }))
}
// A running JDK rejects an argument with a non-zero result code.
Err(AttachError::Command { message, .. }) => failed(message),
Err(e) => attach_failure(probe, jvm, e, "heap dump").map(memory),
}
}
fn memory(reason: JvmMemoryUnavailable) -> JvmHeapOutcome {
JvmHeapOutcome::Unavailable(JvmHeapUnavailable::Memory { reason })
}
fn discard(dir: &Path) {
if let Err(e) = std::fs::remove_dir_all(dir)
&& e.kind() != std::io::ErrorKind::NotFound
{
tracing::warn!("could not delete heap snapshot directory {}: {e}", dir.display());
}
}
/// A dump may be large, so it is deleted off the async threads.
async fn discard_later(dir: PathBuf) {
let _ = tokio::task::spawn_blocking(move || discard(&dir)).await;
}
+251
Переглянути файл
@@ -0,0 +1,251 @@
//! A suspended JVM's instances, fields and referrers, read through the session's
//! [`JvmInspector`]. An object is only valid within the stop it was read in.
use super::launch_pid;
use crate::session::Session;
use ide_core::event::{DebugEvent, JvmBackendState};
use ide_core::{IdeError, IdeResult};
use ide_jvm_inspect::{
DEFAULT_REFERRERS, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, MAX_INSTANCES,
MAX_REFERRERS, ObjectId, VmTarget,
};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::sync::Arc;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmObjectRef {
/// The `generation` of the [`JvmInstancesResult`] the object came from.
pub generation: u64,
pub id: ObjectId,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInspectUnavailable {
/// The session has no inspector: not a JVM session, not ready yet, or turned off.
NoBackend,
/// The debug adapter lacks what the inspector needs; turned off for the session.
Unsupported {
detail: String,
},
NotSuspended,
/// The object is from an earlier stop, or the program stopped or resumed meanwhile.
Stale,
Collected,
ClassNotLoaded,
VmNotFound,
/// Transient: ask again.
TimedOut,
BackendClosed,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmInstancesResult {
Ready {
/// Goes into the [`JvmObjectRef`] of each instance.
generation: u64,
instances: Vec<JvmValue>,
/// The limit was reached; there may be more.
truncated: bool,
},
Unavailable {
reason: JvmInspectUnavailable,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmFieldsResult {
Ready { fields: Vec<JvmField> },
Unavailable { reason: JvmInspectUnavailable },
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmReferrersResult {
Ready { referrers: Vec<JvmReferrer>, truncated: bool },
Unavailable { reason: JvmInspectUnavailable },
}
/// The session's JVM for the inspector: java-debug's VM is found by the launched program's pid
/// or by the port of an attach (the thread lists of two JVMs can be identical).
pub(crate) fn vm_target(s: &Session) -> VmTarget {
let (pid, attach) = if s.request == "launch" {
(u32::try_from(launch_pid(s)).ok().filter(|&p| p > 0), None)
} else {
(None, s.attach_target.get())
};
VmTarget {
pid,
attach_host: attach.map(|(host, _)| host.clone()).filter(|h| !h.is_empty()),
attach_port: attach.map(|&(_, port)| port),
generation: s.suspended.ticket(),
}
}
pub(crate) fn set_inspector(s: &Session, inspector: Arc<dyn JvmInspector>) {
*s.jvm_inspector.lock() = Some(inspector);
s.emit(DebugEvent::Capabilities);
}
pub(crate) async fn instances(s: &Session, class_name: &str, limit: u32) -> IdeResult<JvmInstancesResult> {
let limit = limit.clamp(1, MAX_INSTANCES);
let call = match Call::begin(s, None).await {
Ok(call) => call,
Err(reason) => return Ok(JvmInstancesResult::Unavailable { reason }),
};
let generation = call.target.generation;
let result = call.run(call.inspector.instances(&call.target, class_name, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(instances) => {
JvmInstancesResult::Ready { generation, truncated: instances.len() >= limit as usize, instances }
}
Err(reason) => JvmInstancesResult::Unavailable { reason },
})
}
pub(crate) async fn fields(s: &Session, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmFieldsResult::Unavailable { reason }),
};
let result = call.run(call.inspector.fields(&call.target, object.id)).await;
Ok(match settle(call.finish(result))? {
Ok(fields) => JvmFieldsResult::Ready { fields },
Err(reason) => JvmFieldsResult::Unavailable { reason },
})
}
pub(crate) async fn referrers(s: &Session, object: JvmObjectRef, limit: u32) -> IdeResult<JvmReferrersResult> {
let limit = if limit == 0 { DEFAULT_REFERRERS } else { limit.min(MAX_REFERRERS) };
let call = match Call::begin(s, Some(object.generation)).await {
Ok(call) => call,
Err(reason) => return Ok(JvmReferrersResult::Unavailable { reason }),
};
let result = call.run(call.inspector.referrers(&call.target, object.id, limit)).await;
Ok(match settle(call.finish(result))? {
Ok(referrers) => JvmReferrersResult::Ready { truncated: referrers.len() >= limit as usize, referrers },
Err(reason) => JvmReferrersResult::Unavailable { reason },
})
}
/// One inspector request. Holds the session's inspect lock and a clone of its inspector, which
/// [`Call::run`] lets go of when the session ends.
pub(super) struct Call<'s> {
session: &'s Session,
pub(super) inspector: Arc<dyn JvmInspector>,
pub(super) target: VmTarget,
_turn: tokio::sync::MutexGuard<'s, ()>,
}
impl<'s> Call<'s> {
/// `object_generation`: the stop the request's object was read in.
pub(super) async fn begin(
s: &'s Session,
object_generation: Option<u64>,
) -> Result<Call<'s>, JvmInspectUnavailable> {
// Checked under the lock: a request queued behind another must see the state it runs in.
let turn = s.inspect_lock.lock().await;
let Some(inspector) = s.jvm_inspector.lock().clone() else {
return Err(JvmInspectUnavailable::NoBackend);
};
if !s.suspended.get() {
return Err(JvmInspectUnavailable::NotSuspended);
}
let target = vm_target(s);
if object_generation.is_some_and(|g| g != target.generation) {
return Err(JvmInspectUnavailable::Stale);
}
Ok(Call { session: s, inspector, target, _turn: turn })
}
pub(super) async fn run<T>(&self, request: InspectFuture<'_, T>) -> Result<T, InspectError> {
tokio::select! {
result = request => result,
() = self.session.ended.cancelled() => Err(InspectError::Closed),
}
}
/// The result unless the program stopped or resumed while it was read: a request of several
/// commands can straddle a stop. `Unsupported` turns the backend off.
pub(super) fn finish<T>(self, result: Result<T, InspectError>) -> Result<T, InspectError> {
let s = self.session;
match result {
Err(InspectError::Unsupported(detail)) => {
turn_off(s, &self.inspector, &detail);
Err(InspectError::Unsupported(detail))
}
_ if !s.suspended.get() || s.suspended.ticket() != self.target.generation => Err(InspectError::Stale),
result => result,
}
}
}
/// An adapter that lacks what the inspector needs will not gain it during the session.
fn turn_off(s: &Session, inspector: &Arc<dyn JvmInspector>, detail: &str) {
let removed = {
let mut slot = s.jvm_inspector.lock();
// Not one installed since this request began.
let current = slot.as_ref().is_some_and(|i| std::ptr::addr_eq(Arc::as_ptr(i), Arc::as_ptr(inspector)));
current && slot.take().is_some()
};
if removed {
s.emit(DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail: detail.to_owned() } });
s.emit(DebugEvent::Capabilities);
}
}
/// A failure the inspector could not classify is an error, not a reason shown to the user.
fn settle<T>(result: Result<T, InspectError>) -> IdeResult<Result<T, JvmInspectUnavailable>> {
use JvmInspectUnavailable as U;
let error = match result {
Ok(value) => return Ok(Ok(value)),
Err(error) => error,
};
Ok(Err(match error {
InspectError::NotSuspended => U::NotSuspended,
InspectError::Stale => U::Stale,
InspectError::Collected => U::Collected,
InspectError::ClassNotLoaded => U::ClassNotLoaded,
InspectError::VmNotFound => U::VmNotFound,
InspectError::Unsupported(detail) => U::Unsupported { detail },
// Also the Kotlin link's `Busy`: the backend stays.
InspectError::TimedOut => U::TimedOut,
InspectError::Closed => U::BackendClosed,
InspectError::Failed(message) => return Err(IdeError::internal(format!("JVM inspector: {message}"))),
}))
}
/// Reports a Kotlin link that stops inspecting or closes. Holds neither the link nor the
/// session: it ends once the session drops the link, and never drops it itself (after
/// `VirtualMachine.Dispose` the link is closed but still forwarding the reply).
pub(crate) fn watch_link(s: &Arc<Session>, link: &ide_jdwp::JdwpLink) {
let mut states = link.subscribe();
let session = Arc::downgrade(s);
tokio::spawn(async move {
let mut reported = None;
loop {
let state = match &*states.borrow_and_update() {
ide_jdwp::LinkState::Degraded(detail) => Some(JvmBackendState::Degraded { detail: detail.clone() }),
ide_jdwp::LinkState::Closed => Some(JvmBackendState::Closed),
_ => None,
};
if let Some(state) = state
&& reported.as_ref() != Some(&state)
{
let Some(s) = session.upgrade() else { return };
if s.ended.is_cancelled() || s.terminated.load(std::sync::atomic::Ordering::Acquire) {
return;
}
s.emit(DebugEvent::JvmBackend { state: state.clone() });
reported = Some(state);
}
if states.changed().await.is_err() {
return;
}
}
});
}
+338
Переглянути файл
@@ -0,0 +1,338 @@
//! Class-table snapshots of a debugged JVM, read over the HotSpot attach listener or, where
//! that is unavailable, counted by the debugger; and the JVM's objects through the debugger.
mod fallback;
mod heap;
mod inspect;
mod scan;
mod snapshot;
#[cfg(test)]
pub(crate) mod test_support;
#[cfg(test)]
mod tests;
use crate::session::Session;
use crate::session::Suspension;
use ide_core::IdeResult;
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicI64, Ordering};
use std::time::Duration;
pub use heap::*;
pub use inspect::{JvmFieldsResult, JvmInspectUnavailable, JvmInstancesResult, JvmObjectRef, JvmReferrersResult};
pub(crate) use inspect::{fields, instances, referrers, set_inspector, watch_link};
#[cfg(all(test, unix))]
pub(crate) use scan::watch_for_attached_jvm;
pub(crate) use scan::{JvmFinder, attach_target, find_attached_jvm, find_local_jvm, local_jdwp_port};
use snapshot::{AttachProbe, JdkProbe, take_snapshot};
pub const JVM_MEMORY_ADAPTERS: &[&str] = &["java-debug", crate::kotlin::ADAPTER, crate::kotlin_lsp::ADAPTER];
pub const HISTOGRAM_TIMEOUT: Duration = Duration::from_secs(10);
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmClassRow {
pub name: String,
pub module: Option<String>,
pub instances: u64,
pub bytes: u64,
/// Change since the previous snapshot; `None` on the first one and for ambiguous rows.
pub delta_instances: Option<i64>,
pub delta_bytes: Option<i64>,
/// Several classes share this name (different class loaders), so it has no delta.
pub ambiguous: bool,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct JvmMemorySnapshot {
/// Largest `bytes` first.
pub rows: Vec<JvmClassRow>,
pub total_instances: u64,
pub total_bytes: u64,
/// Empty when unknown.
pub java_version: String,
pub taken_while_suspended: bool,
/// A previous snapshot from the same source exists; deltas are against it.
pub has_baseline: bool,
/// Wall time of the histogram or count request.
pub elapsed_ms: u32,
pub source: JvmMemorySource,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum JvmMemorySource {
/// `GC.class_histogram -all`: sizes, and objects not yet collected.
Histogram,
/// JDWP `InstanceCounts`: reachable objects only, no sizes (`bytes` is 0).
Debugger,
}
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryUnavailable {
/// An attach session whose JVM is remote, or local but not found by its JDWP port.
AttachedJvmNotFound,
/// An attach through the IDE's listen-mode relay: the JVM connected to the IDE, so it has
/// no JDWP listener to be found by.
ListenModeNotSupported,
UnsupportedPlatform,
NoJvmProcess,
ListenerNotRunning,
/// `java_major` is 0 when the version is unknown.
UnsafeWhileSuspended {
java_major: u32,
},
ProgramExited,
TimedOut,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Type)]
#[serde(tag = "status", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum JvmMemoryResult {
Ready {
snapshot: JvmMemorySnapshot,
},
Unavailable {
reason: JvmMemoryUnavailable,
/// The debugger could count the classes if the program were paused.
can_count_when_paused: bool,
},
}
/// Instances and bytes per class name, for the next snapshot's deltas; `None` for a name
/// several classes shared, so it gets no delta next time either.
type ClassCounts = HashMap<String, Option<(u64, u64)>>;
/// Per-session baseline; its mutex also keeps one class-table request per session at a time.
#[derive(Default)]
pub struct JvmMemoryState {
/// Shared with the blocking task, so a task that panics leaves it in place.
previous: Option<(JvmMemorySource, Arc<ClassCounts>)>,
}
impl JvmMemoryState {
/// The previous snapshot's counts if it came from `source`: counts from the other source
/// are not comparable (only reachable objects versus all of them).
pub(crate) fn baseline(&self, source: JvmMemorySource) -> Option<Arc<ClassCounts>> {
self.previous.as_ref().filter(|(s, _)| *s == source).map(|(_, counts)| counts.clone())
}
pub(crate) fn remember(&mut self, source: JvmMemorySource, counts: ClassCounts) {
self.previous = Some((source, Arc::new(counts)));
}
}
pub(crate) async fn session_classes(session: Arc<Session>) -> IdeResult<JvmMemoryResult> {
until_located(&session).await;
let launched = histogram_pid(&session);
// Detached, so a caller that gives up cannot release the lock while the JVM is still
// answering and let a second histogram request run next to it.
tokio::spawn(async move {
let target = session_target(&session, launched);
let probe = |suspended| -> Arc<dyn AttachProbe> { Arc::new(JdkProbe { suspended }) };
snapshot(&session.jvm_memory, &session.suspended, probe, target, Some(&session)).await
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?
}
/// Bounds the wait of [`until_located`]; `configure` has its own timeouts, this only guards them.
const LOCATE_WAIT: Duration = Duration::from_secs(30);
/// Until `configure` has recorded where `session`'s JVM is. The Memory view asks as soon as the
/// session appears; looked up earlier, an attached JVM is reported as missing.
async fn until_located(session: &Session) {
let mut located = session.jvm_located.subscribe();
tokio::select! {
_ = located.wait_for(|v| *v) => {}
() = session.ended.cancelled() => {}
() = tokio::time::sleep(LOCATE_WAIT) => {}
}
}
/// `launched`: [`histogram_pid`] of `session`.
fn session_target(session: &Session, launched: Result<Option<i32>, JvmMemoryUnavailable>) -> JvmTarget<'_> {
match launched {
Ok(Some(pid)) => JvmTarget::Launched(pid),
Ok(None) => JvmTarget::Attached { pid: &session.jvm_pid, jdwp_port: session.jdwp_port.get().copied() },
Err(reason) => JvmTarget::Unavailable(reason),
}
}
/// The pid of a launch session's program (`None` for an attach session), or why no histogram
/// can be taken.
fn histogram_pid(session: &Session) -> Result<Option<i32>, JvmMemoryUnavailable> {
if !JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str()) {
return Err(JvmMemoryUnavailable::NoJvmProcess);
}
if !cfg!(unix) {
return Err(JvmMemoryUnavailable::UnsupportedPlatform);
}
if session.request == "launch" {
return match i32::try_from(launch_pid(session)).ok().filter(|&p| p > 0) {
Some(pid) => Ok(Some(pid)),
None => Err(JvmMemoryUnavailable::NoJvmProcess),
};
}
if session.jvm_pid.load(Ordering::Acquire) <= 0 {
if session.terminated.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ProgramExited);
}
if session.listen_relay.load(Ordering::Acquire) {
return Err(JvmMemoryUnavailable::ListenModeNotSupported);
}
}
Ok(None)
}
/// The JVM a launch session started: the one the IDE started for an app-launched (Kotlin)
/// session, else the program the adapter reported; 0 when neither is known.
pub(crate) fn launch_pid(session: &Session) -> i64 {
match session.jvm_pid.load(Ordering::Acquire) {
p if p > 0 => p,
_ => session.debuggee_pid.load(Ordering::Acquire),
}
}
#[derive(Clone, Copy)]
pub(crate) enum JvmTarget<'a> {
/// The program a launch session started: its pid and everything it started.
Launched(i32),
/// The JVM of an attach session. While `pid` is 0 it is looked up by `jdwp_port`, which is
/// `None` for a remote JVM.
Attached { pid: &'a AtomicI64, jdwp_port: Option<u16> },
/// Known without asking any JVM: no histogram can be taken, for this reason.
Unavailable(JvmMemoryUnavailable),
}
enum Lookup {
Tree(i32),
Known(i32),
Scan(u16),
}
/// `probe` is built from the suspension read once the lock is held: a request queued behind
/// another must not decide on a state from before the program hit a breakpoint. Where no
/// histogram can be taken, `debugger` (the session) may count the classes instead.
pub(crate) async fn snapshot(
state: &tokio::sync::Mutex<JvmMemoryState>,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
debugger: Option<&Session>,
) -> IdeResult<JvmMemoryResult> {
let mut state = state.lock().await;
let outcome = histogram(&mut state, suspended, probe, target).await?;
fallback::or_debugger_counts(&mut state, outcome, debugger).await
}
async fn histogram(
state: &mut JvmMemoryState,
suspended: &Suspension,
probe: impl FnOnce(bool) -> Arc<dyn AttachProbe>,
target: JvmTarget<'_>,
) -> IdeResult<Result<JvmMemorySnapshot, JvmMemoryUnavailable>> {
let suspended = suspended.get();
let lookup = match lookup(target) {
Ok(lookup) => lookup,
Err(reason) => return Ok(Err(reason)),
};
let probe = probe(suspended);
let previous = state.baseline(JvmMemorySource::Histogram);
let (found, outcome) = tokio::task::spawn_blocking(move || match resolve_jvm(lookup, &*probe, suspended) {
Ok((tree, found)) => (found, take_snapshot(&*probe, &tree, suspended, previous.as_deref())),
Err(reason) => (None, Ok(Err(reason))),
})
.await
.map_err(|e| ide_core::IdeError::internal(format!("JVM memory snapshot failed: {e}")))?;
keep_found(target, found);
Ok(outcome?.map(|(snapshot, counts)| {
state.remember(JvmMemorySource::Histogram, counts);
snapshot
}))
}
/// Where to look for `target`'s JVM, or why there is none to ask.
fn lookup(target: JvmTarget<'_>) -> Result<Lookup, JvmMemoryUnavailable> {
match target {
JvmTarget::Launched(pid) => Ok(Lookup::Tree(pid)),
JvmTarget::Attached { pid, jdwp_port } => match i32::try_from(pid.load(Ordering::Acquire)) {
Ok(p) if p > 0 => Ok(Lookup::Known(p)),
_ => jdwp_port.map(Lookup::Scan).ok_or(JvmMemoryUnavailable::AttachedJvmNotFound),
},
JvmTarget::Unavailable(reason) => Err(reason),
}
}
/// Blocking. The pids to pick the JVM from, and the pid a scan found (for [`keep_found`]).
fn resolve_jvm(
lookup: Lookup,
probe: &dyn AttachProbe,
suspended: bool,
) -> Result<(Vec<i32>, Option<i32>), JvmMemoryUnavailable> {
match lookup {
Lookup::Tree(pid) => Ok((pid_tree(pid), None)),
Lookup::Known(pid) => Ok((vec![pid], None)),
Lookup::Scan(port) => scan_for_jvm(probe, port, suspended).map(|pid| (vec![pid], Some(pid))),
}
}
fn keep_found(target: JvmTarget<'_>, found: Option<i32>) {
if let (Some(found), JvmTarget::Attached { pid, .. }) = (found, target) {
// The attach-time scan may have found it meanwhile; either answer is the same JVM.
let _ = pid.compare_exchange(0, i64::from(found), Ordering::AcqRel, Ordering::Acquire);
}
}
/// A suspended JDK below 17 hangs on an attach request, so while suspended only a JVM known to
/// be 17 or later is asked anything. Pass a version read without attaching (the `release` file).
fn safe_while_suspended(version: Option<&ide_jvm_attach::JavaVersion>) -> bool {
version.is_some_and(|v| ide_jvm_attach::histogram_safe_while_suspended(v.major))
}
/// The JVM listening on JDWP `port`. While suspended the scan reaches the session's own JVM
/// too, so JVMs unsafe to ask are skipped, and when one of them may be the session's the
/// answer is a refusal rather than "not found".
fn scan_for_jvm(probe: &dyn AttachProbe, port: u16, suspended: bool) -> Result<i32, JvmMemoryUnavailable> {
// Majors of the JVMs left out; 0 for an unknown version.
let skipped = std::cell::RefCell::new(std::collections::BTreeSet::new());
let may_query = |pid: u32| {
if !suspended {
return true;
}
let version = probe.version(pid);
let safe = safe_while_suspended(version.as_ref());
if !safe {
skipped.borrow_mut().insert(version.map_or(0, |v| v.major));
}
safe
};
match probe.find_by_jdwp_port(port, &may_query).and_then(|p| i32::try_from(p).ok()) {
Some(pid) if pid > 0 => Ok(pid),
_ => {
let skipped = skipped.into_inner();
Err(match skipped.first() {
None => JvmMemoryUnavailable::AttachedJvmNotFound,
Some(&major) => JvmMemoryUnavailable::UnsafeWhileSuspended {
java_major: if skipped.len() == 1 { major } else { 0 },
},
})
}
}
}
fn pid_tree(pid: i32) -> Vec<i32> {
#[cfg(unix)]
{
ide_core::process::process_tree(pid)
}
#[cfg(not(unix))]
{
vec![pid]
}
}
+146
Переглянути файл
@@ -0,0 +1,146 @@
//! Finding the JVM behind an attach session by its JDWP port.
use super::safe_while_suspended;
use crate::session::Session;
use serde_json::Value;
use std::net::IpAddr;
use std::sync::Arc;
use std::sync::atomic::Ordering;
use std::time::Duration;
/// One try of the scan at attach time, and the limit per socket within it: a silent socket's
/// query thread lives that long, so a longer per-socket limit piles threads up across tries.
const ATTACH_SCAN_TRY: Duration = Duration::from_millis(500);
const ATTACH_SCAN_INTERVAL: Duration = Duration::from_millis(150);
/// The JVM's attach socket can appear a little after the JDWP "Listening" banner, which is
/// when the IDE attaches.
const ATTACH_SCAN_WINDOW: Duration = Duration::from_secs(3);
/// The single scan a refresh makes while the attach-time one found nothing.
pub(super) const RESCAN_PER_SOCKET: Duration = Duration::from_secs(1);
pub(super) const RESCAN_BUDGET: Duration = Duration::from_secs(2);
/// JDWP port of an attach to this machine (`hostName` loopback or absent): only such a JVM has
/// an attach socket the IDE can find.
pub(crate) fn local_jdwp_port(arguments: &Value) -> Option<u16> {
let local = match arguments.get("hostName") {
None | Some(Value::Null) => true,
Some(Value::String(host)) => {
let host = host.trim();
let ip = host.strip_prefix('[').and_then(|h| h.strip_suffix(']')).unwrap_or(host);
host.eq_ignore_ascii_case("localhost") || ip.parse::<IpAddr>().is_ok_and(|ip| ip.is_loopback())
}
Some(_) => false,
};
jdwp_port(arguments).filter(|_| local)
}
/// `port` of an attach request: a number or a numeric string, never 0.
fn jdwp_port(arguments: &Value) -> Option<u16> {
let port = match arguments.get("port")? {
Value::Number(n) => n.as_u64().and_then(|p| u16::try_from(p).ok()),
Value::String(s) => s.trim().parse().ok(),
_ => None,
};
port.filter(|&p| p != 0)
}
/// `hostName` (empty when absent) and `port` of an attach request, local or not.
pub(crate) fn attach_target(arguments: &Value) -> Option<(String, u16)> {
let host = arguments.get("hostName").and_then(Value::as_str).unwrap_or_default();
Some((host.to_owned(), jdwp_port(arguments)?))
}
/// Pid of the local JVM listening on a JDWP port (first argument), given whether the session is
/// suspended (second). Replaceable so tests never send attach requests to real JVMs.
pub(crate) type JvmFinder = Arc<dyn Fn(u16, bool) -> Option<u32> + Send + Sync>;
/// The [`JvmFinder`] of the attach-time scan.
pub(crate) fn find_local_jvm(port: u16, suspended: bool) -> Option<u32> {
let may_query = |pid| !suspended || safe_while_suspended(ide_jvm_attach::java_version_of(pid).as_ref());
ide_jvm_attach::find_jvm_by_jdwp_port(port, ATTACH_SCAN_TRY, ATTACH_SCAN_TRY, may_query)
}
/// Looks for the JVM behind a local attach: one try now, the rest in the background, so a JVM
/// that never opens an attach socket holds the attach up by one try at most. Call it before the
/// debugger connects: the JVM clears its JDWP listener address then, and a scan can no longer
/// tell it by its port.
pub(crate) async fn find_attached_jvm(session: &Arc<Session>, port: u16, finder: JvmFinder) {
let _ = session.jdwp_port.set(port);
let find = move |suspended: bool| finder(port, suspended);
let first = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || first(false)).await {
let _ = session.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
return;
}
drop(watch_for_attached_jvm(session, port, find, ATTACH_SCAN_INTERVAL, ATTACH_SCAN_WINDOW));
}
/// Retries `find` for the session's `jvm_pid`, holding the session weakly so an ended session
/// is not kept alive. `find` is told whether the session is suspended.
pub(crate) fn watch_for_attached_jvm<F>(
session: &Arc<Session>,
port: u16,
find: F,
interval: Duration,
window: Duration,
) -> tokio::task::JoinHandle<()>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let weak = Arc::downgrade(session);
tokio::spawn(async move {
let gate = || match weak.upgrade() {
Some(s) if !s.terminated.load(Ordering::Acquire) && !s.conn.is_closed() => {
if s.jvm_pid.load(Ordering::Acquire) > 0 {
ScanGate::Ended
} else {
ScanGate::Scan { suspended: s.suspended.get() }
}
}
_ => ScanGate::Ended,
};
let Some(pid) = retry_find(find, gate, interval, window).await else {
tracing::debug!("no local JVM found for JDWP port {port} yet");
return;
};
if let Some(s) = weak.upgrade() {
let _ = s.jvm_pid.compare_exchange(0, i64::from(pid), Ordering::AcqRel, Ordering::Acquire);
}
})
}
#[derive(Clone, Copy, Debug, PartialEq)]
pub(crate) enum ScanGate {
Scan { suspended: bool },
Ended,
}
/// Calls `find` on a blocking thread, on each try `gate` allows, until it returns a pid, `gate`
/// reports the end or `window` has passed.
pub(crate) async fn retry_find<F>(
find: F,
gate: impl Fn() -> ScanGate,
interval: Duration,
window: Duration,
) -> Option<u32>
where
F: Fn(bool) -> Option<u32> + Clone + Send + 'static,
{
let deadline = tokio::time::Instant::now() + window;
loop {
match gate() {
ScanGate::Ended => break,
ScanGate::Scan { suspended } => {
let find = find.clone();
if let Ok(Some(pid)) = tokio::task::spawn_blocking(move || find(suspended)).await {
return Some(pid);
}
}
}
if tokio::time::Instant::now() + interval >= deadline {
break;
}
tokio::time::sleep(interval).await;
}
None
}
+214
Переглянути файл
@@ -0,0 +1,214 @@
//! One class histogram of a JVM and its deltas against the previous one.
use super::scan::{RESCAN_BUDGET, RESCAN_PER_SOCKET};
use super::{
ClassCounts, HISTOGRAM_TIMEOUT, JvmClassRow, JvmMemorySnapshot, JvmMemorySource, JvmMemoryUnavailable,
safe_while_suspended,
};
use ide_core::IdeResult;
use ide_jvm_attach::{AttachError, HeapDumpReply, HistogramRow, JavaVersion};
use std::collections::HashMap;
use std::path::Path;
use std::time::Duration;
/// Version probe for a JVM whose `release` file could not be read.
const VERSION_TIMEOUT: Duration = Duration::from_secs(2);
const FLAGS_TIMEOUT: Duration = Duration::from_secs(2);
/// The attach calls a snapshot or a heap dump makes, so tests can stand in for a JVM.
pub(crate) trait AttachProbe: Send + Sync {
fn has_listener(&self, pid: u32) -> bool;
/// Asks only the JVMs whose pid `may_query` accepts.
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32>;
fn version(&self, pid: u32) -> Option<JavaVersion>;
fn histogram(&self, pid: u32) -> Result<String, AttachError>;
fn alive(&self, pid: u32) -> bool;
fn vm_flags(&self, pid: u32) -> Result<String, AttachError>;
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError>;
/// `None` where the free space is not known.
fn free_bytes(&self, dir: &Path) -> Option<u64>;
}
pub(super) struct JdkProbe {
pub(super) suspended: bool,
}
impl AttachProbe for JdkProbe {
fn has_listener(&self, pid: u32) -> bool {
ide_jvm_attach::has_listener(pid)
}
fn find_by_jdwp_port(&self, port: u16, may_query: &dyn Fn(u32) -> bool) -> Option<u32> {
ide_jvm_attach::find_jvm_by_jdwp_port(port, RESCAN_PER_SOCKET, RESCAN_BUDGET, may_query)
}
fn version(&self, pid: u32) -> Option<JavaVersion> {
ide_jvm_attach::java_version_of(pid).or_else(|| {
// The version decides whether attaching to a suspended VM hangs it, so never
// attach to learn it while suspended.
if self.suspended {
return None;
}
let out = ide_jvm_attach::jcmd(pid, "VM.version", VERSION_TIMEOUT).ok()?;
ide_jvm_attach::parse_vm_version(&out)
})
}
fn histogram(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::jcmd(pid, "GC.class_histogram -all", HISTOGRAM_TIMEOUT)
}
fn alive(&self, pid: u32) -> bool {
alive(pid)
}
fn vm_flags(&self, pid: u32) -> Result<String, AttachError> {
ide_jvm_attach::vm_flags(pid, FLAGS_TIMEOUT)
}
fn heap_dump(&self, pid: u32, path: &Path, timeout: Duration) -> Result<HeapDumpReply, AttachError> {
ide_jvm_attach::heap_dump(pid, path, timeout)
}
fn free_bytes(&self, dir: &Path) -> Option<u64> {
ide_heap::free_bytes(dir)
}
}
fn alive(pid: u32) -> bool {
#[cfg(unix)]
{
// 0 and negative pids address process groups.
let Ok(pid) = i32::try_from(pid) else { return false };
// SAFETY: signal 0 only checks that the process exists and may be signalled.
pid > 0
&& (unsafe { libc::kill(pid, 0) } == 0
|| std::io::Error::last_os_error().raw_os_error() == Some(libc::EPERM))
}
#[cfg(not(unix))]
{
let _ = pid;
false
}
}
/// Why `pid`'s attach listener cannot answer: it stopped listening or the program ended.
pub(super) fn gone(probe: &dyn AttachProbe, pid: u32) -> JvmMemoryUnavailable {
if probe.alive(pid) { JvmMemoryUnavailable::ListenerNotRunning } else { JvmMemoryUnavailable::ProgramExited }
}
/// The reason for an attach request (`what`) to `jvm` that failed, or an error where no reason
/// fits.
pub(super) fn attach_failure(
probe: &dyn AttachProbe,
jvm: u32,
error: AttachError,
what: &str,
) -> IdeResult<JvmMemoryUnavailable> {
use JvmMemoryUnavailable as U;
match error {
AttachError::Timeout => Ok(U::TimedOut),
AttachError::Refused | AttachError::NoListener => Ok(gone(probe, jvm)),
AttachError::ForeignListener => {
tracing::warn!("attach socket of JVM {jvm} is no longer private to this user; not using it");
Ok(U::ListenerNotRunning)
}
AttachError::Unsupported => Ok(U::UnsupportedPlatform),
e => Err(ide_core::IdeError::internal(format!("{what} of JVM {jvm}: {e}"))),
}
}
pub(crate) fn pick_jvm_pid(tree: &[i32], has_listener: impl Fn(u32) -> bool) -> Option<u32> {
tree.iter().filter_map(|&p| u32::try_from(p).ok()).filter(|&p| p > 0).find(|&p| has_listener(p))
}
pub(crate) fn diff_rows(rows: Vec<HistogramRow>, previous: Option<&ClassCounts>) -> (Vec<JvmClassRow>, ClassCounts) {
let mut seen: HashMap<&str, u32> = HashMap::with_capacity(rows.len());
for r in &rows {
*seen.entry(r.name.as_str()).or_default() += 1;
}
let ambiguous: Vec<bool> = rows.iter().map(|r| seen[r.name.as_str()] > 1).collect();
drop(seen);
let mut counts = ClassCounts::with_capacity(rows.len());
let mut out: Vec<JvmClassRow> = rows
.into_iter()
.zip(ambiguous)
.map(|(r, ambiguous)| {
counts.insert(r.name.clone(), (!ambiguous).then_some((r.instances, r.bytes)));
let delta = match previous {
_ if ambiguous => None,
None => None,
Some(p) => match p.get(&r.name) {
Some(None) => None,
// Absent: new since the previous snapshot, so it grew from nothing.
before => {
let (instances, bytes) = before.copied().flatten().unwrap_or_default();
Some((delta(r.instances, instances), delta(r.bytes, bytes)))
}
},
};
JvmClassRow {
name: r.name,
module: r.module,
instances: r.instances,
bytes: r.bytes,
delta_instances: delta.map(|d| d.0),
delta_bytes: delta.map(|d| d.1),
ambiguous,
}
})
.collect();
out.sort_by(|a, b| b.bytes.cmp(&a.bytes).then_with(|| a.name.cmp(&b.name)));
(out, counts)
}
/// Exact while the difference fits in an i64 (two's complement).
fn delta(now: u64, before: u64) -> i64 {
now.wrapping_sub(before) as i64
}
/// `pid_tree` starts with the session's reported pid, followed by its descendants.
pub(crate) fn take_snapshot(
probe: &dyn AttachProbe,
pid_tree: &[i32],
suspended: bool,
previous: Option<&ClassCounts>,
) -> IdeResult<Result<(JvmMemorySnapshot, ClassCounts), JvmMemoryUnavailable>> {
use JvmMemoryUnavailable as U;
let Some(root) = pid_tree.first().and_then(|&p| u32::try_from(p).ok()) else {
return Ok(Err(U::NoJvmProcess));
};
let Some(jvm) = pick_jvm_pid(pid_tree, |p| probe.has_listener(p)) else {
return Ok(Err(gone(probe, root)));
};
let version = probe.version(jvm);
if suspended && !safe_while_suspended(version.as_ref()) {
return Ok(Err(U::UnsafeWhileSuspended { java_major: version.map_or(0, |v| v.major) }));
}
let started = std::time::Instant::now();
let text = match probe.histogram(jvm) {
Ok(text) => text,
Err(e) => return attach_failure(probe, jvm, e, "class histogram").map(Err),
};
let elapsed_ms = u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX);
let histogram = ide_jvm_attach::parse_class_histogram(&text);
if histogram.skipped_lines > 0 {
tracing::debug!("class histogram of JVM {jvm}: {} unreadable lines", histogram.skipped_lines);
}
let (rows, counts) = diff_rows(histogram.rows, previous);
let snapshot = JvmMemorySnapshot {
total_instances: rows.iter().map(|r| r.instances).sum(),
total_bytes: rows.iter().map(|r| r.bytes).sum(),
rows,
java_version: version.map(|v| v.text).unwrap_or_default(),
taken_while_suspended: suspended,
has_baseline: previous.is_some(),
elapsed_ms,
source: JvmMemorySource::Histogram,
};
Ok(Ok((snapshot, counts)))
}
+119
Переглянути файл
@@ -0,0 +1,119 @@
//! A scripted [`JvmInspector`] and a session without an adapter behind it.
use crate::SessionOwner;
use crate::adapter::{AdapterSpec, Transport};
use crate::connection::Connection;
use crate::session::Session;
use ide_core::{DebugSessionId, IdeEvent, SharedSink};
use ide_jvm_inspect::{
ClassCount, InspectError, InspectFuture, JvmField, JvmInspector, JvmReferrer, JvmValue, ObjectId, VmTarget,
};
use parking_lot::Mutex;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::sync::Notify;
#[derive(Default)]
pub(crate) struct FakeInspector {
pub targets: Mutex<Vec<VmTarget>>,
/// The `limit` of each `instances` / `referrers` call.
pub limits: Mutex<Vec<u32>>,
pub calls: AtomicUsize,
/// Returned by every call when set.
pub error: Mutex<Option<InspectError>>,
pub counts: Mutex<Vec<ClassCount>>,
pub instances: Mutex<Vec<JvmValue>>,
/// Every call signals `entered`, then waits for `release`.
pub hold: bool,
pub entered: Notify,
pub release: Notify,
}
impl FakeInspector {
pub fn failing(error: InspectError) -> Self {
Self { error: Mutex::new(Some(error)), ..Self::default() }
}
pub fn calls(&self) -> usize {
self.calls.load(Ordering::SeqCst)
}
async fn answer<T: Send>(&self, target: &VmTarget, limit: Option<u32>, ok: T) -> Result<T, InspectError> {
self.calls.fetch_add(1, Ordering::SeqCst);
self.targets.lock().push(target.clone());
self.limits.lock().extend(limit);
if self.hold {
self.entered.notify_one();
self.release.notified().await;
}
match self.error.lock().clone() {
Some(e) => Err(e),
None => Ok(ok),
}
}
}
impl JvmInspector for FakeInspector {
fn class_counts<'a>(&'a self, target: &'a VmTarget) -> InspectFuture<'a, Vec<ClassCount>> {
Box::pin(self.answer(target, None, self.counts.lock().clone()))
}
fn instances<'a>(&'a self, target: &'a VmTarget, _: &'a str, limit: u32) -> InspectFuture<'a, Vec<JvmValue>> {
Box::pin(self.answer(target, Some(limit), self.instances.lock().clone()))
}
fn fields<'a>(&'a self, target: &'a VmTarget, _: ObjectId) -> InspectFuture<'a, Vec<JvmField>> {
Box::pin(self.answer(target, None, Vec::new()))
}
fn referrers<'a>(&'a self, target: &'a VmTarget, _: ObjectId, limit: u32) -> InspectFuture<'a, Vec<JvmReferrer>> {
Box::pin(self.answer(target, Some(limit), Vec::new()))
}
}
pub(crate) struct BareSession {
pub session: Arc<Session>,
pub events: Arc<Mutex<Vec<IdeEvent>>>,
/// The adapter's end of the connection, kept open.
_adapter: tokio::io::DuplexStream,
}
/// A java-debug session that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session(request: &str) -> BareSession {
bare_session_of("java-debug", request)
}
/// A session of adapter `name` that no adapter answers; must be called inside a Tokio runtime.
pub(crate) fn bare_session_of(name: &str, request: &str) -> BareSession {
let events: Arc<Mutex<Vec<IdeEvent>>> = Arc::default();
let sink_events = events.clone();
let sink: SharedSink = Arc::new(move |e: IdeEvent| sink_events.lock().push(e));
let (client, adapter) = tokio::io::duplex(1 << 10);
let (reader, writer) = tokio::io::split(client);
let (conn, _) = Connection::start(reader, writer);
let spec = AdapterSpec {
adapter: name.into(),
command: String::new(),
args: Vec::new(),
transport: Transport::Stdio,
install_hint: None,
cwd: std::env::temp_dir(),
env: Vec::new(),
server_host: None,
connect: None,
pinned: None,
};
let session = Session::new(
DebugSessionId(1),
SessionOwner::new("test", sink),
"test".into(),
name.into(),
None,
request.into(),
conn,
spec,
None,
None,
);
BareSession { session: Arc::new(session), events, _adapter: adapter }
}
File diff suppressed because it is too large. Load diff
+11
Переглянути файл
@@ -9,6 +9,17 @@ use std::{
pub const ADAPTER: &str = "kotlin-debug-adapter";
/// fwcd's adapter or kotlin-lsp's: both debug a JVM the IDE starts, through the JDWP link.
pub fn is_kotlin(adapter: &str) -> bool {
adapter == ADAPTER || adapter == crate::kotlin_lsp::ADAPTER
}
/// fwcd's adapter resumes the VM while constructing its debuggee, before its breakpoint hooks
/// exist, so only its link holds resumes back until the attach is done.
pub(crate) fn link_options(adapter: &str) -> ide_jdwp::LinkOptions {
ide_jdwp::LinkOptions { hold_resumes: adapter == ADAPTER }
}
pub struct SourceMirror {
dir: tempfile::TempDir,
paths: HashMap<PathBuf, PathBuf>,
-166
Переглянути файл
@@ -1,166 +0,0 @@
//! Keep a suspended JVM stopped until Kotlin's adapter has installed its listeners.
//! The official 0.4.4 adapter resumes VMStart while constructing its debuggee, before
//! installing breakpoint hooks. DAP attach completes only after those hooks exist.
use ide_core::IdeResult;
use std::{io, time::Duration};
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt},
net::{TcpListener, TcpStream},
sync::oneshot,
task::JoinHandle,
};
pub(crate) struct StartupRelay {
release: Option<oneshot::Sender<()>>,
task: JoinHandle<()>,
}
impl StartupRelay {
pub async fn start(host: String, port: u16) -> IdeResult<(Self, u16)> {
let listener = TcpListener::bind(("127.0.0.1", 0)).await?;
let local = listener.local_addr()?.port();
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move {
let connect = async {
let (adapter, _) = listener.accept().await?;
let jvm = TcpStream::connect((host.as_str(), port)).await?;
adapter.set_nodelay(true)?;
jvm.set_nodelay(true)?;
Ok::<_, io::Error>((adapter, jvm))
};
if let Ok(Ok((adapter, jvm))) = tokio::time::timeout(Duration::from_secs(60), connect).await {
let (mut ar, mut aw) = adapter.into_split();
let (mut jr, mut jw) = jvm.into_split();
let upstream = forward(&mut ar, &mut jw, ready);
let downstream = tokio::io::copy(&mut jr, &mut aw);
// Close both sockets as soon as either side disconnects.
tokio::select! {
r = upstream => if let Err(e) = r { tracing::debug!("Kotlin JDWP relay: {e}"); },
_ = downstream => {},
}
}
});
Ok((Self { release: Some(release), task }, local))
}
pub fn release(&mut self) {
if let Some(ready) = self.release.take() {
let _ = ready.send(());
}
}
}
impl Drop for StartupRelay {
fn drop(&mut self) {
self.task.abort();
}
}
async fn packet(reader: &mut (impl AsyncRead + Unpin)) -> io::Result<Vec<u8>> {
let mut header = [0; 11];
reader.read_exact(&mut header).await?;
let len = u32::from_be_bytes(header[..4].try_into().unwrap()) as usize;
if !(11..=16 * 1024 * 1024).contains(&len) {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP packet length"));
}
let mut packet = vec![0; len];
packet[..11].copy_from_slice(&header);
reader.read_exact(&mut packet[11..]).await?;
Ok(packet)
}
async fn forward(
reader: &mut (impl AsyncRead + Unpin),
writer: &mut (impl AsyncWrite + Unpin),
mut ready: oneshot::Receiver<()>,
) -> io::Result<()> {
let mut handshake = [0; 14];
reader.read_exact(&mut handshake).await?;
if &handshake != b"JDWP-Handshake" {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Invalid JDWP handshake"));
}
writer.write_all(&handshake).await?;
let mut released = false;
let mut held: Vec<Vec<u8>> = Vec::new();
loop {
// Retain the in-flight read when attach completes, including partial packets.
let read = packet(reader);
tokio::pin!(read);
let packet = tokio::select! {
result = &mut read => result?,
result = &mut ready, if !released => {
result.map_err(|_| io::Error::new(io::ErrorKind::ConnectionAborted, "Kotlin attach cancelled"))?;
released = true;
for packet in held.drain(..) {
writer.write_all(&packet).await?;
}
read.await?
}
};
// JDWP VirtualMachine.Resume (1/9), ThreadReference.Resume (11/3).
// Other commands must proceed: attach itself queries threads and installs requests.
if !released && packet[8] == 0 && matches!((packet[9], packet[10]), (1, 9) | (11, 3)) {
if held.len() >= 1024 {
return Err(io::Error::new(io::ErrorKind::InvalidData, "Too many pending JDWP resumes"));
}
held.push(packet);
} else {
writer.write_all(&packet).await?;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn command(id: u8, set: u8, cmd: u8) -> Vec<u8> {
vec![0, 0, 0, 11, 0, 0, 0, id, 0, set, cmd]
}
#[tokio::test]
async fn holds_resumes_but_allows_setup_and_preserves_partial_reads() {
let (mut adapter, mut reader) = tokio::io::duplex(4096);
let (mut writer, mut jvm) = tokio::io::duplex(4096);
let (release, ready) = oneshot::channel();
let task = tokio::spawn(async move { forward(&mut reader, &mut writer, ready).await });
adapter.write_all(b"JDWP-Handshake").await.unwrap();
let mut handshake = [0; 14];
jvm.read_exact(&mut handshake).await.unwrap();
assert_eq!(&handshake, b"JDWP-Handshake");
let resume = command(1, 1, 9);
let thread_resume = command(2, 11, 3);
let setup = command(3, 1, 4);
adapter.write_all(&resume).await.unwrap();
adapter.write_all(&thread_resume).await.unwrap();
adapter.write_all(&setup).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), setup);
let next = command(4, 1, 9);
adapter.write_all(&next[..6]).await.unwrap();
release.send(()).unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), resume);
assert_eq!(packet(&mut jvm).await.unwrap(), thread_resume);
adapter.write_all(&next[6..]).await.unwrap();
assert_eq!(packet(&mut jvm).await.unwrap(), next);
drop(adapter);
assert!(task.await.unwrap().is_err());
}
#[tokio::test]
async fn dropping_session_closes_relay() {
let (relay, port) = StartupRelay::start("127.0.0.1".into(), 1).await.unwrap();
let mut adapter = TcpStream::connect(("127.0.0.1", port)).await.unwrap();
drop(relay);
let result = tokio::time::timeout(Duration::from_secs(1), adapter.read_u8()).await.unwrap();
assert!(result.is_err());
}
#[tokio::test]
async fn rejects_invalid_packet_lengths() {
for len in [0_u32, 10, 16 * 1024 * 1024 + 1] {
let mut bytes = command(1, 1, 9);
bytes[..4].copy_from_slice(&len.to_be_bytes());
assert_eq!(packet(&mut bytes.as_slice()).await.unwrap_err().kind(), io::ErrorKind::InvalidData);
}
}
}
+147
Переглянути файл
@@ -0,0 +1,147 @@
//! JetBrains kotlin-lsp's debugger: a DAP server inside the language server, reached through
//! the `debugServer` port it opens once the project is imported.
//!
//! Known limits, with no guard here: an evaluate that calls a function with a breakpoint, right
//! after a step over a suspension point, loses every later stop at that breakpoint; while
//! stepping, breakpoints hit by other coroutines are skipped.
use ide_core::{IdeError, IdeResult};
use parking_lot::Mutex;
use std::collections::HashSet;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
pub const ADAPTER: &str = "kotlin-lsp";
/// The `adapterID` its debug server expects in `initialize`.
pub const DAP_ADAPTER_ID: &str = "intellij_jvm";
/// The path of a `file:` URI, as kotlin-lsp reports some sources (percent-encoded); `None` for
/// anything else, including a URI naming another host.
pub(crate) fn source_path(path: &str) -> Option<PathBuf> {
let rest = path.strip_prefix("file:")?;
let path = match rest.strip_prefix("//") {
Some(authority) => {
let slash = authority.find('/')?;
if !matches!(&authority[..slash], "" | "localhost") {
return None;
}
&authority[slash..]
}
None => rest,
};
let decoded = percent_encoding::percent_decode_str(path).decode_utf8().ok()?;
// `/C:/p/App.kt` on Windows.
let decoded = match decoded.strip_prefix('/') {
Some(drive) if cfg!(windows) && drive.as_bytes().get(1) == Some(&b':') => drive,
_ => &decoded,
};
Some(PathBuf::from(decoded))
}
pub(crate) fn frame_name(name: &str) -> &str {
name.strip_prefix("at ").unwrap_or(name)
}
/// What kotlin-lsp's DAP server does unlike other adapters.
///
/// It never answers a request that carries a thread id from an earlier stop (K1 G4), so such a
/// request would wait out `REQUEST_TIMEOUT`: [`Self::check_thread`] refuses it first. Only thread
/// ids were measured; a stale `frameId` or `variablesReference` may wait the same way. A step
/// ends with `reason: "breakpoint"`; a real breakpoint hit during a step is then reported as a
/// step too, since the stop carries neither `hitBreakpointIds` nor a description.
#[derive(Default)]
pub(crate) struct Quirks {
step_pending: AtomicBool,
/// The stop (a `Suspension` ticket) and the thread ids seen in it.
threads: Mutex<(u64, HashSet<i64>)>,
}
impl Quirks {
/// Before `next`, `stepIn`, `stepOut` or a forced step.
pub fn step_sent(&self) {
self.step_pending.store(true, Ordering::Release);
}
/// Before `continue` or run to cursor, or after a step request failed.
pub fn resume_sent(&self) {
self.step_pending.store(false, Ordering::Release);
}
/// The reason to report for a `stopped` event.
pub fn stop_reason(&self, reason: String) -> String {
let stepped = self.step_pending.swap(false, Ordering::AcqRel);
if stepped && reason == "breakpoint" { "step".into() } else { reason }
}
/// Thread ids seen in stop `stop`: a `stopped` event's, or a `threads` reply to a request sent
/// during that stop. A reply from an older stop changes nothing.
pub fn saw_threads(&self, stop: u64, ids: impl IntoIterator<Item = i64>) {
let mut threads = self.threads.lock();
if stop > threads.0 {
*threads = (stop, ids.into_iter().collect());
} else if stop == threads.0 {
threads.1.extend(ids);
}
}
/// Refuses a thread id that is not one of current stop `stop`'s, once some are known.
pub fn check_thread(&self, stop: u64, suspended: bool, id: i64) -> IdeResult<()> {
let threads = self.threads.lock();
if suspended && threads.0 == stop && !threads.1.is_empty() && !threads.1.contains(&id) {
return Err(IdeError::invalid(format!("thread {id} belongs to an earlier stop")));
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
#[test]
fn file_uris_become_paths() {
assert_eq!(source_path("file:///p/a%20b/App.kt").as_deref(), Some(Path::new("/p/a b/App.kt")));
assert_eq!(source_path("file://localhost/p/x.kt").as_deref(), Some(Path::new("/p/x.kt")));
assert_eq!(source_path("/p/x.kt"), None);
}
#[test]
fn frame_names_lose_the_at_prefix() {
assert_eq!(frame_name("at com.x.AppKt.main(App.kt:13)"), "com.x.AppKt.main(App.kt:13)");
assert_eq!(frame_name("main"), "main");
}
#[test]
fn only_a_pending_step_turns_a_breakpoint_into_a_step() {
let quirks = Quirks::default();
quirks.step_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "step");
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
quirks.step_sent();
quirks.resume_sent();
assert_eq!(quirks.stop_reason("breakpoint".into()), "breakpoint");
}
#[test]
fn threads_are_checked_against_the_current_stop() {
let quirks = Quirks::default();
quirks.saw_threads(3, [18]);
assert!(quirks.check_thread(3, true, 18).is_ok());
assert!(quirks.check_thread(3, true, 6).is_err());
assert!(quirks.check_thread(4, true, 6).is_ok(), "nothing known about stop 4 yet");
assert!(quirks.check_thread(3, false, 6).is_ok());
}
#[test]
fn an_older_threads_reply_does_not_change_a_newer_stop() {
let quirks = Quirks::default();
quirks.saw_threads(4, [2]);
quirks.saw_threads(3, [1]);
assert!(quirks.check_thread(4, true, 1).is_err());
quirks.saw_threads(4, [7]);
assert!(quirks.check_thread(4, true, 2).is_ok() && quirks.check_thread(4, true, 7).is_ok());
quirks.saw_threads(5, [9]);
assert!(quirks.check_thread(5, true, 2).is_err());
}
}
+183 -30
Переглянути файл
@@ -13,8 +13,9 @@ pub mod attach;
mod cargo;
pub mod connection;
pub mod jdwp;
mod jvm_memory;
pub mod kotlin;
mod kotlin_jdwp;
pub mod kotlin_lsp;
mod session;
use adapter::{AdapterLauncher, AdapterSpec, ProcessLauncher};
@@ -22,6 +23,7 @@ use connection::{Connection, DapError};
use futures::future::join_all;
use ide_core::{DebugSessionId, IdeError, IdeResult, SharedSink};
use ide_language::LanguageRegistry;
pub use jvm_memory::*;
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
@@ -126,6 +128,14 @@ pub struct DebugCapabilities {
pub log_points: bool,
/// HotSwap ("Reload Changed Classes"): java-debug's `redefineClasses` request.
pub hot_code_replace: bool,
/// The session debugs a JVM, so it has a Memory view; [`DebugManager::jvm_memory_classes`]
/// says why when it has no data (platform, remote JVM, ...).
pub jvm_memory: bool,
/// The session has a JVM inspector: instances, fields and referrers while suspended. Can
/// change during the session ([`ide_core::event::DebugEvent::Capabilities`]).
pub jvm_instances: bool,
/// The session can be asked for a heap dump ([`DebugManager::jvm_heap_dump`]).
pub jvm_heap: bool,
}
/// Result of [`DebugManager::redefine_classes`].
@@ -302,6 +312,7 @@ pub struct DebugManager {
data_breakpoints: Mutex<Vec<DataBreakpoint>>,
/// In-flight stops, by session: a second Stop sets `stop_force`, the waiting one kills.
stopping: Mutex<HashMap<DebugSessionId, Arc<Session>>>,
jvm_finder: Mutex<jvm_memory::JvmFinder>,
}
/// rdbg (Ruby's debug gem) starts the debuggee itself and ignores `launch`'s program,
@@ -566,9 +577,21 @@ pub(crate) fn capabilities_from(caps: &Value) -> DebugCapabilities {
hit_conditional_breakpoints: has("supportsHitConditionalBreakpoints"),
log_points: has("supportsLogPoints"),
hot_code_replace: false,
jvm_memory: false,
jvm_instances: false,
jvm_heap: false,
}
}
pub(crate) fn session_capabilities(session: &Session) -> DebugCapabilities {
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_memory = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps.jvm_instances = session.jvm_inspector.lock().is_some();
caps.jvm_heap = JVM_MEMORY_ADAPTERS.contains(&session.adapter.as_str());
caps
}
/// States of a file's line breakpoints that no session has been sent.
fn unsent(breakpoints: &[Breakpoint]) -> Vec<BreakpointState> {
breakpoints
@@ -588,9 +611,19 @@ fn unverified(enabled: impl Iterator<Item = bool>) -> Vec<BreakpointState> {
.collect()
}
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (on Windows
/// `__debug_bin.exe` plus `__debug_bin.exe<digits>` in DAP mode) and removes it on a
/// graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// A step request's outcome; a failed one leaves no step pending, so the next stop keeps its reason.
fn step_settled(session: &Session, sent: Result<Value, DapError>) -> Result<Value, DapError> {
if sent.is_err()
&& let Some(quirks) = session.kotlin_lsp_quirks()
{
quirks.resume_sent();
}
sent
}
/// Delve compiles the debuggee to `__debug_bin` in the session's cwd (in DAP mode a
/// temp-file name, `__debug_bin<digits>`; on Windows with `.exe` before the digits) and
/// removes it on a graceful exit — a shutdown it doesn't always reach, so the files pile up. Sweep
/// the leftovers once the session is down. Runs after `session_closed`, so Delve
/// has already exited or been killed and holds no file locks; a leftover of a
/// still-building run is removed on that run's own shutdown.
@@ -599,14 +632,10 @@ fn remove_delve_binaries(cwd: &Path) {
for entry in entries.flatten() {
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
// Windows writes `__debug_bin.exe` (and `.exe<digits>` in DAP mode); other
// platforms keep the bare name. Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin");
let leftover = suffix.is_some_and(|s| {
s.is_empty()
|| s == ".exe"
|| (s.starts_with(".exe") && s[".exe".len()..].chars().all(|c| c.is_ascii_digit()))
});
// `__debug_bin`, then `.exe` on Windows, then DAP mode's temp-file digits.
// Suffixes like `.go` are user files.
let suffix = name.strip_prefix("__debug_bin").map(|s| s.strip_prefix(".exe").unwrap_or(s));
let leftover = suffix.is_some_and(|s| s.chars().all(|c| c.is_ascii_digit()));
if !leftover || entry.file_type().is_ok_and(|t| !t.is_file()) {
continue;
}
@@ -657,9 +686,15 @@ impl DebugManager {
function_breakpoints: Mutex::default(),
data_breakpoints: Mutex::default(),
stopping: Mutex::default(),
jvm_finder: Mutex::new(Arc::new(jvm_memory::find_local_jvm)),
})
}
#[cfg(test)]
pub(crate) fn set_jvm_finder(&self, finder: jvm_memory::JvmFinder) {
*self.jvm_finder.lock() = finder;
}
fn session(&self, s: DebugSessionId) -> IdeResult<Arc<Session>> {
self.sessions.lock().get(&s).cloned().ok_or_else(|| no_session(s))
}
@@ -953,6 +988,7 @@ impl DebugManager {
arguments["projectRoot"] = json!(mirror.root());
*s.kotlin_mirror.lock() = Some(mirror);
}
let is_kotlin_lsp = s.adapter == kotlin_lsp::ADAPTER;
let caps = s
.conn
.request_timeout(
@@ -960,7 +996,7 @@ impl DebugManager {
json!({
"clientID": "openidea",
"clientName": "OpenIDEA",
"adapterID": s.adapter,
"adapterID": if is_kotlin_lsp { kotlin_lsp::DAP_ADAPTER_ID } else { s.adapter.as_str() },
"locale": "en",
"linesStartAt1": true,
"columnsStartAt1": true,
@@ -970,7 +1006,8 @@ impl DebugManager {
"supportsRunInTerminalRequest": true,
"supportsStartDebuggingRequest": true,
"supportsProgressReporting": true,
"supportsInvalidatedEvent": false,
// kotlin-lsp sends `invalidated` either way.
"supportsInvalidatedEvent": is_kotlin_lsp,
"supportsMemoryReferences": false,
"supportsArgsCanBeInterpretedByShell": false,
}),
@@ -979,14 +1016,32 @@ impl DebugManager {
.await?;
*s.caps.lock() = if caps.is_object() { caps } else { json!({}) };
// The Kotlin adapter ignores the resolved classpath and project runtime on launch.
let request = if s.adapter == kotlin::ADAPTER && s.request == "launch" {
// The Kotlin adapters ignore the resolved classpath and project runtime on launch.
let request = if kotlin::is_kotlin(&s.adapter) && s.request == "launch" {
session::start_kotlin_debuggee(s, &mut arguments).await?;
"attach"
} else {
&s.request
};
if s.adapter == kotlin::ADAPTER {
// Both blocks below run before the Kotlin link replaces the JVM's address with its own.
if request == "attach"
&& let Some(target) = jvm_memory::attach_target(&arguments)
{
let _ = s.attach_target.set(target);
}
if cfg!(unix)
&& s.request == "attach"
&& JVM_MEMORY_ADAPTERS.contains(&s.adapter.as_str())
&& let Some(port) = jvm_memory::local_jdwp_port(&arguments)
{
if jdwp::is_listen_relay(port) {
s.listen_relay.store(true, Ordering::Release);
} else {
let finder = self.jvm_finder.lock().clone();
jvm_memory::find_attached_jvm(s, port, finder).await;
}
}
if kotlin::is_kotlin(&s.adapter) {
let host = arguments
.get("hostName")
.and_then(Value::as_str)
@@ -998,11 +1053,14 @@ impl DebugManager {
.and_then(|p| u16::try_from(p).ok())
.filter(|p| *p != 0)
.ok_or_else(|| IdeError::invalid("Invalid Kotlin JDWP port"))?;
let (relay, port) = kotlin_jdwp::StartupRelay::start(host, port).await?;
*s.kotlin_relay.lock() = Some(relay);
let (link, port) = ide_jdwp::JdwpLink::start(host, port, kotlin::link_options(&s.adapter)).await?;
*s.jvm_inspector.lock() = Some(Arc::new(ide_jdwp::LinkInspector::new(link.clone())));
jvm_memory::watch_link(s, &link);
*s.jdwp_link.lock() = Some(link);
arguments["hostName"] = json!("127.0.0.1");
arguments["port"] = json!(port);
}
s.jvm_located.send_replace(true);
// Launch/attach may wait for a slow runtime or a remote target. Let the adapter report
// failure; Stop cancels configuration and a closed connection fails the pending reply.
// Many adapters answer only after `configurationDone`, so don't wait for the reply yet.
@@ -1097,8 +1155,8 @@ impl DebugManager {
reply.await.unwrap_or(Err(DapError::Closed))?;
}
}
if let Some(relay) = s.kotlin_relay.lock().as_mut() {
relay.release();
if let Some(link) = s.jdwp_link.lock().as_ref() {
link.release();
}
Ok(())
}
@@ -1331,8 +1389,11 @@ impl DebugManager {
}
pub async fn threads(&self, s: DebugSessionId) -> IdeResult<Vec<ThreadInfo>> {
let body = self.session(s)?.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
Ok(body
let session = self.session(s)?;
// Before sending: a stop that overtakes the reply must not take the reply's ids as its own.
let stop = session.suspended.ticket();
let body = session.conn.request_timeout("threads", Value::Null, REQUEST_TIMEOUT).await?;
let threads: Vec<ThreadInfo> = body
.get("threads")
.and_then(Value::as_array)
.map(|t| {
@@ -1343,11 +1404,16 @@ impl DebugManager {
})
.collect()
})
.unwrap_or_default())
.unwrap_or_default();
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.saw_threads(stop, threads.iter().map(|t| t.id));
}
Ok(threads)
}
pub async fn stack_trace(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<Vec<StackFrame>> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
let body = session
.conn
.request_timeout(
@@ -1356,11 +1422,15 @@ impl DebugManager {
REQUEST_TIMEOUT,
)
.await?;
let kotlin_lsp = session.kotlin_lsp_quirks().is_some();
let frame = |f: &Value| {
let mut frame = frame_from(f);
if let Some(client) = frame.path.as_deref().and_then(|p| session.client_path(&p.to_string_lossy())) {
frame.path = Some(client.into());
}
if kotlin_lsp {
frame.name = kotlin_lsp::frame_name(&frame.name).to_owned();
}
frame
};
Ok(body.get("stackFrames").and_then(Value::as_array).map(|f| f.iter().map(frame).collect()).unwrap_or_default())
@@ -1472,8 +1542,15 @@ impl DebugManager {
pub async fn resume(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.resume_sent();
}
let ticket = session.suspended.ticket();
let body = session.conn.request_timeout("continue", json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
// DAP: a response without the flag continued every thread.
let all = body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true);
session.suspended.resumed(ticket, thread_id, all);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: all });
Ok(())
}
@@ -1513,7 +1590,15 @@ impl DebugManager {
StepKind::Out => "stepOut",
};
let session = self.session(s)?;
session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await?;
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
let sent = session.conn.request_timeout(command, json!({ "threadId": thread_id }), REQUEST_TIMEOUT).await;
step_settled(&session, sent)?;
// A step runs its thread; other threads stopped by their own events stay stopped.
session.suspended.resumed(ticket, thread_id, false);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1530,6 +1615,8 @@ impl DebugManager {
force: bool,
) -> IdeResult<()> {
let session = self.session(s)?;
// Before touching breakpoints: a refused `continue` would leave them replaced.
session.check_thread(thread_id)?;
if force {
self.suspend_breakpoints(&session, Some(&path)).await?;
}
@@ -1604,6 +1691,7 @@ impl DebugManager {
/// Force Step Over: `next` with every breakpoint suspended until the step ends.
pub async fn force_step_over(&self, s: DebugSessionId, thread_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session.check_thread(thread_id)?;
self.suspend_breakpoints(&session, None).await?;
self.step(s, thread_id, StepKind::Over).await
}
@@ -1611,10 +1699,17 @@ impl DebugManager {
/// Step into a specific call of the current line (Smart Step Into).
pub async fn step_in_target(&self, s: DebugSessionId, thread_id: i64, target_id: i64) -> IdeResult<()> {
let session = self.session(s)?;
session
session.check_thread(thread_id)?;
if let Some(quirks) = session.kotlin_lsp_quirks() {
quirks.step_sent();
}
let ticket = session.suspended.ticket();
let sent = session
.conn
.request_timeout("stepIn", json!({ "threadId": thread_id, "targetId": target_id }), REQUEST_TIMEOUT)
.await?;
.await;
step_settled(&session, sent)?;
session.suspended.resumed(ticket, thread_id, false);
session.emit(ide_core::event::DebugEvent::Continued { thread_id, all_threads_continued: true });
Ok(())
}
@@ -1648,10 +1743,68 @@ impl DebugManager {
}
pub fn capabilities(&self, s: DebugSessionId) -> IdeResult<DebugCapabilities> {
Ok(session_capabilities(&*self.session(s)?))
}
/// Class table of the session's JVM (`GC.class_histogram -all`, or counted by the debugger
/// while suspended where no histogram can be taken), with deltas against the session's
/// previous snapshot from the same source.
pub async fn jvm_memory_classes(&self, s: DebugSessionId) -> IdeResult<JvmMemoryResult> {
let session = self.session(s)?;
let mut caps = capabilities_from(&session.caps.lock());
caps.hot_code_replace = HOT_CODE_REPLACE_ADAPTERS.contains(&session.adapter.as_str());
Ok(caps)
jvm_memory::session_classes(session).await
}
/// Writes a heap dump of the session's JVM to `dir/dump.hprof`, one class-table or dump
/// request per session at a time. Anything but `Ready` deletes `dir`, unless `s` is not a
/// session. On `cancel` the result is `Cancelled` at once, while a dump the JVM is writing
/// still holds the session's turn until the JVM answers.
pub async fn jvm_heap_dump(
&self,
s: DebugSessionId,
dir: PathBuf,
cancel: ide_core::CancellationToken,
) -> IdeResult<JvmHeapOutcome> {
let session = self.session(s)?;
jvm_memory::session_heap_dump(session, dir, cancel).await
}
/// Gives the session the backend for [`Self::jvm_memory_instances`] and the debugger's class
/// counts; the UI is told to reload the session's capabilities.
pub fn set_jvm_inspector(
&self,
s: DebugSessionId,
inspector: Arc<dyn ide_jvm_inspect::JvmInspector>,
) -> IdeResult<()> {
let session = self.session(s)?;
if session.ended.is_cancelled() {
return Err(no_session(s));
}
jvm_memory::set_inspector(&session, inspector);
Ok(())
}
/// Up to `limit` (clamped to `1..=MAX_INSTANCES`) instances of `class_name` while suspended.
pub async fn jvm_memory_instances(
&self,
s: DebugSessionId,
class_name: String,
limit: u32,
) -> IdeResult<JvmInstancesResult> {
jvm_memory::instances(&*self.session(s)?, &class_name, limit).await
}
pub async fn jvm_memory_fields(&self, s: DebugSessionId, object: JvmObjectRef) -> IdeResult<JvmFieldsResult> {
jvm_memory::fields(&*self.session(s)?, object).await
}
/// Objects that point at `object`; `limit` 0 means `DEFAULT_REFERRERS`, at most `MAX_REFERRERS`.
pub async fn jvm_memory_referrers(
&self,
s: DebugSessionId,
object: JvmObjectRef,
limit: u32,
) -> IdeResult<JvmReferrersResult> {
jvm_memory::referrers(&*self.session(s)?, object, limit).await
}
/// HotSwap: reloads the classes compiled since the session started (or since the
+348 -42
Переглянути файл
@@ -3,8 +3,11 @@
use crate::adapter::AdapterSpec;
use crate::connection::{Connection, DapError, Incoming};
use crate::kotlin_lsp;
use crate::{Breakpoint, BreakpointState, DataBreakpoint, DebugManager, FunctionBreakpoint, SessionOwner};
use ide_core::event::DebugEvent;
#[cfg(unix)]
use ide_core::process::process_tree;
use ide_core::{DebugSessionId, IdeEvent, TaskId};
use parking_lot::Mutex;
use serde_json::{Value, json};
@@ -62,6 +65,15 @@ pub(crate) struct Session {
pub stop_force: AtomicBool,
/// `systemProcessId` from the adapter's `process` event: the program being debugged.
pub debuggee_pid: AtomicI64,
/// The JVM an app-launched (Kotlin) session started, or the local JVM an attach session
/// found by its JDWP port. Separate from `debuggee_pid`, which decides how Stop signals the
/// program and is 0 for Kotlin.
pub jvm_pid: AtomicI64,
/// JDWP port of an attach to a local JVM, to look the JVM up again; unset for a remote one.
pub jdwp_port: std::sync::OnceLock<u16>,
/// The attach goes through the listen-mode relay of [`crate::jdwp::listen_for_jvm`].
pub listen_relay: AtomicBool,
pub suspended: Suspension,
/// Pids signalled for the current stop: the reported pid and everything it started (an
/// adapter may report the launcher shell, whose child is the program).
#[cfg_attr(not(unix), allow(dead_code))]
@@ -75,8 +87,107 @@ pub(crate) struct Session {
/// Resolved path → the path as the IDE knows it, for adapters given resolved paths.
resolved_paths: Mutex<HashMap<PathBuf, PathBuf>>,
progress: Mutex<HashMap<String, TaskId>>,
pub kotlin_relay: Mutex<Option<crate::kotlin_jdwp::StartupRelay>>,
/// Relay between a Kotlin adapter and its JVM. Dropping it closes both sockets.
pub jdwp_link: parking_lot::Mutex<Option<ide_jdwp::JdwpLink>>,
pub jvm_inspector: parking_lot::Mutex<Option<Arc<dyn ide_jvm_inspect::JvmInspector>>>,
/// One inspector request per session at a time.
pub inspect_lock: tokio::sync::Mutex<()>,
/// Cancelled when the session ends, so a request still waiting on the inspector lets go of
/// it (and of the JDWP link it holds).
pub ended: ide_core::CancellationToken,
/// `hostName` (empty when absent) and `port` of the session's `attach` request, as the IDE
/// was given them: Kotlin then replaces them with the link's. Unlike `jdwp_port`, also set
/// for a remote JVM.
pub attach_target: std::sync::OnceLock<(String, u16)>,
/// Set once `configure` has recorded where the program's JVM is (the fields above, the
/// Kotlin link), just before the `launch`/`attach` request goes out.
pub jvm_located: watch::Sender<bool>,
pub kotlin_mirror: Mutex<Option<crate::kotlin::SourceMirror>>,
pub jvm_memory: tokio::sync::Mutex<crate::JvmMemoryState>,
/// Used only by a kotlin-lsp session: see [`Session::kotlin_lsp_quirks`].
pub(crate) quirks: kotlin_lsp::Quirks,
}
/// Which threads of the program are suspended: each from its `stopped` event until a
/// `continued` event or a successful resume/step request for it. Never cleared elsewhere, since
/// a stale "suspended" only makes a later check refuse. fwcd and java-debug stop a breakpoint's
/// thread alone, so a thread can run while another stays stopped.
///
/// A resume reply ends only the stops taken before the request was sent: a thread's next
/// `stopped` event can overtake the reply. The check and the removal happen under one lock, so
/// no stop can land between them and be lost.
#[derive(Default)]
pub(crate) struct Suspension(Mutex<Stops>);
#[derive(Default)]
struct Stops {
/// Counts stops, and moves back to an earlier stopped thread: an object read under one
/// ticket is stale under another.
ticket: u64,
/// Threads stopped alone by their event, oldest first, with the ticket of that stop.
threads: Vec<(i64, u64)>,
/// The ticket of a stop of every thread (`allThreadsStopped`, or no thread id): any resume
/// ends it.
all: Option<u64>,
}
impl Stops {
fn latest(&self) -> Option<u64> {
self.threads.last().map(|&(_, ticket)| ticket).max(self.all)
}
/// Ends the stops taken by ticket `upto` of `thread`, or of every thread.
fn release(&mut self, thread: Option<i64>, every: bool, upto: u64) {
let latest = self.latest();
if every || thread.is_none() || self.all.is_some_and(|t| t <= upto) {
self.all = self.all.filter(|&t| t > upto);
self.threads.retain(|&(_, t)| t > upto);
} else {
self.threads.retain(|&(id, t)| Some(id) != thread || t > upto);
}
// Back on an earlier thread's stop, as the JDWP link moves to it with a new generation.
let now = self.latest();
if now.is_some() && now != latest {
self.ticket += 1;
}
}
}
impl Suspension {
/// Whether some thread is stopped.
pub fn get(&self) -> bool {
let stops = self.0.lock();
stops.all.is_some() || !stops.threads.is_empty()
}
/// A `stopped` event; `all`: its `allThreadsStopped`.
pub fn stopped(&self, thread: Option<i64>, all: bool) {
let mut stops = self.0.lock();
stops.ticket += 1;
let ticket = stops.ticket;
match thread.filter(|_| !all) {
Some(thread) => {
stops.threads.retain(|&(id, _)| id != thread);
stops.threads.push((thread, ticket));
}
None => stops.all = Some(ticket),
}
}
/// A `continued` event; `all`: its `allThreadsContinued`.
pub fn continued(&self, thread: Option<i64>, all: bool) {
self.0.lock().release(thread, all, u64::MAX);
}
/// Take before sending a resume-like request and pass to [`Self::resumed`].
pub fn ticket(&self) -> u64 {
self.0.lock().ticket
}
/// After a successful resume-like request for `thread`; `all`: every thread ran.
pub fn resumed(&self, ticket: u64, thread: i64, all: bool) {
self.0.lock().release(Some(thread), all, ticket);
}
}
/// Adapters that only match files by their resolved path: Godot maps a script to
@@ -125,6 +236,10 @@ impl Session {
terminated: AtomicBool::new(false),
stop_force: AtomicBool::new(false),
debuggee_pid: AtomicI64::new(0),
jvm_pid: AtomicI64::new(0),
jdwp_port: std::sync::OnceLock::new(),
listen_relay: AtomicBool::new(false),
suspended: Suspension::default(),
stop_pids: parking_lot::Mutex::new(Vec::new()),
#[cfg(windows)]
stop_tree: Mutex::new(None),
@@ -132,8 +247,27 @@ impl Session {
breakpoint_ids: Mutex::new(HashMap::new()),
resolved_paths: Mutex::new(HashMap::new()),
progress: Mutex::new(HashMap::new()),
kotlin_relay: Mutex::new(None),
jdwp_link: parking_lot::Mutex::new(None),
jvm_inspector: parking_lot::Mutex::new(None),
inspect_lock: tokio::sync::Mutex::default(),
ended: ide_core::CancellationToken::new(),
attach_target: std::sync::OnceLock::new(),
jvm_located: watch::channel(false).0,
kotlin_mirror: Mutex::new(None),
jvm_memory: tokio::sync::Mutex::default(),
quirks: kotlin_lsp::Quirks::default(),
}
}
pub fn kotlin_lsp_quirks(&self) -> Option<&kotlin_lsp::Quirks> {
(self.adapter == kotlin_lsp::ADAPTER).then_some(&self.quirks)
}
/// Call before sending a request for thread `id`.
pub fn check_thread(&self, id: i64) -> ide_core::IdeResult<()> {
match self.kotlin_lsp_quirks() {
Some(quirks) => quirks.check_thread(self.suspended.ticket(), self.suspended.get(), id),
None => Ok(()),
}
}
@@ -165,6 +299,11 @@ impl Session {
/// A path from the adapter as the IDE knows it (undoes [`Session::adapter_path`]).
pub fn client_path(&self, path: &str) -> Option<PathBuf> {
if self.adapter == kotlin_lsp::ADAPTER
&& let Some(path) = kotlin_lsp::source_path(path)
{
return Some(path);
}
if let Some(real) = self.kotlin_mirror.lock().as_ref().and_then(|m| m.client_path(Path::new(path))) {
return Some(real);
}
@@ -294,7 +433,10 @@ impl Session {
/// Kills the adapter and every debuggee we started.
pub fn kill_processes(&self) {
self.kotlin_relay.lock().take();
self.ended.cancel();
// The inspector holds a clone of the link, so both go.
self.jvm_inspector.lock().take();
self.jdwp_link.lock().take();
#[cfg(unix)]
if self.request == "launch" {
for group in launch_groups(self) {
@@ -351,32 +493,6 @@ pub(crate) async fn remember_debuggee(s: &Arc<Session>) {
let _ = s;
}
/// The program and everything it started: an adapter may report the launcher shell, whose
/// child is the debugged program, so stopping only the reported pid leaves the program alive.
#[cfg(unix)]
fn process_tree(pid: i32) -> Vec<i32> {
use std::collections::HashMap;
let Ok(out) = std::process::Command::new("ps").args(["-A", "-o", "pid=,ppid="]).output() else {
return vec![pid];
};
let mut children: HashMap<i32, Vec<i32>> = HashMap::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
let mut it = line.split_whitespace();
let (Some(p), Some(parent)) = (it.next(), it.next()) else { continue };
let (Ok(p), Ok(parent)) = (p.parse::<i32>(), parent.parse::<i32>()) else { continue };
children.entry(parent).or_default().push(p);
}
let mut tree = vec![pid];
let mut at = 0;
while at < tree.len() {
if let Some(kids) = children.get(&tree[at]) {
tree.extend(kids.iter().copied());
}
at += 1;
}
tree
}
/// Asks the program itself to exit (`force` = SIGKILL) without ending the adapter session:
/// its output keeps flowing over DAP while it shuts down. The first call remembers the pid
/// tree, because the reported pid may die before its program does.
@@ -600,18 +716,31 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
s.emit(DebugEvent::Initialized);
}
"stopped" => {
let all = body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false);
s.suspended.stopped(int("threadId"), all);
let mut reason = str_("reason").unwrap_or_else(|| "pause".into());
if let Some(quirks) = s.kotlin_lsp_quirks() {
quirks.saw_threads(s.suspended.ticket(), int("threadId"));
reason = quirks.stop_reason(reason);
}
mgr.restore_breakpoints(s).await;
s.emit(DebugEvent::Stopped {
thread_id: int("threadId").unwrap_or(0),
reason: str_("reason").unwrap_or_else(|| "pause".into()),
reason,
description: str_("description").or_else(|| str_("text")),
all_threads_stopped: body.get("allThreadsStopped").and_then(Value::as_bool).unwrap_or(false),
all_threads_stopped: all,
});
}
"continued" => {
let all = body.get("allThreadsContinued").and_then(Value::as_bool);
// DAP: an event without the flag continues its thread alone. The UI keeps treating
// it as all threads; the suspension, which holds only threads that stopped, need not.
s.suspended.continued(int("threadId"), all.unwrap_or(false));
s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: all.unwrap_or(true),
});
}
"continued" => s.emit(DebugEvent::Continued {
thread_id: int("threadId").unwrap_or(0),
all_threads_continued: body.get("allThreadsContinued").and_then(Value::as_bool).unwrap_or(true),
}),
"output" => {
let category = str_("category").unwrap_or_else(|| "console".into());
if category == "telemetry" {
@@ -695,6 +824,14 @@ async fn on_event(mgr: &Arc<DebugManager>, s: &Arc<Session>, event: &str, mut bo
change_type: crate::hot_code_replace_type(body.get("changeType")),
message: str_("message").unwrap_or_default(),
}),
// Sent only by adapters told `supportsInvalidatedEvent` (kotlin-lsp).
"invalidated" => s.emit(DebugEvent::Invalidated {
areas: body
.get("areas")
.and_then(Value::as_array)
.map(|a| a.iter().filter_map(Value::as_str).map(str::to_owned).collect())
.unwrap_or_default(),
}),
_ => tracing::trace!("dap: ignored event {event}"),
}
}
@@ -917,6 +1054,19 @@ fn forward_output<R: AsyncRead + Unpin + Send + 'static>(s: &Session, mut stream
});
}
/// The attach listener must be on the command line before the user's arguments, which may
/// override it; the memory view needs it because the JVM is started by the IDE, not attached to.
/// Not on Windows (`windows`), which the memory view does not support.
fn kotlin_jvm_args(user: Vec<String>, windows: bool) -> Vec<String> {
if windows || !ide_jvm_attach::wants_listener_flag(&user.join(" ")) {
return user;
}
let mut args = Vec::with_capacity(user.len() + 1);
args.push(ide_jvm_attach::START_LISTENER_FLAG.to_owned());
args.extend(user);
args
}
/// Launch with the IDE's resolved runtime and classpath; the adapter only attaches.
pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) -> ide_core::IdeResult<()> {
use tokio::io::{AsyncBufReadExt, BufReader};
@@ -931,7 +1081,7 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
std::env::join_paths(strings("classPaths")).map_err(|e| ide_core::IdeError::invalid(e.to_string()))?;
let mut cmd = tokio::process::Command::new(java);
ide_core::process::prepare(cmd.as_std_mut());
cmd.args(strings("vmArguments"))
cmd.args(kotlin_jvm_args(strings("vmArguments"), cfg!(windows)))
// Let the JVM pick its own port (`:0`) and report it below: pre-picking a free
// port here races with other processes that may take it before the JVM binds.
.arg("-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=127.0.0.1:0")
@@ -965,11 +1115,12 @@ pub(crate) async fn start_kotlin_debuggee(s: &Arc<Session>, args: &mut Value) ->
// forced stop), the session's job on Windows, so a stop reaches what it forks.
ide_core::process::own_group(cmd.as_std_mut());
let mut child = cmd.spawn()?;
#[cfg(windows)]
if let Some(pid) = child.id()
&& let Some(job) = s.job.lock().as_ref()
{
job.assign_pid(pid);
if let Some(pid) = child.id() {
s.jvm_pid.store(i64::from(pid), Ordering::Release);
#[cfg(windows)]
if let Some(job) = s.job.lock().as_ref() {
job.assign_pid(pid);
}
}
if let Some(err) = child.stderr.take() {
forward_output(s, err, "stderr");
@@ -1010,7 +1161,162 @@ fn parse_jdwp_port(line: &str) -> Option<u16> {
#[cfg(test)]
mod tests {
use super::parse_jdwp_port;
use super::{Suspension, kotlin_jvm_args, parse_jdwp_port};
#[test]
fn a_resume_reply_clears_the_stop_it_was_sent_for() {
let s = Suspension::default();
assert!(!s.get());
s.stopped(Some(1), true);
let ticket = s.ticket();
s.resumed(ticket, 1, true);
assert!(!s.get());
}
#[test]
fn a_stop_after_the_resume_request_keeps_the_session_suspended() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
// The next breakpoint's `stopped` event overtakes the reply to `continue`.
s.continued(Some(1), true);
s.stopped(Some(1), true);
s.resumed(ticket, 1, true);
assert!(s.get());
// A reply for the new stop still clears it.
s.resumed(s.ticket(), 1, true);
assert!(!s.get());
}
#[test]
fn a_reply_after_a_continued_event_changes_nothing() {
let s = Suspension::default();
s.stopped(Some(1), true);
let ticket = s.ticket();
s.continued(Some(1), true);
s.resumed(ticket, 1, true);
assert!(!s.get());
s.stopped(Some(1), true);
assert!(s.get());
}
#[test]
fn concurrent_stops_are_never_lost() {
let s = std::sync::Arc::new(Suspension::default());
s.stopped(Some(1), true);
let ticket = s.ticket();
let stopping = {
let s = s.clone();
std::thread::spawn(move || (0..10_000).for_each(|_| s.stopped(Some(1), true)))
};
for _ in 0..10_000 {
s.resumed(ticket, 1, true);
}
stopping.join().unwrap();
// However they interleave, a resume for the old ticket never clears a later stop.
assert!(s.get());
assert_eq!(s.ticket(), ticket + 10_000);
}
/// fwcd and java-debug stop a breakpoint's thread alone (`SUSPEND_EVENT_THREAD`): 7, then 8.
fn two_threads_stopped() -> Suspension {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), false);
s
}
#[test]
fn continuing_the_latest_stopped_thread_keeps_the_other_as_a_new_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(8), false);
assert!(s.get());
// Objects read on thread 8's stop are stale.
assert_eq!(s.ticket(), ticket + 1);
s.continued(Some(7), false);
assert!(!s.get());
}
#[test]
fn continuing_an_earlier_stopped_thread_keeps_the_latest_stop() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(7), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn continuing_all_threads_ends_every_stop() {
let s = two_threads_stopped();
s.continued(Some(7), true);
assert!(!s.get());
let s = two_threads_stopped();
s.continued(None, false);
assert!(!s.get());
}
#[test]
fn continuing_a_thread_that_did_not_stop_changes_nothing() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.continued(Some(5), false);
assert!(s.get());
assert_eq!(s.ticket(), ticket);
}
#[test]
fn a_step_reply_resumes_its_thread_alone() {
let s = two_threads_stopped();
let ticket = s.ticket();
s.resumed(ticket, 8, false);
assert!(s.get());
assert_eq!(s.ticket(), ticket + 1);
s.resumed(s.ticket(), 7, true);
assert!(!s.get());
}
#[test]
fn a_thread_stopping_again_before_its_resume_reply_stays_stopped() {
let s = two_threads_stopped();
let ticket = s.ticket();
// Thread 7's step ends, and its `stopped` event overtakes the reply to `next`.
s.stopped(Some(7), false);
s.resumed(ticket, 7, false);
s.continued(Some(8), false);
assert!(s.get());
}
#[test]
fn a_stop_of_every_thread_ends_with_any_resume() {
let s = Suspension::default();
s.stopped(Some(7), false);
s.stopped(Some(8), true);
s.continued(Some(8), false);
assert!(!s.get());
// A stop without a thread.
s.stopped(None, false);
s.resumed(s.ticket(), 5, false);
assert!(!s.get());
}
#[test]
fn kotlin_launch_starts_attach_listener_first() {
assert_eq!(
kotlin_jvm_args(vec!["-Xmx1g".into()], false),
vec!["-XX:+StartAttachListener".to_string(), "-Xmx1g".into()]
);
assert_eq!(
kotlin_jvm_args(vec!["-XX:+DisableAttachMechanism".into()], false),
vec!["-XX:+DisableAttachMechanism".to_string()]
);
}
#[test]
fn kotlin_launch_on_windows_gets_no_attach_listener() {
assert_eq!(kotlin_jvm_args(vec!["-Xmx1g".into()], true), vec!["-Xmx1g".to_string()]);
}
#[test]
fn reads_the_port_the_jvm_chose() {
+762 -23
Переглянути файл
@@ -1,5 +1,7 @@
//! Protocol flow against an in-memory fake adapter.
mod jvm_inspect;
use super::*;
use crate::adapter::{AdapterIo, LaunchFuture};
use ide_core::IdeEvent;
@@ -120,6 +122,28 @@ struct Script {
supports_terminate: bool,
/// Answer `setBreakpoints` only once this many are waiting (a client that waits for each never gets one).
hold_set_breakpoints: usize,
/// Send a `continued` event right after the first `stopped` one.
continued_after_stop: bool,
/// Close the connection when a `pause` request arrives (the adapter dies).
hang_up_on_pause: bool,
/// Close the connection at the first request after the first `stopped` event.
hang_up_after_stop: bool,
/// Answer `continue` with this body and keep the program running, instead of letting it exit.
continue_reply: Option<Value>,
/// Answer `continue` and stop at a breakpoint again, instead of letting the program exit.
stop_after_continue: bool,
/// The `reason` of the stop a `next` ends with (`step` when unset; kotlin-lsp says `breakpoint`).
step_reason: Option<&'static str>,
/// Refuse `next`, then stop at a breakpoint (one the program hit meanwhile).
refuse_next: bool,
/// Send `invalidated { areas: ["variables"] }` right after the first `stopped` event.
invalidated_after_stop: bool,
/// On `threads`: stop again on thread 2 first, then answer with thread 1 of the earlier stop.
stop_during_threads: bool,
/// Name of the first stack frame (`main` when unset).
frame_name: Option<&'static str>,
/// Answer `initialize` only after this long (a slow adapter keeps the session connecting).
delay_initialize: Option<Duration>,
initialized_delay: Duration,
launch_delay: Duration,
/// Leave startup pending before `initialized` (false) or after configuration (true).
@@ -144,6 +168,17 @@ impl Default for Script {
output_after_terminate: false,
supports_terminate: true,
hold_set_breakpoints: 0,
continued_after_stop: false,
hang_up_on_pause: false,
hang_up_after_stop: false,
continue_reply: None,
stop_after_continue: false,
step_reason: None,
refuse_next: false,
invalidated_after_stop: false,
stop_during_threads: false,
frame_name: None,
delay_initialize: None,
initialized_delay: Duration::ZERO,
launch_delay: Duration::ZERO,
pending_startup: None,
@@ -188,6 +223,7 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
let mut seq = 1000;
let mut pending_launch: Option<i64> = None;
let mut held: Vec<Value> = Vec::new();
let mut stopped = false;
let mut running = false;
macro_rules! send {
($v:expr) => {{
@@ -208,8 +244,14 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
if msg["type"] == "response" {
continue;
}
if script.hang_up_after_stop && stopped {
break;
}
match msg["command"].as_str().unwrap_or_default() {
"initialize" => {
if let Some(delay) = script.delay_initialize {
tokio::time::sleep(delay).await;
}
ok!(
msg,
json!({
@@ -303,9 +345,18 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
ok!(msg, Value::Null);
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "restart", "threadId": 1}}));
}
"next" if script.refuse_next => {
send!(
json!({"type": "response", "request_seq": msg["seq"], "command": "next", "success": false, "message": "refused"})
);
// Lets the client handle the refusal before the stop.
tokio::time::sleep(Duration::from_millis(50)).await;
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1}}));
}
"next" => {
ok!(msg, Value::Null);
send!(json!({"type": "event", "event": "stopped", "body": {"reason": "step", "threadId": 1}}));
let reason = script.step_reason.unwrap_or("step");
send!(json!({"type": "event", "event": "stopped", "body": {"reason": reason, "threadId": 1}}));
}
"setVariable" => {
ok!(msg, json!({"value": format!("{}!", msg["arguments"]["value"].as_str().unwrap()), "type": "int"}))
@@ -341,15 +392,32 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
stopped = true;
if script.invalidated_after_stop {
send!(
json!({"type": "event", "event": "invalidated", "body": {"areas": ["variables"], "threadId": 1}})
);
}
if script.continued_after_stop {
send!(json!({"type": "event", "event": "continued", "body": {"threadId": 1}}));
}
send!(
json!({"type": "event", "event": "breakpoint", "body": {"reason": "changed", "breakpoint": {"id": 1, "verified": false, "message": "moved", "line": 9, "source": {"path": script.source_path}}}})
);
}
"threads" if script.stop_during_threads => {
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 2, "allThreadsStopped": true}})
);
// Lets the client handle the stop before it reads the reply.
tokio::time::sleep(Duration::from_millis(50)).await;
ok!(msg, json!({"threads": [{"id": 1, "name": "\"main\"@1"}]}));
}
"threads" => ok!(msg, json!({"threads": [{"id": 1, "name": "MainThread"}]})),
"stackTrace" => ok!(
msg,
json!({"stackFrames": [
{"id": 10, "name": "main", "line": 5, "column": 3, "source": {"path": script.source_path.as_deref().unwrap_or("/p/a.py")}},
{"id": 10, "name": script.frame_name.unwrap_or("main"), "line": 5, "column": 3, "source": {"path": script.source_path.as_deref().unwrap_or("/p/a.py")}},
{"id": 11, "name": "lib", "line": 1, "column": 1, "source": {"path": "/usr/lib/x.py", "presentationHint": "deemphasize"}},
]})
),
@@ -372,11 +440,19 @@ async fn fake_adapter<S: AsyncRead + AsyncWrite + Unpin + Send + 'static>(stream
{"label": ""},
]})
),
"continue" if script.continue_reply.is_some() => ok!(msg, script.continue_reply.clone().unwrap()),
"continue" if script.stop_after_continue => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(
json!({"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "allThreadsStopped": true}})
);
}
"continue" => {
ok!(msg, json!({"allThreadsContinued": true}));
send!(json!({"type": "event", "event": "exited", "body": {"exitCode": 3}}));
send!(json!({"type": "event", "event": "terminated"}));
}
"pause" if script.hang_up_on_pause => break,
"stepIn" | "stepOut" | "pause" => ok!(msg, Value::Null),
"redefineClasses" => {
send!(
@@ -430,6 +506,8 @@ fn harness(script: Script) -> Harness {
Arc::new(LanguageRegistry::from_configs(Vec::new())),
Arc::new(FakeLauncher { log: log.clone(), script }),
);
// A local attach must not send requests to the developer's own JVMs.
manager.set_jvm_finder(Arc::new(|_, _| None));
Harness { manager, log, events, owner }
}
@@ -1352,6 +1430,36 @@ async fn hot_swap_only_for_java_debug() {
assert_eq!(failed.error_message.as_deref(), Some("boom"));
}
/// A loopback port nothing listens on until `listen` is called on the socket, and that no parallel
/// test can bind meanwhile. A port picked by binding to 0 and closing the listener can be taken
/// by another test before the code under test connects to it.
fn reserved_port() -> (tokio::net::TcpSocket, u16) {
// Without SO_REUSEADDR, which every `TcpListener::bind` sets: no other socket may bind it.
let socket = tokio::net::TcpSocket::new_v4().unwrap();
socket.bind(([127, 0, 0, 1], 0).into()).unwrap();
let port = socket.local_addr().unwrap().port();
(socket, port)
}
/// An editor that starts listening on the reserved port after `delay` and serves one session.
#[cfg(unix)]
fn fake_editor(socket: tokio::net::TcpSocket, delay: Duration, log: Log) {
tokio::spawn(async move {
tokio::time::sleep(delay).await;
let listener = socket.listen(1).unwrap();
let (stream, _) = listener.accept().await.unwrap();
// Closed once its session is accepted, so a later start finds no editor open.
drop(listener);
fake_adapter(stream, log, Script::default()).await;
});
}
#[tokio::test]
async fn a_reserved_port_cannot_be_taken_by_a_parallel_test() {
let (_socket, port) = reserved_port();
assert!(tokio::net::TcpListener::bind(("127.0.0.1", port)).await.is_err(), "port {port} is still free to bind");
}
fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegistry> {
let lang = ide_language::LanguageConfig {
id: ide_core::LanguageId::new("gdscript"),
@@ -1364,6 +1472,7 @@ fn godot_registry(port: u16, command: &str, args: &[&str]) -> Arc<LanguageRegist
indent: None,
ui_mode: None,
framework_ui_modes: Vec::new(),
framework_files: Vec::new(),
formatter: None,
scratch_template: None,
tree_sitter: None,
@@ -1419,7 +1528,7 @@ async fn godot_connects_to_the_editor_adapter() {
#[tokio::test]
async fn godot_without_editor_or_executable_says_what_to_do() {
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (_no_editor, port) = reserved_port();
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(godot_registry(port, "openidea-no-such-godot", &[]));
let err = manager.start(godot_launch(&std::env::temp_dir()), owner).await.unwrap_err();
@@ -1432,15 +1541,9 @@ async fn godot_without_editor_or_executable_says_what_to_do() {
#[cfg(unix)]
#[tokio::test]
async fn godot_starts_a_headless_editor_when_none_is_open() {
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (editor, port) = reserved_port();
let log: Log = Arc::default();
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
fake_editor(editor, Duration::from_millis(300), log.clone());
let (owner, _events) = recording_owner("main");
// `sleep` stands in for `godot --headless --editor --path ${root} …`.
let manager = DebugManager::new(godot_registry(port, "sleep", &["30"]));
@@ -1643,7 +1746,7 @@ async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
std::fs::write(&program, "#!/bin/sh\nexec sleep 30\n").unwrap();
std::fs::set_permissions(&program, std::fs::Permissions::from_mode(0o755)).unwrap();
let port = std::net::TcpListener::bind(("127.0.0.1", 0)).unwrap().local_addr().unwrap().port();
let (editor, port) = reserved_port();
let mut gdscript = parse_definition(include_str!("../../../languages/gdscript.toml")).unwrap();
let debugger = gdscript.debugger.as_mut().unwrap();
debugger.args.clear();
@@ -1660,13 +1763,7 @@ async fn a_plugin_language_runs_its_debug_adapter_from_the_approved_program() {
let _language = store.add("acme.godot".into(), gdscript);
let log: Log = Arc::default();
let server_log = log.clone();
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
let listener = tokio::net::TcpListener::bind(("127.0.0.1", port)).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
fake_adapter(stream, server_log, Script::default()).await;
});
fake_editor(editor, Duration::from_millis(300), log.clone());
let (owner, _events) = recording_owner("main");
let manager = DebugManager::new(registry.clone());
let id = manager.start(godot_launch(&dir), owner).await.unwrap();
@@ -1999,25 +2096,667 @@ async fn stop_kills_the_program_started_by_the_reported_launcher() {
fn delve_leftovers_are_removed_without_touching_other_files() {
let dir = std::env::temp_dir().join(format!("openidea-delve-bin-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
for name in ["__debug_bin", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
for name in ["__debug_bin", "__debug_bin833767175", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
std::fs::write(dir.join(name), b"bin").unwrap();
}
// Lookalikes that are not Delve's own binaries stay: sources, a future
// in-progress build, and a directory that happens to share the name.
for name in ["__debug_bin.go", "__debug_bin.go.exe", "main.go"] {
for name in ["__debug_bin.go", "__debug_bin.go.exe", "__debug_bin12.go", "main.go"] {
std::fs::write(dir.join(name), b"keep").unwrap();
}
std::fs::create_dir(dir.join("__debug_bin.dir")).unwrap();
remove_delve_binaries(&dir);
for name in ["__debug_bin", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
for name in ["__debug_bin", "__debug_bin833767175", "__debug_bin.exe", "__debug_bin.exe3323301925"] {
assert!(!dir.join(name).exists(), "{name} should be gone");
}
for name in ["__debug_bin.go", "__debug_bin.go.exe", "main.go", "__debug_bin.dir"] {
for name in ["__debug_bin.go", "__debug_bin.go.exe", "__debug_bin12.go", "main.go", "__debug_bin.dir"] {
assert!(dir.join(name).exists(), "{name} should stay");
}
std::fs::remove_dir_all(&dir).unwrap();
}
fn is_suspended(h: &Harness, id: DebugSessionId) -> bool {
h.manager.session(id).unwrap().suspended.get()
}
#[tokio::test]
async fn stopped_event_marks_session_suspended() {
let h = harness(Script::default());
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert!(is_suspended(&h, id));
}
#[tokio::test]
async fn resume_clears_suspended_without_continued_event() {
let h = harness(Script::default());
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
// The fake adapter ends the session after `continue`, so keep the session alive here.
let session = h.manager.session(id).unwrap();
assert!(session.suspended.get());
h.manager.resume(id, 1).await.unwrap();
assert!(!session.suspended.get());
}
/// After the fake adapter's stop of every thread ends, threads 7 then 8 stop alone, as a
/// breakpoint does on fwcd and java-debug (`SUSPEND_EVENT_THREAD`).
async fn threads_stopped(script: Script) -> (Harness, Arc<Session>, DebugSessionId) {
let h = harness(script);
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
session.suspended.continued(Some(1), true);
session.suspended.stopped(Some(7), false);
session.suspended.stopped(Some(8), false);
(h, session, id)
}
#[tokio::test]
async fn resume_of_one_thread_keeps_the_others_suspended() {
let only_it = Script { continue_reply: Some(json!({"allThreadsContinued": false})), ..Script::default() };
let (h, session, id) = threads_stopped(only_it).await;
h.manager.resume(id, 7).await.unwrap();
assert!(session.suspended.get());
h.manager.resume(id, 8).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn resume_reply_without_the_flag_continues_every_thread() {
let (h, session, id) = threads_stopped(Script { continue_reply: Some(json!({})), ..Script::default() }).await;
h.manager.resume(id, 7).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn step_of_one_thread_keeps_the_others_suspended() {
let (h, session, id) = threads_stopped(Script::default()).await;
let ticket = session.suspended.ticket();
h.manager.step(id, 8, StepKind::Into).await.unwrap();
assert!(session.suspended.get());
// Thread 8 was the latest stop: what was read on it is stale.
assert_eq!(session.suspended.ticket(), ticket + 1);
h.manager.step_in_target(id, 7, 1).await.unwrap();
assert!(!session.suspended.get());
}
#[tokio::test]
async fn continued_event_clears_suspended() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(config()).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
assert!(!is_suspended(&h, id));
}
fn java_config(request: &str) -> LaunchConfig {
LaunchConfig { adapter: "java-debug".into(), request: request.into(), extra_json: None, ..config() }
}
fn remote_attach(adapter: &str, project: &std::path::Path) -> LaunchConfig {
LaunchConfig {
adapter: adapter.into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(json!({"projectRoot": project, "hostName": "10.0.0.5", "port": 5005}).to_string()),
..java_config("attach")
}
}
#[tokio::test]
async fn attach_session_reports_capability() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
for adapter in ["java-debug", kotlin::ADAPTER] {
let id = h.start(remote_attach(adapter, project.path())).await.unwrap();
let caps = h.manager.capabilities(id).unwrap();
assert!(caps.jvm_memory, "{adapter}");
assert_eq!(caps.jvm_heap, caps.jvm_memory, "{adapter}");
}
let id = h.start(config()).await.unwrap();
let caps = h.manager.capabilities(id).unwrap();
assert!(!caps.jvm_memory);
assert_eq!(caps.jvm_heap, caps.jvm_memory);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::NoJvmProcess, can_count_when_paused: false }
);
}
#[cfg(unix)]
#[tokio::test]
async fn ended_attach_session_without_pid_reports_program_exited() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.manager.session(id).unwrap().terminated.store(true, Ordering::Release);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::ProgramExited, can_count_when_paused: false }
);
}
#[cfg(unix)]
#[tokio::test]
async fn listen_mode_attach_is_not_scanned() {
let h = harness(Script::default());
let scans = Arc::new(Mutex::new(Vec::new()));
let counted = scans.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
counted.lock().push(port);
None
}));
let (_jvm, relay) = jdwp::tests::connected_jvm().await;
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "127.0.0.1", "port": relay}).to_string());
let id = h.start(cfg).await.unwrap();
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable {
reason: JvmMemoryUnavailable::ListenModeNotSupported,
can_count_when_paused: false
}
);
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(scans.lock().is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn remote_host_is_not_scanned() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jdwp_port.get(), None);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable {
reason: JvmMemoryUnavailable::AttachedJvmNotFound,
can_count_when_paused: false
}
);
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 0);
}
/// The Memory view asks as soon as the session appears, while its JVM is still being looked up:
/// answering then would claim there is no JVM to ask.
#[cfg(unix)]
#[tokio::test]
async fn memory_asked_while_an_attach_connects_waits_for_its_jvm() {
let h = harness(Script { delay_initialize: Some(Duration::from_millis(300)), ..Script::default() });
let project = tempfile::tempdir().unwrap();
let (manager, owner) = (h.manager.clone(), h.owner.clone());
let config = remote_attach("java-debug", project.path());
let start = tokio::spawn(async move { manager.start(config, owner).await });
h.wait_event(|e| matches!(e, DebugEvent::Started { .. })).await;
let id = DebugSessionId(*event_sessions(&h.events).first().unwrap());
let session = h.manager.session(id).unwrap();
assert!(session.attach_target.get().is_none(), "asked too late to cover the connecting session");
h.manager.jvm_memory_classes(id).await.unwrap();
assert!(session.attach_target.get().is_some(), "answered before the attach was looked at");
start.await.unwrap().unwrap();
}
/// A JVM clears its JDWP listener address once a debugger connects, so only a scan made before
/// the `attach` request can tell it by its port.
#[cfg(unix)]
#[tokio::test]
async fn local_attach_finds_its_jvm_before_the_debugger_connects() {
let h = harness(Script::default());
let log = h.log.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
std::thread::sleep(Duration::from_millis(50));
let attached = log.lock().iter().any(|m| m["command"] == "attach");
(port == 5005 && !attached).then_some(4242)
}));
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "127.0.0.1", "port": 5005}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn local_attach_remembers_its_jdwp_port() {
let h = harness(Script::default());
let ports = Arc::new(Mutex::new(Vec::new()));
let asked = ports.clone();
h.manager.set_jvm_finder(Arc::new(move |port, _| {
asked.lock().push(port);
Some(4242)
}));
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "localhost", "port": "5005"}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.jdwp_port.get(), Some(&5005));
assert_eq!(h.request("attach")["arguments"]["port"], "5005");
for _ in 0..200 {
if session.jvm_pid.load(Ordering::Acquire) != 0 {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
assert_eq!(ports.lock().first(), Some(&5005));
}
/// A fake `find_jvm_by_jdwp_port` for the JVM 4242, recording whether each scan ran while
/// suspended. A JVM not `safe_while_suspended` (JDK below 17) is skipped by a suspended scan.
#[cfg(unix)]
fn counting_find(
safe_while_suspended: bool,
) -> (Arc<Mutex<Vec<bool>>>, impl Fn(bool) -> Option<u32> + Clone + Send + 'static) {
let scans = Arc::new(Mutex::new(Vec::new()));
let counted = scans.clone();
(scans, move |suspended| {
counted.lock().push(suspended);
(!suspended || safe_while_suspended).then_some(4242)
})
}
#[cfg(unix)]
async fn suspended_remote_attach(h: &Harness) -> Arc<Session> {
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
assert!(session.suspended.get());
session
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_skips_an_old_jdk_while_suspended() {
let h = harness(Script::default());
let session = suspended_remote_attach(&h).await;
let (scans, find) = counting_find(false);
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(!scans.lock().is_empty() && scans.lock().iter().all(|&suspended| suspended));
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 0);
session.suspended.continued(None, true);
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(scans.lock().last(), Some(&false));
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_finds_a_jdk_17_or_later_while_suspended() {
let h = harness(Script::default());
let session = suspended_remote_attach(&h).await;
let (scans, find) = counting_find(true);
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(*scans.lock(), [true]);
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 4242);
}
#[cfg(unix)]
#[tokio::test]
async fn background_scan_keeps_a_known_pid_and_stops_with_the_session() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(remote_attach("java-debug", project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
session.suspended.continued(None, true);
session.jvm_pid.store(7, Ordering::Release);
let watch = |find| {
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5))
};
let (scans, find) = counting_find(true);
tokio::time::timeout(Duration::from_secs(2), watch(find)).await.unwrap().unwrap();
assert!(scans.lock().is_empty(), "a known pid needs no scan");
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 7);
// A refresh stores the pid while this scan is still running.
session.jvm_pid.store(0, Ordering::Release);
let racing = session.clone();
let find = move |_| {
racing.jvm_pid.store(7, Ordering::Release);
Some(4242)
};
let task =
jvm_memory::watch_for_attached_jvm(&session, 5005, find, Duration::from_millis(5), Duration::from_secs(5));
tokio::time::timeout(Duration::from_secs(2), task).await.unwrap().unwrap();
assert_eq!(session.jvm_pid.load(Ordering::Acquire), 7, "a pid stored first is kept");
h.manager.stop(id).await.unwrap();
let (scans, find) = counting_find(true);
tokio::time::timeout(Duration::from_secs(2), watch(find)).await.unwrap().unwrap();
assert!(scans.lock().is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn unknown_pid_reports_no_jvm_process() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_memory);
assert_eq!(
h.manager.jvm_memory_classes(id).await.unwrap(),
JvmMemoryResult::Unavailable { reason: JvmMemoryUnavailable::NoJvmProcess, can_count_when_paused: false }
);
}
fn kotlin_attach(project: &std::path::Path, host: &str, port: u16) -> LaunchConfig {
LaunchConfig {
adapter: kotlin::ADAPTER.into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(json!({"projectRoot": project, "hostName": host, "port": port}).to_string()),
..java_config("attach")
}
}
#[test]
fn link_options_hold_only_for_fwcd() {
assert!(kotlin::link_options(kotlin::ADAPTER).hold_resumes);
assert!(!kotlin::link_options(kotlin_lsp::ADAPTER).hold_resumes);
assert!(!kotlin::link_options("x").hold_resumes);
}
#[tokio::test]
async fn kotlin_session_installs_link_inspector() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some());
assert!(session.jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn attach_target_recorded_before_kotlin_rewrite() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "10.0.0.5", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.attach_target.get(), Some(&("10.0.0.5".to_owned(), 5005)));
let sent = &h.request("attach")["arguments"];
assert_eq!(sent["hostName"], "127.0.0.1");
let link_port = sent["port"].as_u64().unwrap();
assert!(link_port != 0 && link_port != 5005);
}
#[tokio::test]
async fn java_attach_records_target() {
let h = harness(Script::default());
let mut cfg = java_config("attach");
cfg.extra_json = Some(json!({"hostName": "localhost", "port": 5005}).to_string());
let id = h.start(cfg).await.unwrap();
let session = h.manager.session(id).unwrap();
assert_eq!(session.attach_target.get(), Some(&("localhost".to_owned(), 5005)));
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
#[tokio::test]
async fn attach_target_recorded_for_any_adapter() {
let h = harness(Script::default());
let mut cfg = java_config("attach");
// "x" is not a registered adapter; debugpy has no JVM link or memory support.
cfg.adapter = "debugpy".into();
cfg.extra_json = Some(json!({"hostName": "h", "port": 7}).to_string());
let id = h.start(cfg).await.unwrap();
assert_eq!(h.manager.session(id).unwrap().attach_target.get(), Some(&("h".to_owned(), 7)));
}
#[tokio::test]
async fn launch_sessions_have_no_attach_target() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert_eq!(h.manager.session(id).unwrap().attach_target.get(), None);
}
#[tokio::test]
async fn closing_the_session_drops_the_link() {
let h = harness(Script { hang_up_on_pause: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005)).await.unwrap();
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some() && session.jvm_inspector.lock().is_some());
let _ = session.conn.request_timeout("pause", json!({"threadId": 1}), Duration::from_secs(2)).await;
for _ in 0..200 {
if h.manager.session(id).is_err() {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
/// kotlin-lsp's debugger, reached through the language server's `debugServer` port (the fake ignores it).
fn kotlin_lsp_attach(project: &std::path::Path) -> LaunchConfig {
LaunchConfig {
adapter: kotlin_lsp::ADAPTER.into(),
request: "attach".into(),
cwd: Some(project.to_owned().into()),
extra_json: Some(
json!({"debugServer": 4711, "hostName": "127.0.0.1", "port": 5005, "projectRoot": project}).to_string(),
),
..config()
}
}
#[tokio::test]
async fn kotlin_lsp_initializes_as_intellij_jvm() {
let project = tempfile::tempdir().unwrap();
for (config, adapter_id, invalidated) in [
(kotlin_lsp_attach(project.path()), "intellij_jvm", true),
(kotlin_attach(project.path(), "127.0.0.1", 5005), "kotlin-debug-adapter", false),
] {
let h = harness(Script::default());
let id = h.start(config).await.unwrap();
let init = &h.request("initialize")["arguments"];
assert_eq!(init["adapterID"], adapter_id);
assert_eq!(init["supportsInvalidatedEvent"], invalidated, "{adapter_id}");
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn kotlin_lsp_gets_real_paths_and_its_link_holds_nothing() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let path = project.path().join("Main.kt");
std::fs::write(&path, "package org.example\nfun main() {}\n").unwrap();
h.manager.set_breakpoints(path.clone(), vec![bp(1)]).await.unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
assert_eq!(h.request("setBreakpoints")["arguments"]["source"]["path"], json!(path));
let sent = &h.request("attach")["arguments"];
assert_eq!(sent["projectRoot"], json!(project.path()));
assert!(sent.get("debugServer").is_none());
assert_eq!(sent["hostName"], "127.0.0.1");
let link_port = sent["port"].as_u64().unwrap();
assert!(link_port != 0 && link_port != 5005);
let session = h.manager.session(id).unwrap();
assert!(session.kotlin_mirror.lock().is_none());
assert!(session.jdwp_link.lock().is_some() && session.jvm_inspector.lock().is_some());
assert_eq!(session.attach_target.get(), Some(&("127.0.0.1".to_owned(), 5005)));
assert!(!kotlin::link_options(&session.adapter).hold_resumes);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn kotlin_lsp_waits_for_the_configuration_done_reply() {
let project = tempfile::tempdir().unwrap();
// fwcd never answers it (`kotlin_configures_without_configuration_done_response`).
let h = harness(Script { ignore_configuration_done: true, ..Default::default() });
let fwcd = h.start(kotlin_attach(project.path(), "127.0.0.1", 5005));
let id = tokio::time::timeout(Duration::from_secs(2), fwcd).await.unwrap().unwrap();
h.manager.stop(id).await.unwrap();
tokio::time::pause();
let h = harness(Script { ignore_configuration_done: true, ..Default::default() });
let started = tokio::time::Instant::now();
let err = h.start(kotlin_lsp_attach(project.path())).await.unwrap_err();
assert!(started.elapsed() >= REQUEST_TIMEOUT, "{:?}", started.elapsed());
assert!(err.message.contains("configurationDone"), "{err:?}");
assert!(h.manager.sessions(None).is_empty());
}
#[tokio::test]
async fn kotlin_lsp_sessions_have_the_memory_tab() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_memory);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn losing_the_kotlin_lsp_connection_ends_the_session() {
let h = harness(Script { hang_up_after_stop: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some());
let _ = session.conn.request_timeout("threads", Value::Null, Duration::from_secs(2)).await;
h.wait_event(|e| matches!(e, DebugEvent::Terminated)).await;
assert!(h.manager.sessions(None).is_empty());
assert!(session.jdwp_link.lock().is_none());
assert!(session.jvm_inspector.lock().is_none());
}
/// The reasons of the first `count` stops, once that many arrived.
async fn stop_reasons(h: &Harness, count: usize) -> Vec<String> {
for _ in 0..200 {
let reasons: Vec<String> = h
.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event: DebugEvent::Stopped { reason, .. }, .. } => Some(reason.clone()),
_ => None,
})
.collect();
if reasons.len() >= count {
return reasons;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
panic!("{count} stops not received; got {:?}", h.events.lock());
}
#[tokio::test]
async fn a_step_stop_is_reported_as_a_step() {
let project = tempfile::tempdir().unwrap();
for (config, reason) in [(kotlin_lsp_attach(project.path()), "step"), (config(), "breakpoint")] {
let h = harness(Script { step_reason: Some("breakpoint"), ..Script::default() });
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.step(id, 1, StepKind::Over).await.unwrap();
assert_eq!(stop_reasons(&h, 2).await, ["breakpoint", reason]);
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn a_breakpoint_after_continue_stays_a_breakpoint() {
let h = harness(Script { step_reason: Some("breakpoint"), stop_after_continue: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.step(id, 1, StepKind::Over).await.unwrap();
stop_reasons(&h, 2).await;
// The fake ends `stepIn` with no stop: the `continue` after it decides the next one.
h.manager.step(id, 1, StepKind::Into).await.unwrap();
h.manager.resume(id, 1).await.unwrap();
assert_eq!(stop_reasons(&h, 3).await, ["breakpoint", "step", "breakpoint"]);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn a_breakpoint_after_a_refused_step_stays_a_breakpoint() {
let h = harness(Script { refuse_next: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert!(h.manager.step(id, 1, StepKind::Over).await.is_err());
assert_eq!(stop_reasons(&h, 2).await, ["breakpoint", "breakpoint"]);
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn a_thread_from_an_earlier_stop_is_refused() {
let project = tempfile::tempdir().unwrap();
for (config, kotlin_lsp) in [(kotlin_lsp_attach(project.path()), true), (config(), false)] {
let h = harness(Script::default());
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(h.manager.threads(id).await.unwrap().len(), 1);
let started = tokio::time::Instant::now();
let stale = h.manager.stack_trace(id, 99).await;
let asked =
|command: &str| h.log.lock().iter().any(|m| m["command"] == command && m["arguments"]["threadId"] == 99);
if kotlin_lsp {
assert!(stale.is_err() && started.elapsed() < Duration::from_millis(100), "{stale:?}");
assert!(h.manager.step(id, 99, StepKind::Over).await.is_err());
assert!(h.manager.resume(id, 99).await.is_err());
assert!(!asked("stackTrace") && !asked("next") && !asked("continue"));
} else {
assert!(stale.is_ok() && asked("stackTrace"), "{stale:?}");
}
assert!(h.manager.stack_trace(id, 1).await.is_ok());
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn a_threads_reply_from_before_a_stop_does_not_vouch_for_its_ids() {
let h = harness(Script { stop_during_threads: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let threads = h.manager.threads(id).await.unwrap();
// M3 maps kotlin-lsp's threads by these names.
assert_eq!((threads[0].id, threads[0].name.as_str()), (1, "\"main\"@1"));
stop_reasons(&h, 2).await;
assert!(h.manager.stack_trace(id, 1).await.is_err());
assert!(h.manager.stack_trace(id, 2).await.is_ok());
h.manager.stop(id).await.unwrap();
}
#[tokio::test]
async fn kotlin_lsp_frames_get_real_paths() {
let project = tempfile::tempdir().unwrap();
let at = "at com.x.AppKt.main(App.kt:13)";
for (config, path, name) in [
(kotlin_lsp_attach(project.path()), "/p/App.kt", "com.x.AppKt.main(App.kt:13)"),
(java_config("launch"), "file:///p/App.kt", at),
] {
let script = Script { source_path: Some("file:///p/App.kt".into()), frame_name: Some(at), ..Script::default() };
let h = harness(script);
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
let frames = h.manager.stack_trace(id, 1).await.unwrap();
assert_eq!(frames[0].path.as_ref().map(|p| p.0.clone()), Some(PathBuf::from(path)));
assert_eq!(frames[0].name, name);
h.manager.stop(id).await.unwrap();
}
}
#[tokio::test]
async fn invalidated_events_reach_the_client() {
let h = harness(Script { invalidated_after_stop: true, ..Script::default() });
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_lsp_attach(project.path())).await.unwrap();
match h.wait_event(|e| matches!(e, DebugEvent::Invalidated { .. })).await {
DebugEvent::Invalidated { areas } => assert_eq!(areas, ["variables"]),
other => panic!("{other:?}"),
}
h.manager.stop(id).await.unwrap();
}
#[cfg(unix)]
#[tokio::test]
async fn rdbg_drains_test_output_after_dap_closes_before_killing_the_process() {
+401
Переглянути файл
@@ -0,0 +1,401 @@
//! Instances, fields and referrers through a session's JVM inspector.
use super::*;
use crate::jvm_memory::test_support::FakeInspector;
use ide_core::event::JvmBackendState;
use ide_jvm_inspect::{InspectError, JvmValue, VmTarget};
use tokio::io::AsyncReadExt;
fn debug_events(h: &Harness) -> Vec<DebugEvent> {
h.events
.lock()
.iter()
.filter_map(|e| match e {
IdeEvent::Debug { event, .. } => Some(event.clone()),
_ => None,
})
.collect()
}
fn backend_events(h: &Harness) -> Vec<JvmBackendState> {
debug_events(h)
.into_iter()
.filter_map(|e| match e {
DebugEvent::JvmBackend { state } => Some(state),
_ => None,
})
.collect()
}
/// A java-debug session stopped at its breakpoint, inspected by `fake`.
async fn stopped_with(h: &Harness, config: LaunchConfig, fake: &Arc<FakeInspector>) -> DebugSessionId {
let id = h.start(config).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
id
}
fn object(id: u64) -> JvmValue {
JvmValue::Object { id, type_name: "A".into() }
}
const NOT_SUSPENDED: JvmInstancesResult =
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NotSuspended };
#[tokio::test]
async fn instances_need_a_suspended_session() {
let h = harness(Script { continued_after_stop: true, ..Script::default() });
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Continued { .. })).await;
let fake = Arc::new(FakeInspector::default());
h.manager.set_jvm_inspector(id, fake.clone()).unwrap();
assert_eq!(h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(), NOT_SUSPENDED);
assert_eq!(fake.calls(), 0);
}
#[tokio::test]
async fn instances_without_an_inspector_have_no_backend() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
h.wait_event(|e| matches!(e, DebugEvent::Stopped { .. })).await;
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
}
#[tokio::test]
async fn object_from_previous_stop_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let JvmInstancesResult::Ready { generation, instances, truncated } =
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap()
else {
panic!("instances unavailable");
};
assert_eq!((generation, instances, truncated), (1, vec![object(7)], false));
h.manager.session(id).unwrap().suspended.stopped(None, true);
let stale = JvmObjectRef { generation, id: 7 };
assert_eq!(
h.manager.jvm_memory_fields(id, stale).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(
h.manager.jvm_memory_referrers(id, stale, 10).await.unwrap(),
JvmReferrersResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
assert_eq!(fake.calls(), 1, "a stale object never reaches the inspector");
let current = JvmObjectRef { generation: 2, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, current).await.unwrap(), JvmFieldsResult::Ready { fields: Vec::new() });
}
#[tokio::test]
async fn instances_stay_readable_while_another_thread_is_stopped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let session = h.manager.session(id).unwrap();
// Threads 20, 21, then 22 stop alone (SUSPEND_EVENT_THREAD).
session.suspended.continued(Some(1), true);
for thread in [20, 21, 22] {
session.suspended.stopped(Some(thread), false);
}
let instances = || h.manager.jvm_memory_instances(id, "A".into(), 10);
let result = instances().await.unwrap();
let JvmInstancesResult::Ready { generation, .. } = result else { panic!("{result:?}") };
let read_on_22 = JvmObjectRef { generation, id: 7 };
// An earlier thread resumes: the stop on 22 goes on, and so does what was read on it.
session.suspended.continued(Some(20), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Ready { fields: Vec::new() }
);
// The latest one resumes: 21 is still stopped, but what was read on 22 is stale.
session.suspended.continued(Some(22), false);
assert_eq!(
h.manager.jvm_memory_fields(id, read_on_22).await.unwrap(),
JvmFieldsResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
let result = instances().await.unwrap();
assert!(matches!(result, JvmInstancesResult::Ready { .. }), "{result:?}");
}
/// Starts `instances` on a held fake inspector; returns once the inspector was entered.
async fn held_request(
h: &Harness,
id: DebugSessionId,
fake: &Arc<FakeInspector>,
) -> tokio::task::JoinHandle<IdeResult<JvmInstancesResult>> {
let manager = h.manager.clone();
let request = tokio::spawn(async move { manager.jvm_memory_instances(id, "A".into(), 10).await });
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
request
}
#[tokio::test]
async fn stop_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
fake.instances.lock().push(object(7));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.stopped(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn resume_during_request_is_stale() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().suspended.continued(None, true);
fake.release.notify_one();
assert_eq!(
request.await.unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Stale }
);
}
#[tokio::test]
async fn one_request_at_a_time_per_session() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let first = held_request(&h, id, &fake).await;
let manager = h.manager.clone();
let second = tokio::spawn(async move { manager.jvm_memory_instances(id, "B".into(), 10).await });
tokio::time::sleep(Duration::from_millis(50)).await;
assert_eq!(fake.calls(), 1, "the second request waits for the first");
fake.release.notify_one();
first.await.unwrap().unwrap();
tokio::time::timeout(Duration::from_secs(2), fake.entered.notified()).await.unwrap();
fake.release.notify_one();
second.await.unwrap().unwrap();
assert_eq!(fake.calls(), 2);
}
#[tokio::test]
async fn session_end_lets_go_of_a_waiting_inspector() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector { hold: true, ..FakeInspector::default() });
let id = stopped_with(&h, java_config("launch"), &fake).await;
let request = held_request(&h, id, &fake).await;
h.manager.session(id).unwrap().kill_processes();
assert_eq!(
tokio::time::timeout(Duration::from_secs(2), request).await.unwrap().unwrap().unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::BackendClosed }
);
assert_eq!(Arc::strong_count(&fake), 1, "no clone of the inspector outlives the session");
}
#[tokio::test]
async fn target_carries_pid_for_launch() {
let h = harness(Script { debuggee_pid: Some(4242), java_process_id: true, ..Script::default() });
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: Some(4242), attach_host: None, attach_port: None, generation: 1 }]
);
}
#[tokio::test]
async fn target_carries_attach_port_and_host() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap();
assert_eq!(
*fake.targets.lock(),
[VmTarget { pid: None, attach_host: Some("10.0.0.5".into()), attach_port: Some(5005), generation: 1 }]
);
}
#[tokio::test]
async fn unsupported_backend_turns_instances_off() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Unsupported("x".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let before = debug_events(&h).len();
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::Unsupported { detail: "x".into() } }
);
let after: Vec<_> = debug_events(&h).split_off(before);
assert!(
matches!(
after.as_slice(),
[DebugEvent::JvmBackend { state: JvmBackendState::Unsupported { detail } }, DebugEvent::Capabilities]
if detail == "x"
),
"{after:?}"
);
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
assert_eq!(
h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap(),
JvmInstancesResult::Unavailable { reason: JvmInspectUnavailable::NoBackend }
);
assert_eq!(fake.calls(), 1);
}
/// `error` is reported but leaves the backend on.
async fn keeps_the_backend(error: InspectError, reason: JvmInspectUnavailable) {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(error));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let object = JvmObjectRef { generation: 1, id: 7 };
assert_eq!(h.manager.jvm_memory_fields(id, object).await.unwrap(), JvmFieldsResult::Unavailable { reason });
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(backend_events(&h).is_empty());
assert!(h.manager.session(id).unwrap().jvm_inspector.lock().is_some());
}
#[tokio::test]
async fn collected_keeps_the_backend() {
keeps_the_backend(InspectError::Collected, JvmInspectUnavailable::Collected).await;
}
#[tokio::test]
async fn timed_out_keeps_the_backend() {
keeps_the_backend(InspectError::TimedOut, JvmInspectUnavailable::TimedOut).await;
}
#[tokio::test]
async fn failed_inspection_is_an_error() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::failing(InspectError::Failed("boom".into())));
let id = stopped_with(&h, java_config("launch"), &fake).await;
let err = h.manager.jvm_memory_instances(id, "A".into(), 10).await.unwrap_err();
assert!(err.to_string().contains("boom"), "{err}");
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
}
#[tokio::test]
async fn limits_are_clamped() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
let id = stopped_with(&h, java_config("launch"), &fake).await;
for limit in [5000, 0] {
h.manager.jvm_memory_instances(id, "A".into(), limit).await.unwrap();
}
let object = JvmObjectRef { generation: 1, id: 7 };
for limit in [0, 900, 20] {
h.manager.jvm_memory_referrers(id, object, limit).await.unwrap();
}
assert_eq!(*fake.limits.lock(), [1000, 1, 50, 200, 20]);
}
#[tokio::test]
async fn a_full_page_is_truncated() {
let h = harness(Script::default());
let fake = Arc::new(FakeInspector::default());
fake.instances.lock().extend([object(1), object(2)]);
let id = stopped_with(&h, java_config("launch"), &fake).await;
let truncated = |r: JvmInstancesResult| match r {
JvmInstancesResult::Ready { truncated, .. } => truncated,
other => panic!("{other:?}"),
};
assert!(truncated(h.manager.jvm_memory_instances(id, "A".into(), 2).await.unwrap()));
assert!(!truncated(h.manager.jvm_memory_instances(id, "A".into(), 3).await.unwrap()));
}
#[tokio::test]
async fn capability_follows_the_inspector() {
let h = harness(Script::default());
let id = h.start(java_config("launch")).await.unwrap();
assert!(!h.manager.capabilities(id).unwrap().jvm_instances);
let before = debug_events(&h).len();
h.manager.set_jvm_inspector(id, Arc::new(FakeInspector::default())).unwrap();
assert!(h.manager.capabilities(id).unwrap().jvm_instances);
assert!(matches!(debug_events(&h).split_off(before).as_slice(), [DebugEvent::Capabilities]));
}
#[tokio::test]
async fn remote_attach_counts_classes_through_the_debugger() {
let h = harness(Script::default());
let project = tempfile::tempdir().unwrap();
let fake = Arc::new(FakeInspector::default());
fake.counts.lock().push(ide_jvm_inspect::ClassCount { name: "A".into(), instances: 3 });
let id = stopped_with(&h, remote_attach("java-debug", project.path()), &fake).await;
let result = h.manager.jvm_memory_classes(id).await.unwrap();
assert!(
matches!(&result, JvmMemoryResult::Ready { snapshot }
if snapshot.source == JvmMemorySource::Debugger && snapshot.rows.len() == 1),
"{result:?}"
);
assert_eq!(fake.targets.lock()[0].attach_port, Some(5005));
}
const HANDSHAKE: &[u8; 14] = b"JDWP-Handshake";
/// A Kotlin attach whose link reaches a fake JVM: (adapter end, JVM end) after both handshakes.
async fn linked_kotlin(h: &Harness) -> (DebugSessionId, tokio::net::TcpStream, tokio::net::TcpStream) {
let jvm = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let project = tempfile::tempdir().unwrap();
let id = h.start(kotlin_attach(project.path(), "127.0.0.1", jvm.local_addr().unwrap().port())).await.unwrap();
let link_port = h.request("attach")["arguments"]["port"].as_u64().unwrap() as u16;
let mut adapter = tokio::net::TcpStream::connect(("127.0.0.1", link_port)).await.unwrap();
adapter.write_all(HANDSHAKE).await.unwrap();
let (mut jvm, _) = tokio::time::timeout(Duration::from_secs(2), jvm.accept()).await.unwrap().unwrap();
let mut buf = [0; 14];
jvm.read_exact(&mut buf).await.unwrap();
jvm.write_all(HANDSHAKE).await.unwrap();
adapter.read_exact(&mut buf).await.unwrap();
(id, adapter, jvm)
}
/// A command the adapter sends with an id from the link's own range: the link then only forwards.
async fn degrade(adapter: &mut tokio::net::TcpStream) {
let mut packet = 11u32.to_be_bytes().to_vec();
packet.extend(ide_jdwp::OWN_IDS.start.to_be_bytes());
packet.extend([0, 1, 1]);
adapter.write_all(&packet).await.unwrap();
}
async fn wait_backend(h: &Harness, want: impl Fn(&JvmBackendState) -> bool) {
h.wait_event(|e| matches!(e, DebugEvent::JvmBackend { state } if want(state))).await;
}
#[tokio::test]
async fn kotlin_link_reports_degraded_then_closed() {
let h = harness(Script::default());
let (id, mut adapter, jvm) = linked_kotlin(&h).await;
degrade(&mut adapter).await;
wait_backend(&h, |s| matches!(s, JvmBackendState::Degraded { .. })).await;
drop(jvm);
wait_backend(&h, |s| *s == JvmBackendState::Closed).await;
assert_eq!(backend_events(&h).len(), 2, "{:?}", backend_events(&h));
let session = h.manager.session(id).unwrap();
assert!(session.jdwp_link.lock().is_some(), "a closed link is kept until the session ends");
}
#[tokio::test]
async fn kotlin_link_is_quiet_once_the_program_ended() {
let h = harness(Script::default());
let (id, _adapter, jvm) = linked_kotlin(&h).await;
let session = h.manager.session(id).unwrap();
// The adapter reported the end; the session keeps its link until it shuts down.
session.terminated.store(true, Ordering::Release);
drop(jvm);
for _ in 0..100 {
if session.jdwp_link.lock().as_ref().is_some_and(|l| l.state() == ide_jdwp::LinkState::Closed) {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(backend_events(&h).is_empty(), "{:?}", backend_events(&h));
}
+44 -17
Переглянути файл
@@ -136,24 +136,19 @@ pub struct DockerRegistry {
pub username: String,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct DockerSettings {
#[serde(default = "default_connections")]
#[serde(default)]
pub connections: Vec<DockerConnection>,
#[serde(default)]
pub registries: Vec<DockerRegistry>,
}
impl Default for DockerSettings {
fn default() -> Self {
Self { connections: default_connections(), registries: Vec::new() }
}
}
/// Before anything is configured the CLI's own daemon is shown, as before connections existed.
fn default_connections() -> Vec<DockerConnection> {
vec![DockerConnection { id: "default".into(), name: "Docker".into(), endpoint: DockerEndpoint::Default }]
/// Whether `c` is the connection older versions injected when nothing was configured:
/// Services lists only connections the user added, so this one is dropped on load.
fn is_injected_default(c: &DockerConnection) -> bool {
c.id == "default" && c.name == "Docker" && c.endpoint == DockerEndpoint::Default
}
const SETTINGS_FILE: &str = "docker.json";
@@ -161,10 +156,17 @@ const PASSWORDS_FILE: &str = "docker-passwords.json";
pub fn load_settings(data_dir: &Path) -> DockerSettings {
match std::fs::read(data_dir.join(SETTINGS_FILE)) {
Ok(bytes) => serde_json::from_slice(&bytes).unwrap_or_else(|e| {
tracing::warn!("{SETTINGS_FILE}: {e}");
DockerSettings::default()
}),
// A file written before the opt-in Services rework carries the injected default
// (saved along with a registry or another connection); drop it unless edited.
Ok(bytes) => serde_json::from_slice(&bytes)
.map(|mut s: DockerSettings| {
s.connections.retain(|c| !is_injected_default(c));
s
})
.unwrap_or_else(|e| {
tracing::warn!("{SETTINGS_FILE}: {e}");
DockerSettings::default()
}),
Err(_) => DockerSettings::default(),
}
}
@@ -373,7 +375,8 @@ mod tests {
fn settings_round_trip_and_passwords() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
assert_eq!(load_settings(dir).connections[0].endpoint, DockerEndpoint::Default);
// Nothing configured: no connection is invented.
assert!(load_settings(dir).connections.is_empty());
let settings = DockerSettings {
connections: vec![DockerConnection {
id: "c1".into(),
@@ -395,8 +398,32 @@ mod tests {
assert!(auth_for(dir, "postgres").is_none());
save_settings(dir, &DockerSettings { registries: Vec::new(), ..settings }).unwrap();
assert_eq!(registry_password(dir, "r1"), None);
// An empty list stays empty (the default connection is only for a missing file).
// An empty list stays empty.
save_settings(dir, &DockerSettings { connections: Vec::new(), registries: Vec::new() }).unwrap();
assert!(load_settings(dir).connections.is_empty());
}
#[test]
fn load_drops_the_injected_default_connection() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path();
let injected =
|| DockerConnection { id: "default".into(), name: "Docker".into(), endpoint: DockerEndpoint::Default };
let registry =
DockerRegistry { id: "r1".into(), name: "GHCR".into(), address: "ghcr.io".into(), username: "me".into() };
// Saved untouched by an older version (e.g. along with a registry): dropped on load.
save_settings(dir, &DockerSettings { connections: vec![injected()], registries: vec![registry.clone()] })
.unwrap();
let loaded = load_settings(dir);
assert!(loaded.connections.is_empty());
assert_eq!(loaded.registries, vec![registry]);
// A connection the user renamed or re-pointed stays theirs.
let renamed = DockerConnection { name: "Local engine".into(), ..injected() };
save_settings(dir, &DockerSettings { connections: vec![renamed.clone()], registries: Vec::new() }).unwrap();
assert_eq!(load_settings(dir).connections, vec![renamed]);
let repointed =
DockerConnection { endpoint: DockerEndpoint::Socket { path: "/var/run/docker.sock".into() }, ..injected() };
save_settings(dir, &DockerSettings { connections: vec![repointed.clone()], registries: Vec::new() }).unwrap();
assert_eq!(load_settings(dir).connections, vec![repointed]);
}
}
+17 -2
Переглянути файл
@@ -71,7 +71,6 @@ impl Capability {
match self {
Self::ProjectRead | Self::EditorRead | Self::Storage | Self::Events | Self::VcsRead => Risk::Low,
Self::EditorWrite
| Self::ProjectWrite
| Self::DatabaseSources
| Self::VcsWrite
| Self::Net(_)
@@ -80,7 +79,14 @@ impl Capability {
| Self::Secrets
| Self::InputEvents
| Self::Clipboard => Risk::Medium,
Self::IdeCommands | Self::Ipc(_) | Self::Process(_) | Self::FsRead(_) | Self::FsWrite(_) => Risk::High,
// A trusted project's own files name the programs the IDE starts (its settings, its
// build scripts), so writing them is running code.
Self::ProjectWrite
| Self::IdeCommands
| Self::Ipc(_)
| Self::Process(_)
| Self::FsRead(_)
| Self::FsWrite(_) => Risk::High,
}
}
@@ -402,6 +408,15 @@ mod tests {
assert!(Capability::Storage.auto_migrates() && !Capability::EditorWrite.auto_migrates());
}
/// A trusted project's files name programs the IDE starts by itself: the git and language
/// server executables in `.openidea/settings.toml`, its build scripts. Writing them is
/// running code, and the install dialog says so.
#[test]
fn writing_project_files_is_a_high_risk_grant() {
assert_eq!(Capability::ProjectWrite.risk(), Risk::High);
assert!(!Capability::ProjectWrite.auto_migrates());
}
#[test]
fn host_patterns() {
assert!(host_matches("crates.io", "crates.io"));
+110 -3
Переглянути файл
@@ -352,6 +352,22 @@ pub struct IndexEntry {
/// the IDE before showing it.
#[serde(default)]
pub icon: Option<PluginIcon>,
/// Where to read more: the manifest's homepage, else the registry's source repository.
#[serde(default)]
pub homepage: Option<String>,
/// This version's release notes (the manifest's `change-notes`).
#[serde(default)]
pub notes: Option<String>,
/// Downloads of the plugin's packages, as counted by whoever serves them.
#[serde(default)]
pub downloads: Option<u64>,
/// Picked by the marketplace's maintainers.
#[serde(default)]
pub featured: bool,
/// Project-relative paths whose presence makes the plugin worth suggesting
/// (`pom.xml`, `config/routes.rb`).
#[serde(default)]
pub suggest: Vec<String>,
}
impl IndexEntry {
@@ -389,10 +405,31 @@ impl Index {
/// revocations, publisher keys (revoked ones) and `moved` — and lists no
/// plugins ([`Index::is_newer`]): the IDE needs updating to install from it.
pub fn parse(text: &str) -> Result<Self, String> {
let (index, dropped) = Self::parse_dropping(text)?;
if let Some(first) = dropped.first() {
// One line per index, and a bounded one: whoever wrote the index chose the text.
let id: String = first.id.chars().take(80).collect();
tracing::warn!(
"index: {} entries are not offered, their id or version is not one a manifest may carry (first: {id:?})",
dropped.len()
);
}
Ok(index)
}
/// [`Index::parse`], with the entries it does not offer: those whose id or
/// version no manifest may carry, so no IDE could install them.
pub fn parse_dropping(text: &str) -> Result<(Self, Vec<IndexEntry>), String> {
let value: serde_json::Value = serde_json::from_str(text).map_err(|e| format!("index: {e}"))?;
let format = value.get("format").and_then(serde_json::Value::as_u64).ok_or("index: `format` is missing")?;
if format == u64::from(INDEX_FORMAT) {
return serde_json::from_value(value).map_err(|e| format!("index: {e}"));
let mut index: Index = serde_json::from_value(value).map_err(|e| format!("index: {e}"))?;
// What no manifest may carry is not offered: the id and version also name files on disk.
let (offered, dropped) = std::mem::take(&mut index.plugins)
.into_iter()
.partition(|e| crate::validate_plugin_id(&e.id).is_ok() && e.semver().is_some());
index.plugins = offered;
return Ok((index, dropped));
}
if format < u64::from(INDEX_FORMAT) {
return Err(format!("index format {format} is not supported (this IDE reads {INDEX_FORMAT})"));
@@ -401,13 +438,14 @@ impl Index {
let items = value.get(field).and_then(serde_json::Value::as_array);
items.into_iter().flatten().filter_map(|v| serde_json::from_value(v.clone()).ok()).collect()
}
Ok(Index {
let index = Index {
format: u32::try_from(format).unwrap_or(u32::MAX),
publishers: each(&value, "publishers"),
plugins: Vec::new(),
revoked: each(&value, "revoked"),
moved: value.get("moved").and_then(serde_json::Value::as_str).map(str::to_owned),
})
};
Ok((index, Vec::new()))
}
/// Written in a format newer than this IDE reads: it lists no plugins.
@@ -685,9 +723,33 @@ mod tests {
engine: "^1.0".into(),
categories: Vec::new(),
icon: None,
homepage: None,
notes: None,
downloads: None,
featured: false,
suggest: Vec::new(),
}
}
#[test]
fn listing_details_are_optional() {
let mut json = serde_json::to_value(entry("acme.a", "acme")).unwrap();
for key in ["homepage", "notes", "downloads", "featured", "suggest"] {
json.as_object_mut().unwrap().remove(key);
}
assert_eq!(serde_json::from_value::<IndexEntry>(json).unwrap(), entry("acme.a", "acme"));
let full = IndexEntry {
homepage: Some("https://example.com".into()),
notes: Some("Fixes".into()),
downloads: Some(42),
featured: true,
suggest: vec!["pom.xml".into()],
..entry("acme.a", "acme")
};
let back: IndexEntry = serde_json::from_str(&serde_json::to_string(&full).unwrap()).unwrap();
assert_eq!(back, full);
}
fn version(id: &str, version: &str, engine: &str) -> IndexEntry {
IndexEntry { version: version.into(), engine: engine.into(), ..entry(id, "acme") }
}
@@ -868,6 +930,51 @@ mod tests {
}
}
#[test]
fn an_index_offers_only_the_ids_and_versions_a_manifest_may_carry() {
let invalid_ids = [
"../x",
"../../escape",
"acme/../x",
r"acme.\..\x",
"acme./../../../escape",
"/abs",
r"C:\x",
"ACME.x",
"acme.",
"acme..x",
"acme.x y",
"",
];
let invalid_versions =
["../1.0.0", "1.0.0/../../../escape", r"1.0.0\..\x", "1.0.0-..", " 1.0.0", "1.0", "latest", ""];
let valid = [("acme.x", "1.0.0"), ("acme.tools-2_b", "1.0.0-rc.1+b.2")];
let mut listed: Vec<(&str, &str)> = valid.to_vec();
listed.extend(invalid_ids.iter().map(|id| (*id, "1.0.0")));
listed.extend(invalid_versions.iter().map(|version| ("acme.x", *version)));
let plugins: Vec<serde_json::Value> = listed
.iter()
.map(|(id, version)| {
serde_json::json!({
"id": id, "name": "X", "version": version, "publisher": "acme",
"download": "x.oiplugin", "digest": "d", "engine": "^1.0",
})
})
.collect();
let text = serde_json::json!({ "format": INDEX_FORMAT, "plugins": plugins }).to_string();
let pairs = |entries: &[IndexEntry]| -> Vec<(String, String)> {
entries.iter().map(|e| (e.id.clone(), e.version.clone())).collect()
};
let owned = |pairs: &[(&str, &str)]| -> Vec<(String, String)> {
pairs.iter().map(|(id, version)| ((*id).to_owned(), (*version).to_owned())).collect()
};
assert_eq!(pairs(&Index::parse(&text).unwrap().plugins), owned(&valid));
let (index, dropped) = Index::parse_dropping(&text).unwrap();
assert_eq!(pairs(&index.plugins), owned(&valid));
assert_eq!(pairs(&dropped), owned(&listed[valid.len()..]));
}
#[test]
fn a_newer_index_still_revokes_and_moves() {
let newer = format!(
+10 -2
Переглянути файл
@@ -102,6 +102,14 @@ fn parse<T>(r: serde_json::Result<T>) -> IdeResult<T> {
r.map_err(|e| IdeError::new(ide_core::error::ErrorKind::Protocol, format!("unexpected gh output: {e}")))
}
/// The repository of `root`, free to execute: the host runs the `gh_*` commands only in a
/// trusted project.
fn trusted_repo(root: &Path) -> IdeResult<Repo> {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
repo.set_execution_allowed(true);
Ok(repo)
}
/// Whether gh is installed and signed in, and which GitHub repository `root` maps to.
pub fn availability(root: &Path) -> GhAvailability {
if gh_executable().is_none() {
@@ -233,7 +241,7 @@ pub fn create(
return Err(IdeError::invalid("choose a base branch"));
}
if req.push {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
let repo = trusted_repo(root)?;
repo.push(false, true, cancel, on_output)?;
}
let mut args = vec!["pr", "create", "--title", req.title.trim(), "--body", &req.body, "--base", req.base.trim()];
@@ -354,7 +362,7 @@ pub fn prepare_diff(
cancel: &CancellationToken,
on_output: &dyn Fn(String),
) -> IdeResult<PrDiffRevisions> {
let repo = Repo::discover(root)?.ok_or_else(|| IdeError::invalid("not a git repository"))?;
let repo = trusted_repo(root)?;
if !repo.has_commit(head_oid) || !repo.has_commit(base_oid) {
let remotes = repo.remotes()?;
let remote = remote_for(&remotes, name_with_owner)
+28
Переглянути файл
@@ -0,0 +1,28 @@
[package]
name = "ide-heap"
description = "HotSpot .hprof parser and heap-snapshot analysis for the debugger's JVM Memory view"
version.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
ide-core.workspace = true
memmap2.workspace = true
serde.workspace = true
serde_json.workspace = true
specta.workspace = true
thiserror.workspace = true
tracing.workspace = true
uuid.workspace = true
[target.'cfg(unix)'.dependencies]
libc = "0.2"
[dev-dependencies]
criterion = { version = "0.5", default-features = false }
proptest.workspace = true
tempfile.workspace = true
[[bench]]
name = "heap"
harness = false
+100
Переглянути файл
@@ -0,0 +1,100 @@
//! `cargo bench -p ide-heap`: whole-dump analysis and the dominator tree at a million objects.
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use ide_core::CancellationToken;
use ide_heap::testing::{MiniHeap, XorShift};
use ide_heap::{AnalyzeOptions, BasicType, HeapError, HeapPhase, HeapRootKind, Store, analyze, dominators};
const CHAINS: u64 = 100_000;
const CHAIN_LEN: u64 = 5;
const NODES: u64 = CHAINS * CHAIN_LEN;
/// Every object takes one slot: node `k` at slot `2k`, its payload at `2k + 1`.
const BASE: u64 = 0x1_0000_0000;
const SLOT: u64 = 0x20;
fn node_addr(k: u64) -> u64 {
BASE + 2 * k * SLOT
}
fn payload_addr(k: u64) -> u64 {
node_addr(k) + SLOT
}
/// 100k rooted chains of five `Node { next, payload }`, each node with its own `byte[16]`. Every
/// seventh node points at the payload of the same position in the next chain instead, so that
/// payload has two owners and its own one is garbage: 1M objects in all.
fn million_objects() -> MiniHeap {
let mut heap = MiniHeap::new();
heap.class("Node", Some("java.lang.Object"), &[("next", BasicType::Object), ("payload", BasicType::Object)]);
for k in 0..NODES {
let next = if (k + 1) % CHAIN_LEN == 0 { 0 } else { node_addr(k + 1) };
let payload = if k % 7 == 0 { payload_addr((k + CHAIN_LEN) % NODES) } else { payload_addr(k) };
heap.object(node_addr(k), "Node", &[("next", next), ("payload", payload)]);
heap.prim(payload_addr(k), BasicType::Byte, 16);
if k % CHAIN_LEN == 0 {
heap.root(HeapRootKind::JniGlobal, node_addr(k));
}
}
heap
}
/// CSR of a random tree on `nodes` nodes rooted at 0 plus `extra` random edges, targets of each
/// node in scrambled order.
fn random_graph(nodes: u32, extra: u32, seed: u64) -> (Vec<u32>, Vec<u32>) {
let mut rng = XorShift::new(seed);
let mut edges: Vec<(u32, u32)> = (1..nodes).map(|v| (rng.below(v), v)).collect();
edges.extend((0..extra).map(|_| (rng.below(nodes), rng.below(nodes))));
for i in (1..edges.len()).rev() {
edges.swap(i, rng.below(i as u32 + 1) as usize);
}
let mut start = vec![0u32; nodes as usize + 1];
for &(from, _) in &edges {
start[from as usize + 1] += 1;
}
for v in 0..nodes as usize {
start[v + 1] += start[v];
}
let mut cursor = start.clone();
let mut out = vec![0u32; edges.len()];
for &(from, to) in &edges {
out[cursor[from as usize] as usize] = to;
cursor[from as usize] += 1;
}
(start, out)
}
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn bench(c: &mut Criterion) {
let dir = tempfile::tempdir().expect("temp dir");
let hprof = dir.path().join("dump.hprof");
million_objects().write_to(&hprof);
let (start, out) = random_graph(1_000_000, 500_000, 0x5EED_D0D0);
let cancel = CancellationToken::new();
let opts = AnalyzeOptions { in_memory: true, ..AnalyzeOptions::default() };
let summary = analyze(&hprof, dir.path(), opts, &no_progress, &cancel).expect("analyze").summary;
assert_eq!(summary.objects, 2 * NODES, "{summary:?}");
assert_eq!(summary.unreachable_objects, NODES.div_ceil(7), "{summary:?}");
let mut g = c.benchmark_group("heap");
g.sample_size(10);
g.bench_function("index_1m_objects", |b| {
b.iter(|| {
let opts = AnalyzeOptions { in_memory: true, ..AnalyzeOptions::default() };
let index = analyze(&hprof, dir.path(), opts, &no_progress, &cancel).expect("analyze");
black_box(index.summary.objects)
})
});
g.bench_function("dominators_1m_random", |b| {
b.iter(|| {
let d = dominators(&start, &out, 0, &Store::Ram, &no_progress, &cancel).expect("dominators");
black_box(d.reachable)
})
});
g.finish();
}
criterion_group!(benches, bench);
criterion_main!(benches);
+202
Переглянути файл
@@ -0,0 +1,202 @@
//! Ties the passes together: graph, dominators, shallow and retained sizes, the class table and
//! paths to GC roots, into a [`HeapIndex`].
mod classes;
mod paths;
mod retained;
use std::collections::HashMap;
use std::fs::File;
use std::path::Path;
use std::time::Instant;
use ide_core::CancellationToken;
use memmap2::Mmap;
use crate::column::Store;
use crate::dominators::{Dominators, dominators};
use crate::error::HeapError;
use crate::files::INDEX_FILES;
use crate::graph::{HeapGraph, build_graph};
use crate::index::{HeapIndex, HeapSummary, write_summary};
use crate::progress::{CHECK_EVERY, HeapPhase, Progress};
use crate::shallow::{HistogramEntry, ShallowModel, VmLayout};
#[derive(Clone, Debug, Default)]
pub struct AnalyzeOptions {
/// The JVM's class histogram from the same stop as the dump.
pub histogram: Option<Vec<HistogramEntry>>,
/// `VM.flags` output of the JVM that wrote the dump.
pub vm_flags: Option<String>,
pub java_major: Option<u32>,
/// Keep every array in RAM and write nothing to `index_dir`.
pub in_memory: bool,
}
/// Analyzes the dump at `hprof`, writing the index columns and `summary.json` into `index_dir`
/// (which must exist and may hold the dump itself). `progress` gets every [`HeapPhase`] in order;
/// an `Err` from it, a cancel or any other error removes every index file from `index_dir` and
/// leaves the rest of the directory alone.
pub fn analyze(
hprof: &Path,
index_dir: &Path,
opts: AnalyzeOptions,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<HeapIndex, HeapError> {
let store = if opts.in_memory { Store::Ram } else { Store::Dir(index_dir.to_path_buf()) };
let result = run(hprof, &store, opts, progress, cancel);
// `run` has dropped every column by now: Windows cannot delete a mapped file.
if result.is_err() && !matches!(store, Store::Ram) {
remove_index_files(index_dir);
}
result
}
fn run(
hprof: &Path,
store: &Store,
opts: AnalyzeOptions,
report: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<HeapIndex, HeapError> {
let started = Instant::now();
let progress = Progress::new(report, cancel);
let file = File::open(hprof)?;
// SAFETY: nothing writes the dump while it is analyzed or viewed: a capture finishes writing
// it before analysis and deletes it only with the snapshot that holds this index.
let dump = unsafe { Mmap::map(&file)? };
let graph = build_graph(&dump, store, report, cancel)?;
let Dominators { idom, preorder, .. } =
dominators(&graph.out_start, &graph.out, graph.root, store, report, cancel)?;
let nodes = u64::from(graph.root) + 1;
// Seven walks over the nodes, roughly: sizes, retained, child counts, child sorting, class
// counts and the tree walk, which visits every node twice.
let mut meter = Meter::start(&progress, HeapPhase::Retained, 7 * nodes)?;
let layout = VmLayout::from_vm_flags(opts.vm_flags.as_deref().unwrap_or(""), opts.java_major);
let model = ShallowModel::new(&graph, layout, opts.histogram.as_deref(), cancel)?;
let (shallow8, retained) = retained::sizes(&graph, &model, &idom, &preorder, store, &mut meter)?;
drop(preorder);
let (kids_start, kids) = retained::children(&idom, &retained, graph.root, store, &mut meter)?;
let tree = retained::Tree { idom, shallow8, retained, kids_start, kids };
let (classes, totals) = classes::table(&graph, &tree, store, &mut meter)?;
meter.finish()?;
let mut meter = Meter::start(&progress, HeapPhase::Paths, nodes + graph.out.len() as u64)?;
let gc_parent = paths::gc_parents(&graph, store, &mut meter)?;
meter.finish()?;
progress.report(HeapPhase::Saving, 0, 1)?;
let HeapGraph {
ids,
types,
offsets,
roots,
classes: class_info,
class_slot,
thread_objects,
weak_edges_skipped,
id_size,
..
} = graph;
let summary = HeapSummary {
objects: totals.reachable_objects + totals.unreachable_objects,
reachable_objects: totals.reachable_objects,
reachable_bytes: totals.reachable_bytes,
unreachable_objects: totals.unreachable_objects,
unreachable_bytes: totals.unreachable_bytes,
classes: class_info.iter().filter(|c| c.id != 0).count() as u32,
weak_edges_skipped,
size_source: model.source,
analysis_ms: u32::try_from(started.elapsed().as_millis()).unwrap_or(u32::MAX),
};
if let Store::Dir(dir) = store {
write_summary(dir, &summary)?;
}
let mut root_kinds = HashMap::with_capacity(roots.len());
for (object, kind, serial) in roots {
root_kinds.entry(object).or_insert((kind, serial));
}
progress.report(HeapPhase::Saving, 1, 1)?;
let retained::Tree { idom, shallow8, retained, kids_start, kids } = tree;
Ok(HeapIndex {
hprof: dump,
ids,
types,
offsets,
shallow8,
retained,
idom,
kids_start,
kids,
gc_parent,
classes,
class_info,
class_slot,
roots: root_kinds,
thread_objects,
id_size,
summary,
})
}
/// Removes what an analysis may have written; a file that is not there is fine.
fn remove_index_files(dir: &Path) {
for name in INDEX_FILES {
let path = dir.join(name);
if let Err(err) = std::fs::remove_file(&path)
&& err.kind() != std::io::ErrorKind::NotFound
{
tracing::warn!("could not remove {}: {err}", path.display());
}
}
}
/// Steps of one phase: reports progress every [`CHECK_EVERY`] steps, and each loop looks at the
/// token before its first step.
pub(super) struct Meter<'a> {
progress: &'a Progress<'a>,
phase: HeapPhase,
/// An estimate; `done` is clamped to it.
total: u64,
done: u64,
next_report: u64,
}
impl<'a> Meter<'a> {
fn start(progress: &'a Progress<'a>, phase: HeapPhase, total: u64) -> Result<Self, HeapError> {
progress.report(phase, 0, total)?;
Ok(Self { progress, phase, total, done: 0, next_report: CHECK_EVERY })
}
pub(super) fn begin_loop(&self) -> Result<(), HeapError> {
self.progress.check()
}
pub(super) fn token(&self) -> &'a CancellationToken {
self.progress.token()
}
#[inline]
pub(super) fn tick(&mut self) -> Result<(), HeapError> {
self.advance(1)
}
pub(super) fn advance(&mut self, steps: u64) -> Result<(), HeapError> {
self.done += steps;
if self.done < self.next_report {
return Ok(());
}
self.next_report = self.done + CHECK_EVERY;
self.progress.report(self.phase, self.done.min(self.total), self.total)
}
fn finish(self) -> Result<(), HeapError> {
self.progress.report(self.phase, self.total, self.total)
}
}
#[cfg(test)]
mod tests;
+117
Переглянути файл
@@ -0,0 +1,117 @@
//! The class table: counts and shallow sizes per type slot, retained sizes per type from a walk of
//! the dominator tree, and the reachable / unreachable totals of the summary.
use super::Meter;
use super::retained::Tree;
use crate::column::{NONE, Store};
use crate::error::HeapError;
use crate::graph::{HeapGraph, PRIM_SLOTS};
use crate::hprof::BasicType;
use crate::index::ClassStat;
use crate::names::display_name;
/// Instances and arrays only: class objects are left out.
#[derive(Default)]
pub(super) struct Totals {
pub(super) reachable_objects: u64,
pub(super) reachable_bytes: u64,
pub(super) unreachable_objects: u64,
pub(super) unreachable_bytes: u64,
}
pub(super) fn table(
graph: &HeapGraph,
tree: &Tree,
store: &Store,
meter: &mut Meter,
) -> Result<(Vec<ClassStat>, Totals), HeapError> {
let slots = graph.classes.len() + PRIM_SLOTS as usize;
let mut classes: Vec<ClassStat> = (0..slots as u32)
.map(|slot| ClassStat { name: slot_name(graph, slot), count: 0, shallow: 0, retained: 0, unreachable_count: 0 })
.collect();
let mut totals = Totals::default();
meter.begin_loop()?;
for v in 0..graph.root as usize {
meter.tick()?;
let slot = graph.types[v];
let bytes = u64::from(tree.shallow8[v]) * 8;
let stat = &mut classes[slot as usize];
let instance = slot != graph.class_slot;
if tree.idom[v] == NONE {
stat.unreachable_count += 1;
if instance {
totals.unreachable_objects += 1;
totals.unreachable_bytes += bytes;
}
} else {
stat.count += 1;
stat.shallow += bytes;
if instance {
totals.reachable_objects += 1;
totals.reachable_bytes += bytes;
}
}
}
retained_by_type(graph, tree, &mut classes, store, meter)?;
Ok((classes, totals))
}
/// Adds to each type the retained size of its instances that have no instance of the same type
/// above them in the dominator tree. The walk keeps no stack: it goes up through `idom` and
/// resumes a node's children from `cursor`, while `active[t]` counts the instances of type `t` on
/// the path from the root.
fn retained_by_type(
graph: &HeapGraph,
tree: &Tree,
classes: &mut [ClassStat],
store: &Store,
meter: &mut Meter,
) -> Result<(), HeapError> {
let root = graph.root;
let mut cursor = store.scratch::<u32>(root as usize + 1, 0)?;
let mut active = vec![0u32; classes.len()];
cursor[root as usize] = tree.kids_start[root as usize];
let mut v = root;
meter.begin_loop()?;
loop {
meter.tick()?;
let next = cursor[v as usize];
if next < tree.kids_start[v as usize + 1] {
cursor[v as usize] = next + 1;
let child = tree.kids[next as usize];
let slot = graph.types[child as usize] as usize;
if active[slot] == 0 {
classes[slot].retained += tree.retained[child as usize];
}
active[slot] += 1;
cursor[child as usize] = tree.kids_start[child as usize];
v = child;
} else if v == root {
return Ok(());
} else {
active[graph.types[v as usize] as usize] -= 1;
v = tree.idom[v as usize];
}
}
}
fn slot_name(graph: &HeapGraph, slot: u32) -> String {
match graph.classes.get(slot as usize) {
Some(class) => display_name(&class.name),
None => graph.prim_type(slot).map_or_else(String::new, |ty| prim_array_name(ty).to_owned()),
}
}
fn prim_array_name(ty: BasicType) -> &'static str {
match ty {
BasicType::Object => "java.lang.Object[]",
BasicType::Boolean => "boolean[]",
BasicType::Char => "char[]",
BasicType::Float => "float[]",
BasicType::Double => "double[]",
BasicType::Byte => "byte[]",
BasicType::Short => "short[]",
BasicType::Int => "int[]",
BasicType::Long => "long[]",
}
}
+32
Переглянути файл
@@ -0,0 +1,32 @@
//! Parents on shortest paths from the GC roots, by breadth-first search over strong edges.
use super::Meter;
use crate::column::{Column, NONE, Store};
use crate::error::HeapError;
use crate::files::GC_PARENT_FILE;
use crate::graph::HeapGraph;
pub(super) fn gc_parents(graph: &HeapGraph, store: &Store, meter: &mut Meter) -> Result<Column<u32>, HeapError> {
let root = graph.root;
let nodes = root as usize + 1;
let mut parent = store.column::<u32>(GC_PARENT_FILE, nodes, NONE)?;
let mut queue = store.scratch::<u32>(nodes, 0)?;
parent[root as usize] = root;
queue[0] = root;
let (mut head, mut tail) = (0, 1);
meter.begin_loop()?;
while head < tail {
meter.tick()?;
let v = queue[head];
head += 1;
for &w in graph.out_edges(v) {
meter.tick()?;
if parent[w as usize] == NONE {
parent[w as usize] = v;
queue[tail] = w;
tail += 1;
}
}
}
Ok(parent)
}
+209
Переглянути файл
@@ -0,0 +1,209 @@
//! Shallow and retained sizes, and the dominator tree as child lists sorted by retained size.
use std::cmp::Reverse;
use super::Meter;
use crate::column::{Column, NONE, Store};
use crate::error::HeapError;
use crate::files::{KIDS_FILE, KIDS_START_FILE, RETAINED_FILE, SHALLOW8_FILE};
use crate::graph::{HeapGraph, accumulate};
use crate::progress::CHECK_EVERY;
use crate::shallow::ShallowModel;
/// The dominator tree with sizes; see [`crate::HeapIndex`] for each column.
pub(super) struct Tree {
pub(super) idom: Column<u32>,
pub(super) shallow8: Column<u32>,
pub(super) retained: Column<u64>,
pub(super) kids_start: Column<u32>,
pub(super) kids: Column<u32>,
}
/// `(shallow8, retained)` of every node, the root included. `preorder` is the DFS preorder of the
/// reachable nodes, from the root.
pub(super) fn sizes(
graph: &HeapGraph,
model: &ShallowModel,
idom: &[u32],
preorder: &[u32],
store: &Store,
meter: &mut Meter,
) -> Result<(Column<u32>, Column<u64>), HeapError> {
let nodes = idom.len();
let mut shallow8 = store.column::<u32>(SHALLOW8_FILE, nodes, 0)?;
let mut retained = store.column::<u64>(RETAINED_FILE, nodes, 0)?;
meter.begin_loop()?;
for v in 0..graph.root {
meter.tick()?;
let units = u32::try_from(model.size_of(graph, v).div_ceil(8)).unwrap_or(u32::MAX);
shallow8[v as usize] = units;
if idom[v as usize] != NONE {
retained[v as usize] = u64::from(units) * 8;
}
}
// A dominator is a DFS ancestor, so in reverse preorder every node is complete before it is
// added to its dominator.
meter.begin_loop()?;
for &v in preorder.iter().skip(1).rev() {
meter.tick()?;
let own = retained[v as usize];
retained[idom[v as usize] as usize] += own;
}
Ok((shallow8, retained))
}
/// `(kids_start, kids)`: the children of every reachable node in the dominator tree, as CSR.
pub(super) fn children(
idom: &[u32],
retained: &[u64],
root: u32,
store: &Store,
meter: &mut Meter,
) -> Result<(Column<u32>, Column<u32>), HeapError> {
let nodes = idom.len();
// Counted into `starts[d + 1]`; filled through `starts[d]` as a cursor, which leaves it at d's
// end until the column is shifted one slot.
let mut starts = store.column::<u32>(KIDS_START_FILE, nodes + 1, 0)?;
let mut longest = 0;
meter.begin_loop()?;
for (v, &d) in idom.iter().enumerate() {
meter.tick()?;
if d != NONE && v != root as usize {
let count = &mut starts[d as usize + 1];
*count += 1;
longest = longest.max(*count);
}
}
let total = accumulate(&mut starts, meter.token())?;
let mut kids = store.column::<u32>(KIDS_FILE, total as usize, 0)?;
meter.begin_loop()?;
for (v, &d) in idom.iter().enumerate() {
meter.tick()?;
if d != NONE && v != root as usize {
let cursor = &mut starts[d as usize];
kids[*cursor as usize] = v as u32;
*cursor += 1;
}
}
starts.copy_within(0..nodes, 1);
starts[0] = 0;
sort_children(&mut kids, &starts, retained, longest as usize, CHECK_EVERY as usize, store, meter)?;
Ok((starts, kids))
}
/// Sorts each child list by retained size descending, then by node. A list longer than `run` is
/// sorted in runs of `run` that are then merged, so no single step takes longer than one run.
fn sort_children(
kids: &mut [u32],
starts: &[u32],
retained: &[u64],
longest: usize,
run: usize,
store: &Store,
meter: &mut Meter,
) -> Result<(), HeapError> {
let key = |&k: &u32| (Reverse(retained[k as usize]), k);
let mut buffer = if longest > run { store.scratch::<u32>(longest, 0)? } else { Column::ram(0, 0) };
meter.begin_loop()?;
for v in 0..starts.len() - 1 {
meter.tick()?;
let list = &mut kids[starts[v] as usize..starts[v + 1] as usize];
if list.len() <= run {
list.sort_unstable_by_key(key);
meter.advance(list.len() as u64)?;
continue;
}
for chunk in list.chunks_mut(run) {
meter.begin_loop()?;
chunk.sort_unstable_by_key(key);
meter.advance(chunk.len() as u64)?;
}
let len = list.len();
let (mut from, mut to) = (list, &mut buffer[..len]);
let mut in_buffer = false;
let mut width = run;
while width < len {
meter.begin_loop()?;
for start in (0..len).step_by(2 * width) {
let mid = (start + width).min(len);
let end = (start + 2 * width).min(len);
merge(&from[start..mid], &from[mid..end], &mut to[start..end], &key, meter)?;
}
std::mem::swap(&mut from, &mut to);
in_buffer = !in_buffer;
width *= 2;
}
if in_buffer {
to.copy_from_slice(from);
}
}
Ok(())
}
fn merge<K: Ord>(
left: &[u32],
right: &[u32],
out: &mut [u32],
key: &impl Fn(&u32) -> K,
meter: &mut Meter,
) -> Result<(), HeapError> {
let (mut i, mut j) = (0, 0);
for slot in out.iter_mut() {
meter.tick()?;
let take_left = j == right.len() || (i < left.len() && key(&left[i]) <= key(&right[j]));
if take_left {
*slot = left[i];
i += 1;
} else {
*slot = right[j];
j += 1;
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use ide_core::CancellationToken;
use super::*;
use crate::error::HeapError;
use crate::progress::{HeapPhase, Progress};
use crate::testing::XorShift;
fn sorted_with_run(lists: &[Vec<u32>], retained: &[u64], run: usize) -> Vec<u32> {
let report = |_, _, _| Ok::<(), HeapError>(());
let cancel = CancellationToken::new();
let progress = Progress::new(&report, &cancel);
let mut meter = Meter::start(&progress, HeapPhase::Retained, 1).unwrap();
let mut starts = vec![0u32];
for list in lists {
starts.push(starts.last().unwrap() + list.len() as u32);
}
let mut kids: Vec<u32> = lists.concat();
let longest = lists.iter().map(Vec::len).max().unwrap_or(0);
sort_children(&mut kids, &starts, retained, longest, run, &Store::Ram, &mut meter).unwrap();
kids
}
#[test]
fn long_child_lists_are_merged_from_sorted_runs() {
let mut rng = XorShift::new(7);
let retained: Vec<u64> = (0..500).map(|_| u64::from(rng.below(40))).collect();
let mut nodes: Vec<u32> = (0..500).collect();
for i in (1..nodes.len()).rev() {
nodes.swap(i, rng.below(i as u32 + 1) as usize);
}
// Lists below, at and above the run length, and one that ends on a partial run.
let lists = vec![nodes[..3].to_vec(), nodes[3..7].to_vec(), nodes[7..400].to_vec(), nodes[400..].to_vec()];
for run in [1, 4, 7, 64, 1000] {
let mut expected = Vec::new();
for list in &lists {
let mut list = list.clone();
list.sort_by_key(|&k| (Reverse(retained[k as usize]), k));
expected.extend(list);
}
assert_eq!(sorted_with_run(&lists, &retained, run), expected, "run {run}");
}
}
}
+265
Переглянути файл
@@ -0,0 +1,265 @@
use std::cell::Cell;
use std::collections::BTreeSet;
use std::path::PathBuf;
use super::*;
use crate::column::{Column, NONE};
use crate::files::INDEX_FILES;
use crate::hprof::{BasicType, HeapRootKind};
use crate::index::{ClassStat, HeapSummary};
use crate::shallow::SizeSource;
use crate::testing::{MiniHeap, sample};
const DUMP: &str = "dump.hprof";
const ROOT: Option<&str> = Some("java.lang.Object");
const NODE_FIELDS: [(&str, BasicType); 2] = [("next", BasicType::Object), ("v", BasicType::Long)];
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn in_memory() -> AnalyzeOptions {
AnalyzeOptions { in_memory: true, ..AnalyzeOptions::default() }
}
/// A temp dir holding `heap` as `dump.hprof`, the way a capture lays out its snapshot dir.
fn dump_dir(heap: &MiniHeap) -> (tempfile::TempDir, PathBuf) {
let dir = tempfile::tempdir().unwrap();
let dump = dir.path().join(DUMP);
heap.write_to(&dump);
(dir, dump)
}
fn analyze_with(heap: &MiniHeap, opts: AnalyzeOptions) -> HeapIndex {
let (dir, dump) = dump_dir(heap);
analyze(&dump, dir.path(), opts, &no_progress, &CancellationToken::new()).unwrap()
}
fn file_names(dir: &Path) -> BTreeSet<String> {
std::fs::read_dir(dir).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect()
}
fn node(index: &HeapIndex, addr: u64) -> u32 {
index.ids.rank(addr).unwrap_or_else(|| panic!("{addr:#x} is not in the dump"))
}
fn shallow(index: &HeapIndex, addr: u64) -> u64 {
u64::from(index.shallow8[node(index, addr) as usize]) * 8
}
fn retained(index: &HeapIndex, addr: u64) -> u64 {
index.retained[node(index, addr) as usize]
}
fn idom(index: &HeapIndex, addr: u64) -> u32 {
index.idom[node(index, addr) as usize]
}
fn class<'a>(index: &'a HeapIndex, name: &str) -> &'a ClassStat {
let mut rows = index.classes.iter().filter(|c| c.name == name);
let row = rows.next().unwrap_or_else(|| panic!("no class {name}"));
assert!(rows.next().is_none(), "two classes named {name}");
row
}
fn stat(c: &ClassStat) -> (u64, u64, u64, u64) {
(c.count, c.shallow, c.retained, c.unreachable_count)
}
fn kids_of(index: &HeapIndex, v: u32) -> &[u32] {
&index.kids[index.kids_start[v as usize] as usize..index.kids_start[v as usize + 1] as usize]
}
const H: u64 = 0x1000;
const N1: u64 = 0x1100;
const N2: u64 = 0x1200;
const N3: u64 = 0x1300;
const L: u64 = 0x1400;
const R: u64 = 0x1500;
const W: u64 = 0x1600;
const N4: u64 = 0x1700;
#[test]
fn sample_retained_sizes() {
let index = analyze_with(&sample(), in_memory());
let shallows: Vec<u64> = [H, N1, L, W, R].iter().map(|&a| shallow(&index, a)).collect();
assert_eq!(shallows, [24, 24, 32, 24, 32]);
let retained_sizes: Vec<u64> = [L, N3, N1, N2, H, W].iter().map(|&a| retained(&index, a)).collect();
assert_eq!(retained_sizes, [32, 56, 24, 24, 72, 24]);
let root = index.root();
assert_eq!(
[idom(&index, N1), idom(&index, N2), idom(&index, N3), idom(&index, L)],
[node(&index, H), node(&index, H), root, node(&index, N3)]
);
assert_eq!((idom(&index, R), idom(&index, N4)), (NONE, NONE));
assert_eq!((retained(&index, R), retained(&index, N4)), (0, 0));
// Children by retained size descending, ties by node.
assert_eq!(kids_of(&index, node(&index, H)), [node(&index, N1), node(&index, N2)]);
let top = kids_of(&index, root);
assert_eq!(top[..2], [node(&index, H), node(&index, N3)]);
assert!(top.windows(2).all(|p| (index.retained[p[0] as usize], p[1]) > (index.retained[p[1] as usize], p[0])));
assert_eq!(index.retained[root as usize], top.iter().map(|&k| index.retained[k as usize]).sum::<u64>());
}
#[test]
fn sample_class_table() {
let index = analyze_with(&sample(), in_memory());
assert_eq!(stat(class(&index, "Node")), (3, 72, 104, 1));
assert_eq!(stat(class(&index, "Holder")), (1, 24, 72, 0));
assert_eq!(stat(class(&index, "long[]")), (1, 32, 32, 0));
assert_eq!(stat(class(&index, "java.lang.ref.WeakReference")), (1, 24, 24, 0));
let objects = class(&index, "java.lang.Object[]");
assert_eq!((objects.count, objects.unreachable_count), (0, 1));
// Indexed by type slot, primitive array slots included.
assert_eq!(index.classes.len(), index.class_info.len() + crate::graph::PRIM_SLOTS as usize);
let slot = index.types[node(&index, H) as usize] as usize;
assert_eq!(index.classes[slot].name, "Holder");
assert_eq!(index.class_info[slot].name, "Holder");
}
#[test]
fn sample_summary() {
let index = analyze_with(&sample(), in_memory());
let s = &index.summary;
assert_eq!(
(s.reachable_objects, s.reachable_bytes, s.unreachable_objects, s.unreachable_bytes, s.weak_edges_skipped),
(6, 152, 2, 56, 1)
);
assert_eq!(s.objects, 8);
assert_eq!(s.classes, 7);
assert_eq!(s.size_source, SizeSource::Layout);
}
#[test]
fn sample_gc_parents() {
let index = analyze_with(&sample(), in_memory());
let parent = |addr| index.gc_parent[node(&index, addr) as usize];
let root = index.root();
assert_eq!(index.gc_parent[root as usize], root);
assert_eq!([parent(H), parent(N3), parent(W)], [root; 3]);
assert_eq!([parent(N1), parent(L)], [node(&index, H), node(&index, N3)]);
assert_eq!((parent(R), parent(N4)), (NONE, NONE));
assert_eq!(index.roots[&node(&index, N3)], (HeapRootKind::JavaFrame, Some(1)));
}
#[test]
fn cycle_is_retained_once() {
let mut heap = MiniHeap::new();
heap.class("Node", ROOT, &NODE_FIELDS)
.object(0xA000, "Node", &[("next", 0xB000)])
.object(0xB000, "Node", &[("next", 0xA000)])
.root(HeapRootKind::JniGlobal, 0xA000);
let index = analyze_with(&heap, in_memory());
assert_eq!((retained(&index, 0xA000), retained(&index, 0xB000)), (48, 24));
assert_eq!(class(&index, "Node").retained, 48);
}
#[test]
fn deep_chain_retained_by_class() {
// A small stack proves no pass recurses along the chain.
std::thread::Builder::new()
.stack_size(256 << 10)
.spawn(|| {
const NODES: u64 = 1_000_000;
let addr = |i: u64| 0x10_0000 + i * 0x100;
let mut heap = MiniHeap::new();
heap.class("Node", ROOT, &NODE_FIELDS);
for i in 0..NODES {
let next = if i + 1 < NODES { addr(i + 1) } else { 0 };
heap.object(addr(i), "Node", &[("next", next)]);
}
heap.root(HeapRootKind::JniGlobal, addr(0));
let index = analyze_with(&heap, in_memory());
assert_eq!(retained(&index, addr(0)), 24_000_000);
assert_eq!(retained(&index, addr(NODES - 1)), 24);
assert_eq!(class(&index, "Node").retained, 24_000_000);
assert_eq!(index.summary.reachable_objects, NODES);
})
.unwrap()
.join()
.unwrap();
}
#[test]
fn histogram_sizes_flow_into_retained() {
let histogram = vec![HistogramEntry { name: "Node".to_owned(), instances: 3, bytes: 96 }];
let index = analyze_with(&sample(), AnalyzeOptions { histogram: Some(histogram), ..in_memory() });
assert_eq!(shallow(&index, N3), 32);
assert_eq!(retained(&index, N3), 64);
assert_eq!(index.summary.size_source, SizeSource::Jvm);
}
#[test]
fn vm_flags_of_an_unknown_jdk_set_the_layout() {
// Without compressed oops a Node is 12 + 8 + 8 = 28 → 32.
let opts = AnalyzeOptions { vm_flags: Some("-XX:MaxHeapSize=42949672960".to_owned()), ..in_memory() };
let index = analyze_with(&sample(), opts);
assert_eq!(shallow(&index, N4), 32);
}
#[test]
fn cancel_stops_analysis() {
let (dir, dump) = dump_dir(&sample());
let cancel = CancellationToken::new();
let phases = Cell::new(Vec::new());
let progress = |phase, _, _| {
let mut seen = phases.take();
seen.push(phase);
phases.set(seen);
if phase == HeapPhase::Dominators {
cancel.cancel();
}
Ok(())
};
let result = analyze(&dump, dir.path(), AnalyzeOptions::default(), &progress, &cancel);
assert!(matches!(result, Err(HeapError::Cancelled)), "{:?}", result.err());
assert_eq!(phases.take().last(), Some(&HeapPhase::Dominators), "stops at the first check");
// The dump the capture analyzes in place stays; every file the analysis wrote is gone.
assert_eq!(file_names(dir.path()), BTreeSet::from([DUMP.to_owned()]));
}
#[test]
fn progress_error_stops_analysis_and_removes_its_files() {
let (dir, dump) = dump_dir(&sample());
// The last report, once every index file is written.
let progress = |phase, done, _| match (phase, done) {
(HeapPhase::Saving, 1) => Err(HeapError::NotEnoughDiskSpace { needed: 2, available: 1 }),
_ => Ok(()),
};
let result = analyze(&dump, dir.path(), AnalyzeOptions::default(), &progress, &CancellationToken::new());
assert!(matches!(result, Err(HeapError::NotEnoughDiskSpace { .. })), "{:?}", result.err());
assert_eq!(file_names(dir.path()), BTreeSet::from([DUMP.to_owned()]));
}
#[test]
fn writes_index_files() {
let (dir, dump) = dump_dir(&sample());
let index = analyze(&dump, dir.path(), AnalyzeOptions::default(), &no_progress, &CancellationToken::new()).unwrap();
let names = file_names(dir.path());
for name in [
"types.u32",
"shallow8.u32",
"retained.u64",
"idom.u32",
"kids_start.u32",
"kids.u32",
"gc_parent.u32",
"offsets.u64",
"summary.json",
] {
assert!(names.contains(name), "{name} missing from {names:?}");
}
assert!(names.iter().any(|n| n.starts_with("ids.")), "{names:?}");
// Nothing but the index files and the dump: scratch arrays are gone.
let unexpected: Vec<_> = names.iter().filter(|n| *n != DUMP && !INDEX_FILES.contains(&n.as_str())).collect();
assert!(unexpected.is_empty(), "{unexpected:?}");
let summary: HeapSummary =
serde_json::from_slice(&std::fs::read(dir.path().join("summary.json")).unwrap()).unwrap();
assert_eq!(summary, index.summary);
drop(index);
let retained = Column::<u64>::open(&dir.path().join("retained.u64")).unwrap();
let in_ram = analyze_with(&sample(), in_memory());
assert_eq!(&*retained, &*in_ram.retained);
}
+306
Переглянути файл
@@ -0,0 +1,306 @@
//! Fixed-width arrays that live in RAM or in memory-mapped files, so the analysis arrays of a
//! large heap do not count against the process's anonymous memory.
use std::fs::{File, OpenOptions};
use std::io;
use std::marker::PhantomData;
use std::mem::size_of;
use std::ops::{Deref, DerefMut};
use std::path::{Path, PathBuf};
use memmap2::{Mmap, MmapMut};
/// Index value meaning "no object" in every `u32` column.
pub const NONE: u32 = u32::MAX;
/// # Safety
/// Every bit pattern of `Self` is a valid value and `Self` has no padding, so a mapped file's
/// bytes can be viewed as `[Self]` and a file can be written from one.
pub unsafe trait Pod: Copy + 'static {}
unsafe impl Pod for u8 {}
unsafe impl Pod for u32 {}
unsafe impl Pod for u64 {}
enum Backing<T> {
Ram(Vec<T>),
Map(MmapMut),
ReadOnly(Mmap),
}
/// A fixed-length array of `T`. File contents use the machine's byte order: the arrays are a
/// cache for this machine and are never moved to another one.
pub struct Column<T: Pod> {
backing: Backing<T>,
/// Scratch file to delete on drop where it could not be unlinked while mapped.
delete_on_drop: Option<PathBuf>,
_marker: PhantomData<T>,
}
fn byte_len<T: Pod>(len: usize) -> io::Result<usize> {
len.checked_mul(size_of::<T>())
.filter(|&bytes| bytes <= isize::MAX as usize)
.ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "column is too long"))
}
fn is_zero<T: Pod>(value: &T) -> bool {
// SAFETY: `Pod` types have no padding, so all `size_of::<T>()` bytes are initialized.
let bytes = unsafe { std::slice::from_raw_parts((value as *const T).cast::<u8>(), size_of::<T>()) };
bytes.iter().all(|&b| b == 0)
}
impl<T: Pod> Column<T> {
fn new(backing: Backing<T>, delete_on_drop: Option<PathBuf>) -> Self {
Self { backing, delete_on_drop, _marker: PhantomData }
}
pub fn ram(len: usize, fill: T) -> Self {
Self::new(Backing::Ram(vec![fill; len]), None)
}
/// Creates (or truncates) `path` and maps it; the file stays after the column is dropped.
pub fn file(path: &Path, len: usize, fill: T) -> io::Result<Self> {
let bytes = byte_len::<T>(len)?;
let file = OpenOptions::new().read(true).write(true).create(true).truncate(true).open(path)?;
match Self::map_new(&file, len, bytes, fill) {
Ok(column) => Ok(column),
Err(err) => {
drop(file);
let _ = std::fs::remove_file(path);
Err(err)
}
}
}
/// A temporary column in `dir` that never outlives the process. On unix the file is unlinked
/// right after mapping, so even a crash leaves nothing behind; elsewhere (Windows cannot delete
/// a mapped file) it is deleted on drop.
pub fn scratch(dir: &Path, len: usize, fill: T) -> io::Result<Self> {
let bytes = byte_len::<T>(len)?;
if bytes == 0 {
return Ok(Self::ram(0, fill));
}
let path = dir.join(format!("scratch-{}.tmp", uuid::Uuid::new_v4().simple()));
let file = OpenOptions::new().read(true).write(true).create_new(true).open(&path)?;
let mut column = match Self::map_new(&file, len, bytes, fill) {
Ok(column) => column,
Err(err) => {
drop(file);
let _ = std::fs::remove_file(&path);
return Err(err);
}
};
// If the unlink fails the drop-time delete is the fallback, so the file is never left behind.
if !cfg!(unix) || std::fs::remove_file(&path).is_err() {
column.delete_on_drop = Some(path);
}
Ok(column)
}
/// Maps an existing file read-only; writing through the column panics.
pub fn open(path: &Path) -> io::Result<Self> {
let file = File::open(path)?;
let bytes = file.metadata()?.len();
if bytes % size_of::<T>() as u64 != 0 {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
format!("{} is {bytes} bytes, not a multiple of {}", path.display(), size_of::<T>()),
));
}
if bytes == 0 {
return Ok(Self::ram(0, zero_value()));
}
// SAFETY: the file is private to the snapshot directory; nothing truncates it while mapped.
let map = unsafe { Mmap::map(&file)? };
Ok(Self::new(Backing::ReadOnly(map), None))
}
fn map_new(file: &File, len: usize, bytes: usize, fill: T) -> io::Result<Self> {
if bytes == 0 {
return Ok(Self::ram(0, fill));
}
file.set_len(bytes as u64)?;
// SAFETY: as in `open`; the file was just created by this process.
let map = unsafe { MmapMut::map_mut(file)? };
let mut column = Self::new(Backing::Map(map), None);
// A fresh file reads as zeros, and writing a zero fill would fault in every page.
if !is_zero(&fill) {
debug_assert_eq!(column.len(), len);
column.fill(fill);
}
Ok(column)
}
}
fn zero_value<T: Pod>() -> T {
// SAFETY: every bit pattern of a `Pod` type is a valid value.
unsafe { std::mem::zeroed() }
}
impl<T: Pod> Deref for Column<T> {
type Target = [T];
fn deref(&self) -> &[T] {
match &self.backing {
Backing::Ram(v) => v,
// SAFETY: maps are page-aligned, `T` is `Pod` and the length is a multiple of its width.
Backing::Map(m) => unsafe { std::slice::from_raw_parts(m.as_ptr().cast(), m.len() / size_of::<T>()) },
Backing::ReadOnly(m) => unsafe { std::slice::from_raw_parts(m.as_ptr().cast(), m.len() / size_of::<T>()) },
}
}
}
impl<T: Pod> DerefMut for Column<T> {
fn deref_mut(&mut self) -> &mut [T] {
match &mut self.backing {
Backing::Ram(v) => v,
// SAFETY: as in `deref`; `&mut self` makes the access exclusive.
Backing::Map(m) => unsafe {
std::slice::from_raw_parts_mut(m.as_mut_ptr().cast(), m.len() / size_of::<T>())
},
Backing::ReadOnly(_) => panic!("column opened from a file is read-only"),
}
}
}
impl<T: Pod> Drop for Column<T> {
fn drop(&mut self) {
if let Some(path) = self.delete_on_drop.take() {
// Unmap before deleting: Windows refuses to delete a mapped file.
self.backing = Backing::Ram(Vec::new());
let _ = std::fs::remove_file(path);
}
}
}
/// Where a computation puts its arrays: result columns are named files in `Dir`, temporary arrays
/// are scratch columns in the same directory.
pub enum Store {
Ram,
Dir(PathBuf),
}
impl Store {
/// `name` is the whole file name (`types.u32`); it is used as given.
pub fn column<T: Pod>(&self, name: &str, len: usize, fill: T) -> io::Result<Column<T>> {
match self {
Store::Ram => Ok(Column::ram(len, fill)),
Store::Dir(dir) => Column::file(&dir.join(name), len, fill),
}
}
pub fn scratch<T: Pod>(&self, len: usize, fill: T) -> io::Result<Column<T>> {
match self {
Store::Ram => Ok(Column::ram(len, fill)),
Store::Dir(dir) => Column::scratch(dir, len, fill),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn dir_is_empty(dir: &Path) -> bool {
std::fs::read_dir(dir).unwrap().next().is_none()
}
#[test]
fn file_column_round_trips() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("x.u32");
{
let mut col = Column::<u32>::file(&path, 4, 7).unwrap();
assert_eq!(&*col, &[7, 7, 7, 7]);
col.copy_from_slice(&[1, 2, 3, 4]);
}
let col = Column::<u32>::open(&path).unwrap();
assert_eq!(&*col, &[1, 2, 3, 4]);
}
#[test]
fn scratch_column_leaves_no_file() {
let dir = tempfile::tempdir().unwrap();
{
let mut col = Column::<u64>::scratch(dir.path(), 1024, 9).unwrap();
col[1023] = 5;
assert_eq!((col[0], col[1023]), (9, 5));
// Windows cannot delete a mapped file, so only unix unlinks while the map is alive.
#[cfg(unix)]
assert!(dir_is_empty(dir.path()));
}
assert!(dir_is_empty(dir.path()));
}
#[test]
fn zero_fill_and_non_zero_fill() {
let dir = tempfile::tempdir().unwrap();
let zeros = Column::<u64>::file(&dir.path().join("z.u64"), 3000, 0).unwrap();
assert!(zeros.iter().all(|&v| v == 0));
let ones = Column::<u8>::file(&dir.path().join("o.u8"), 10_000, 0xAB).unwrap();
assert!(ones.iter().all(|&v| v == 0xAB));
let ram = Column::<u32>::ram(5, NONE);
assert_eq!(&*ram, &[NONE; 5]);
}
#[test]
fn empty_columns_work_everywhere() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("e.u32");
assert_eq!(Column::<u32>::ram(0, 1).len(), 0);
assert_eq!(Column::<u32>::file(&path, 0, 1).unwrap().len(), 0);
assert_eq!(Column::<u32>::open(&path).unwrap().len(), 0);
assert_eq!(Column::<u32>::scratch(dir.path(), 0, 1).unwrap().len(), 0);
}
#[test]
fn open_rejects_a_length_that_is_not_a_multiple_of_the_width() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("odd.u32");
std::fs::write(&path, [0u8; 6]).unwrap();
let err = Column::<u32>::open(&path).err().unwrap();
assert_eq!(err.kind(), io::ErrorKind::InvalidData);
}
#[test]
fn file_rejects_a_byte_length_that_overflows() {
let dir = tempfile::tempdir().unwrap();
let err = Column::<u64>::file(&dir.path().join("big.u64"), usize::MAX, 0).err().unwrap();
assert_eq!(err.kind(), io::ErrorKind::InvalidInput);
assert!(dir_is_empty(dir.path()));
}
#[test]
#[should_panic(expected = "read-only")]
fn opened_columns_are_read_only() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("r.u32");
drop(Column::<u32>::file(&path, 2, 0).unwrap());
let mut col = Column::<u32>::open(&path).unwrap();
col[0] = 1;
}
#[test]
fn store_names_files_exactly_as_given() {
let dir = tempfile::tempdir().unwrap();
let store = Store::Dir(dir.path().to_path_buf());
let mut col = store.column::<u32>("types.u32", 3, 0).unwrap();
col[2] = 8;
drop(col);
let names: Vec<_> =
std::fs::read_dir(dir.path()).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect();
assert_eq!(names, ["types.u32"]);
assert_eq!(&*Column::<u32>::open(&dir.path().join("types.u32")).unwrap(), &[0, 0, 8]);
}
#[test]
fn store_scratch_is_anonymous_and_ram_store_touches_no_disk() {
let dir = tempfile::tempdir().unwrap();
let store = Store::Dir(dir.path().to_path_buf());
drop(store.scratch::<u32>(16, 1).unwrap());
assert!(dir_is_empty(dir.path()));
let ram = Store::Ram.column::<u8>("ignored.u8", 4, 2).unwrap();
assert_eq!(&*ram, &[2; 4]);
assert_eq!(&*Store::Ram.scratch::<u8>(2, 3).unwrap(), &[3, 3]);
}
}
+59
Переглянути файл
@@ -0,0 +1,59 @@
//! Class-by-class differences between two snapshots.
use std::collections::HashMap;
use serde::{Deserialize, Serialize};
use crate::index::HeapIndex;
/// One class name in two snapshots; a side without the name has zeros.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapClassDiff {
pub name: String,
pub base_count: u64,
pub count: u64,
pub base_shallow: u64,
pub shallow: u64,
pub base_retained: u64,
pub retained: u64,
/// Several classes have this name on one side or both, so the sides cannot be matched: each
/// side shows the sum over its classes, and there is no difference to show.
pub ambiguous: bool,
}
/// Joined by class name, the only key two dumps share. By the change in retained size
/// descending (none for an ambiguous name), then by name.
pub fn compare(base: &HeapIndex, current: &HeapIndex) -> Vec<HeapClassDiff> {
let mut by_name: HashMap<String, HeapClassDiff> = HashMap::new();
for (index, is_base) in [(base, true), (current, false)] {
for row in index.class_rows() {
let diff = by_name.entry(row.name).or_insert_with_key(|name| HeapClassDiff {
name: name.clone(),
base_count: 0,
count: 0,
base_shallow: 0,
shallow: 0,
base_retained: 0,
retained: 0,
ambiguous: false,
});
let (count, shallow, retained) = if is_base {
(&mut diff.base_count, &mut diff.base_shallow, &mut diff.base_retained)
} else {
(&mut diff.count, &mut diff.shallow, &mut diff.retained)
};
*count = count.saturating_add(row.count);
*shallow = shallow.saturating_add(row.shallow);
*retained = retained.saturating_add(row.retained);
diff.ambiguous |= row.ambiguous;
}
}
let change = |d: &HeapClassDiff| if d.ambiguous { 0 } else { d.retained.abs_diff(d.base_retained) };
let mut diffs: Vec<HeapClassDiff> = by_name.into_values().collect();
diffs.sort_by(|a, b| change(b).cmp(&change(a)).then_with(|| a.name.cmp(&b.name)));
diffs
}
#[cfg(test)]
mod tests;
+78
Переглянути файл
@@ -0,0 +1,78 @@
use super::*;
use crate::hprof::HeapRootKind;
use crate::test_index::analyzed;
use crate::testing::{MiniHeap, sample, sample_with_two_more_nodes};
fn diff<'a>(diffs: &'a [HeapClassDiff], name: &str) -> &'a HeapClassDiff {
let mut rows = diffs.iter().filter(|d| d.name == name);
let row = rows.next().unwrap_or_else(|| panic!("no diff for {name}"));
assert!(rows.next().is_none(), "two diffs for {name}");
row
}
fn sort_key(d: &HeapClassDiff) -> u64 {
if d.ambiguous { 0 } else { d.retained.abs_diff(d.base_retained) }
}
fn assert_sorted(diffs: &[HeapClassDiff]) {
assert!(diffs.windows(2).all(|p| (sort_key(&p[1]), &p[0].name) < (sort_key(&p[0]), &p[1].name)), "{diffs:#?}");
}
#[test]
fn compare_counts_by_class() {
let base = analyzed(&sample());
let mut current = sample_with_two_more_nodes();
current
.class("Extra", Some("java.lang.Object"), &[])
.object(0x4000, "Extra", &[])
.root(HeapRootKind::JniGlobal, 0x4000);
let diffs = compare(&base, &analyzed(&current));
let node = diff(&diffs, "Node");
assert_eq!((node.base_count, node.count), (3, 5));
assert_eq!(node.shallow - node.base_shallow, 48);
assert_eq!((node.base_retained, node.retained), (104, 152));
assert!(!node.ambiguous);
let extra = diff(&diffs, "Extra");
assert_eq!((extra.base_count, extra.base_shallow, extra.base_retained, extra.count), (0, 0, 0, 1));
assert_eq!(diffs[0].name, "Node");
let holder = diff(&diffs, "Holder");
assert_eq!((holder.base_count, holder.count, holder.base_retained, holder.retained), (1, 1, 72, 72));
assert_sorted(&diffs);
// A class only in the base snapshot.
let gone = compare(&analyzed(&current), &base);
let extra = diff(&gone, "Extra");
assert_eq!((extra.base_count, extra.count, extra.shallow, extra.retained), (1, 0, 0, 0));
assert_sorted(&gone);
}
fn two_foos(extra: bool) -> MiniHeap {
let mut heap = sample();
heap.class("Foo", Some("java.lang.Object"), &[])
.object(0x3000, "Foo", &[])
.class("Foo", Some("java.lang.Object"), &[])
.object(0x3100, "Foo", &[]);
if extra {
heap.object(0x3200, "Foo", &[]);
}
for addr in [0x3000, 0x3100, 0x3200] {
heap.root(HeapRootKind::JniGlobal, addr);
}
heap
}
#[test]
fn compare_gives_no_diff_for_a_name_of_several_classes() {
let diffs = compare(&analyzed(&two_foos(false)), &analyzed(&two_foos(true)));
let foo = diff(&diffs, "Foo");
assert!(foo.ambiguous);
assert_eq!((foo.base_count, foo.count), (2, 3));
assert!(diffs.iter().filter(|d| d.name != "Foo").all(|d| !d.ambiguous));
assert_sorted(&diffs);
// Ambiguous on one side only is still ambiguous.
let diffs = compare(&analyzed(&sample()), &analyzed(&two_foos(false)));
let foo = diff(&diffs, "Foo");
assert!(foo.ambiguous);
assert_eq!((foo.base_count, foo.count), (0, 2));
}
+137
Переглянути файл
@@ -0,0 +1,137 @@
//! Disk-space estimates and the per-snapshot directories under the app's cache.
//!
//! A snapshot directory holds the index columns and, for a capture, the dump itself. Its `owner`
//! file names the host process that created it, so a host that starts later can remove the
//! directories of hosts that died without cleaning up.
use std::fs;
use std::io::{self, Read};
use std::path::{Path, PathBuf};
use std::time::{Duration, SystemTime};
/// Space to leave free on top of an estimate.
pub const DISK_MARGIN: u64 = 1 << 30;
/// hprof ≈ `bytes + 16 × instances`; index columns and scratch arrays ≈ `90 × instances`.
const CAPTURE_BYTES_PER_INSTANCE: u64 = 106;
const INDEX_BYTES_PER_OBJECT: u64 = 90;
const OWNER_FILE: &str = "owner";
const TMP_SUFFIX: &str = ".tmp";
/// A directory still being created by a live host has no `owner` for a moment; one that stays
/// unfinished for longer than this was abandoned.
const TMP_GRACE: Duration = Duration::from_secs(60 * 60);
/// Bytes of free space that a dump of `heap_bytes` bytes holding `instances` instances needs,
/// including its index.
pub fn capture_need(heap_bytes: u64, instances: u64) -> u64 {
heap_bytes.saturating_add(CAPTURE_BYTES_PER_INSTANCE.saturating_mul(instances)).saturating_add(DISK_MARGIN)
}
/// Bytes of free space that indexing a dump of `objects` objects needs.
pub fn index_need(objects: u64) -> u64 {
INDEX_BYTES_PER_OBJECT.saturating_mul(objects).saturating_add(DISK_MARGIN)
}
/// Bytes available to this process on the volume holding `dir`; `None` where it is not checked
/// (not unix) or cannot be read, so callers go ahead rather than refuse.
pub fn free_bytes(dir: &Path) -> Option<u64> {
#[cfg(unix)]
{
use std::os::unix::ffi::OsStrExt;
let path = std::ffi::CString::new(dir.as_os_str().as_bytes()).ok()?;
// SAFETY: `path` is NUL-terminated and `stat` is only read after `statvfs` filled it in.
let stat = unsafe {
let mut stat = std::mem::MaybeUninit::<libc::statvfs>::zeroed();
if libc::statvfs(path.as_ptr(), stat.as_mut_ptr()) != 0 {
return None;
}
stat.assume_init()
};
// The field widths differ between unix targets.
#[allow(clippy::unnecessary_cast)]
Some((stat.f_bavail as u64).saturating_mul(stat.f_frsize as u64))
}
#[cfg(not(unix))]
{
let _ = dir;
None
}
}
/// `<local_data_dir>/cache/heap-snapshots`.
pub fn snapshots_root(local_data_dir: &Path) -> PathBuf {
local_data_dir.join("cache").join("heap-snapshots")
}
/// Creates `<root>/<uuid>/` holding an `owner` file with this process's pid. The directory is
/// built as `<uuid>.tmp` and renamed, so another host scanning `root` never sees it without its
/// owner and removes it as abandoned.
pub fn create_snapshot_dir(root: &Path) -> io::Result<PathBuf> {
fs::create_dir_all(root)?;
let name = uuid::Uuid::new_v4().to_string();
let building = root.join(format!("{name}{TMP_SUFFIX}"));
let dir = root.join(&name);
create_private_dir(&building)?;
let published =
fs::write(building.join(OWNER_FILE), std::process::id().to_string()).and_then(|()| fs::rename(&building, &dir));
if let Err(err) = published {
let _ = fs::remove_dir_all(&building);
return Err(err);
}
Ok(dir)
}
/// A dump holds whatever was in the JVM's memory, passwords and tokens included, so on unix only
/// the owner may enter the directory. Windows keeps the ACL inherited from the user's data dir.
fn create_private_dir(path: &Path) -> io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::DirBuilderExt;
fs::DirBuilder::new().mode(0o700).create(path)
}
#[cfg(not(unix))]
{
fs::DirBuilder::new().create(path)
}
}
/// Removes the directories under `root` whose owner file is missing, unreadable or names a pid
/// for which `alive` is false, and unfinished `*.tmp` ones older than an hour. Only directories
/// directly under `root` are touched; a symlink there is left alone.
pub fn remove_stale(root: &Path, alive: impl Fn(u32) -> bool) {
remove_stale_at(root, alive, SystemTime::now());
}
fn remove_stale_at(root: &Path, alive: impl Fn(u32) -> bool, now: SystemTime) {
let Ok(entries) = fs::read_dir(root) else { return };
for entry in entries.flatten() {
let path = entry.path();
// `symlink_metadata` does not follow: a link out of `root` is not a directory here.
let Ok(meta) = fs::symlink_metadata(&path) else { continue };
if !meta.file_type().is_dir() {
continue;
}
let stale = if path.extension().is_some_and(|ext| ext == "tmp") {
meta.modified()
.ok()
.and_then(|modified| now.duration_since(modified).ok())
.is_some_and(|age| age >= TMP_GRACE)
} else {
read_owner(&path).is_none_or(|pid| !alive(pid))
};
if stale && let Err(err) = fs::remove_dir_all(&path) {
tracing::warn!(dir = %path.display(), %err, "could not remove a stale heap snapshot directory");
}
}
}
fn read_owner(dir: &Path) -> Option<u32> {
let mut text = String::new();
fs::File::open(dir.join(OWNER_FILE)).ok()?.take(32).read_to_string(&mut text).ok()?;
text.trim().parse().ok()
}
#[cfg(test)]
mod tests;
+137
Переглянути файл
@@ -0,0 +1,137 @@
use std::time::{Duration, SystemTime};
use super::*;
fn names(dir: &Path) -> Vec<String> {
let mut names: Vec<_> =
std::fs::read_dir(dir).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect();
names.sort();
names
}
fn dir_with_owner(root: &Path, name: &str, owner: Option<&str>) {
let dir = root.join(name);
std::fs::create_dir(&dir).unwrap();
std::fs::write(dir.join("dump.hprof"), b"x").unwrap();
if let Some(owner) = owner {
std::fs::write(dir.join(OWNER_FILE), owner).unwrap();
}
}
#[test]
fn estimates() {
assert_eq!(capture_need(664_018_520, 20_087_506), 664_018_520 + 106 * 20_087_506 + (1 << 30));
assert_eq!(index_need(10), 900 + (1 << 30));
}
#[test]
fn estimates_saturate_instead_of_wrapping() {
assert_eq!(capture_need(u64::MAX, u64::MAX), u64::MAX);
assert_eq!(index_need(u64::MAX), u64::MAX);
}
#[cfg(unix)]
#[test]
fn not_enough_space_for_estimate() {
let dir = tempfile::tempdir().unwrap();
let free = free_bytes(dir.path()).expect("statvfs works on a temp dir");
assert!(free > 0);
assert!(capture_need(u64::MAX / 4, 0) > free);
}
#[cfg(unix)]
#[test]
fn free_bytes_of_a_missing_dir_is_unknown() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(free_bytes(&dir.path().join("missing")), None);
}
#[cfg(not(unix))]
#[test]
fn free_space_is_not_checked_off_unix() {
assert_eq!(free_bytes(Path::new(".")), None);
}
#[test]
fn snapshots_root_is_under_the_cache() {
let local = Path::new("local");
assert_eq!(snapshots_root(local), local.join("cache").join("heap-snapshots"));
}
#[test]
fn snapshot_dir_records_its_owner_and_leaves_no_tmp() {
let root = tempfile::tempdir().unwrap();
let missing_parent = root.path().join("cache").join("heap-snapshots");
let dir = create_snapshot_dir(&missing_parent).unwrap();
assert_eq!(dir.parent(), Some(missing_parent.as_path()));
assert!(uuid::Uuid::parse_str(dir.file_name().unwrap().to_str().unwrap()).is_ok());
assert_eq!(std::fs::read_to_string(dir.join(OWNER_FILE)).unwrap(), std::process::id().to_string());
assert_eq!(names(&missing_parent), [dir.file_name().unwrap().to_str().unwrap()]);
assert_ne!(create_snapshot_dir(&missing_parent).unwrap(), dir);
}
#[cfg(unix)]
#[test]
fn snapshot_dir_is_private_to_the_owner() {
use std::os::unix::fs::PermissionsExt;
let root = tempfile::tempdir().unwrap();
let dir = create_snapshot_dir(root.path()).unwrap();
assert_eq!(std::fs::metadata(&dir).unwrap().permissions().mode() & 0o777, 0o700);
}
#[test]
fn stale_dirs_removed_on_start() {
let root = tempfile::tempdir().unwrap();
dir_with_owner(root.path(), "a", Some("999999"));
dir_with_owner(root.path(), "b", Some(&std::process::id().to_string()));
dir_with_owner(root.path(), "c", None);
remove_stale(root.path(), |pid| pid == std::process::id());
assert_eq!(names(root.path()), ["b"]);
}
#[test]
fn owner_that_is_not_a_pid_is_stale() {
let root = tempfile::tempdir().unwrap();
dir_with_owner(root.path(), "empty", Some(""));
dir_with_owner(root.path(), "word", Some("not-a-pid"));
dir_with_owner(root.path(), "negative", Some("-1"));
dir_with_owner(root.path(), "padded", Some(&format!(" {}\n", std::process::id())));
remove_stale(root.path(), |pid| pid == std::process::id());
assert_eq!(names(root.path()), ["padded"]);
}
#[test]
fn stale_scan_tolerates_a_missing_root_and_leaves_loose_files_alone() {
let root = tempfile::tempdir().unwrap();
remove_stale(&root.path().join("missing"), |_| false);
std::fs::write(root.path().join("notes.txt"), b"keep").unwrap();
remove_stale(root.path(), |_| false);
assert_eq!(names(root.path()), ["notes.txt"]);
}
#[test]
fn unfinished_dir_is_left_alone_for_an_hour() {
let root = tempfile::tempdir().unwrap();
// Another host may be between `create_dir` and the rename of its own snapshot dir.
dir_with_owner(root.path(), "young.tmp", None);
let now = SystemTime::now();
remove_stale_at(root.path(), |_| false, now);
assert_eq!(names(root.path()), ["young.tmp"]);
remove_stale_at(root.path(), |_| false, now + TMP_GRACE - Duration::from_secs(60));
assert_eq!(names(root.path()), ["young.tmp"]);
remove_stale_at(root.path(), |_| false, now + TMP_GRACE + Duration::from_secs(60));
assert!(names(root.path()).is_empty());
}
#[cfg(unix)]
#[test]
fn stale_scan_never_follows_a_symlink_out_of_the_root() {
let outside = tempfile::tempdir().unwrap();
std::fs::write(outside.path().join("precious"), b"x").unwrap();
let root = tempfile::tempdir().unwrap();
std::os::unix::fs::symlink(outside.path(), root.path().join("link")).unwrap();
remove_stale(root.path(), |_| false);
remove_stale_at(root.path(), |_| false, SystemTime::now() + 2 * TMP_GRACE);
assert_eq!(names(outside.path()), ["precious"]);
}
+247
Переглянути файл
@@ -0,0 +1,247 @@
//! Dominator tree of the object graph, by Lengauer–Tarjan with path compression and no recursion:
//! the DFS "stack" is the chain of DFS parents, and `eval` reverses the links of the path it
//! compresses instead of keeping a stack.
//!
//! Between the DFS and the final translation every array is indexed by DFS number, so the root is
//! 0 and a vertex's DFS ancestors all have smaller numbers.
use ide_core::CancellationToken;
use crate::column::{Column, NONE, Store};
use crate::error::HeapError;
use crate::files::IDOM_FILE;
use crate::graph::accumulate;
use crate::progress::{CHECK_EVERY, HeapPhase, Progress};
const PHASE: HeapPhase = HeapPhase::Dominators;
pub struct Dominators {
/// Immediate dominator of each node, saved as `idom.u32` in the store; [`NONE`] for
/// unreachable nodes, `root` for `root`.
pub idom: Column<u32>,
/// Nodes in DFS preorder from `root`, `reachable` entries.
pub preorder: Column<u32>,
pub reachable: u32,
}
/// The dominators of the graph whose node `v` has targets `out[out_start[v]..out_start[v + 1]]`,
/// from `root` (which must be a node). `progress` gets `Dominators` from `(0, total)` to
/// `(total, total)`; an `Err` from it stops the computation with that error.
pub fn dominators(
out_start: &[u32],
out: &[u32],
root: u32,
store: &Store,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<Dominators, HeapError> {
compute(out_start, out, root, store, progress, cancel).map(|(dominators, _)| dominators)
}
/// [`dominators`], plus the number of steps of work it took.
fn compute(
out_start: &[u32],
out: &[u32],
root: u32,
store: &Store,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<(Dominators, u64), HeapError> {
let nodes = out_start.len().saturating_sub(1);
if nodes >= NONE as usize {
return Err(HeapError::TooLarge);
}
assert!((root as usize) < nodes, "root {root} is not one of the {nodes} nodes");
// Three walks of at most `nodes` vertices each: DFS, semidominators, immediate dominators.
let span = nodes as u64;
let mut work = Work::new(Progress::new(progress, cancel), 3 * span);
work.report(0)?;
// DFS: `pre` maps nodes to DFS numbers and `vertex` back; `cursor[d]` is the next edge of
// DFS vertex d to follow.
let mut pre = store.scratch::<u32>(nodes, NONE)?;
let mut vertex = store.scratch::<u32>(nodes, 0)?;
let mut parent = store.scratch::<u32>(nodes, 0)?;
let mut cursor = store.scratch::<u32>(nodes, 0)?;
pre[root as usize] = 0;
vertex[0] = root;
cursor[0] = out_start[root as usize];
let (mut reachable, mut d) = (1u32, 0u32);
loop {
work.tick()?;
let edge = cursor[d as usize];
if edge < out_start[vertex[d as usize] as usize + 1] {
cursor[d as usize] = edge + 1;
let w = out[edge as usize] as usize;
if pre[w] == NONE {
let next = reachable as usize;
pre[w] = reachable;
vertex[next] = w as u32;
parent[next] = d;
cursor[next] = out_start[w];
d = reachable;
reachable += 1;
work.done = u64::from(reachable);
}
} else if d == 0 {
break;
} else {
d = parent[d as usize];
}
}
let r = reachable as usize;
work.report(span)?;
// Predecessors by DFS number, as CSR: counted into `starts[w + 1]`, then filled through
// `starts[w]` as a cursor, which leaves it at w's end until the column is shifted one slot.
let mut semi = cursor;
let mut label = store.scratch::<u32>(r, 0)?;
let mut starts = store.scratch::<u32>(r + 1, 0)?;
for d in 0..r {
work.tick()?;
semi[d] = d as u32;
label[d] = d as u32;
for &w in targets(out_start, out, vertex[d]) {
work.tick()?;
starts[pre[w as usize] as usize + 1] += 1;
}
}
let edges = accumulate(&mut starts, cancel)?;
let mut preds = store.scratch::<u32>(edges as usize, 0)?;
for d in 0..r {
work.tick()?;
for &w in targets(out_start, out, vertex[d]) {
work.tick()?;
let slot = &mut starts[pre[w as usize] as usize];
preds[*slot as usize] = d as u32;
*slot += 1;
}
}
starts.copy_within(0..r, 1);
starts[0] = 0;
drop(pre);
work.report(span)?;
// Semidominators in reverse preorder, linking each finished vertex to its parent. Vertex w
// waits in its semidominator's bucket until a child of that semidominator is linked; then
// `eval` gives w's idom, or a vertex whose idom is also w's, fixed up in preorder below.
// `idom[v]` is the next vertex in v's bucket until v leaves the bucket.
let mut ancestor = store.scratch::<u32>(r, NONE)?;
let mut bucket = store.scratch::<u32>(r, NONE)?;
let mut idom = store.scratch::<u32>(r, NONE)?;
for w in (1..r).rev() {
work.done = span + (r - w) as u64;
work.tick()?;
let mut s = semi[w];
for &v in &preds[starts[w] as usize..starts[w + 1] as usize] {
work.tick()?;
let u = eval(v, &mut ancestor, &mut label, &semi, &mut work)?;
s = s.min(semi[u as usize]);
}
semi[w] = s;
idom[w] = bucket[s as usize];
bucket[s as usize] = w as u32;
let p = parent[w];
ancestor[w] = p;
let mut v = std::mem::replace(&mut bucket[p as usize], NONE);
while v != NONE {
work.tick()?;
let next = idom[v as usize];
let u = eval(v, &mut ancestor, &mut label, &semi, &mut work)?;
idom[v as usize] = if semi[u as usize] < semi[v as usize] { u } else { p };
v = next;
}
}
drop((label, starts, preds, ancestor, bucket, parent));
work.report(2 * span)?;
// Immediate dominators, translated back to nodes; in preorder, so a deferred vertex's
// stand-in already has its final idom.
let mut node_idom = store.column::<u32>(IDOM_FILE, nodes, NONE)?;
let mut preorder = store.scratch::<u32>(r, 0)?;
idom[0] = 0;
node_idom[root as usize] = root;
preorder[0] = root;
for w in 1..r {
work.done = 2 * span + w as u64;
work.tick()?;
if idom[w] != semi[w] {
idom[w] = idom[idom[w] as usize];
}
node_idom[vertex[w] as usize] = vertex[idom[w] as usize];
preorder[w] = vertex[w];
}
work.report(3 * span)?;
Ok((Dominators { idom: node_idom, preorder, reachable }, work.steps))
}
fn targets<'a>(out_start: &[u32], out: &'a [u32], v: u32) -> &'a [u32] {
let v = v as usize;
&out[out_start[v] as usize..out_start[v + 1] as usize]
}
/// Counts steps of work and reports progress, which also looks at the cancellation token, at
/// every phase boundary and after every [`CHECK_EVERY`] steps, whichever loop they are in.
struct Work<'a> {
progress: Progress<'a>,
total: u64,
/// Progress to report at the next check; the loops keep it current.
done: u64,
steps: u64,
next_report: u64,
}
impl<'a> Work<'a> {
fn new(progress: Progress<'a>, total: u64) -> Self {
Self { progress, total, done: 0, steps: 0, next_report: CHECK_EVERY }
}
fn report(&mut self, done: u64) -> Result<(), HeapError> {
self.done = done;
self.next_report = self.steps + CHECK_EVERY;
self.progress.report(PHASE, done, self.total)
}
#[inline]
fn tick(&mut self) -> Result<(), HeapError> {
self.steps += 1;
if self.steps < self.next_report { Ok(()) } else { self.report(self.done) }
}
}
/// The vertex of smallest semidominator on the forest path from `v` up to, not including, its
/// tree's root (`v` itself when it is a root), compressing that path so the root becomes the
/// ancestor of every vertex on it. The walk up reverses the `ancestor` links so the walk down can
/// fold labels nearest-root first, as the recursive compress does, without a stack.
fn eval(v: u32, ancestor: &mut [u32], label: &mut [u32], semi: &[u32], work: &mut Work) -> Result<u32, HeapError> {
let at = |x: u32| x as usize;
if ancestor[at(v)] == NONE {
return Ok(v);
}
let (mut x, mut below) = (v, NONE);
while ancestor[at(ancestor[at(x)])] != NONE {
work.tick()?;
let up = ancestor[at(x)];
ancestor[at(x)] = below;
below = x;
x = up;
}
// `x` is the root's child, already compressed; `below` is the vertex under it.
let root = ancestor[at(x)];
let mut best = label[at(x)];
while below != NONE {
work.tick()?;
let y = at(below);
if semi[at(best)] < semi[at(label[y])] {
label[y] = best;
} else {
best = label[y];
}
below = ancestor[y];
ancestor[y] = root;
}
Ok(label[at(v)])
}
#[cfg(test)]
mod tests;
+348
Переглянути файл
@@ -0,0 +1,348 @@
use std::cell::RefCell;
use std::time::Instant;
use proptest::collection::vec;
use proptest::prelude::*;
use super::*;
use crate::NONE;
use crate::graph::build_graph;
use crate::testing::{XorShift, sample};
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
/// CSR of `edges` over `n` nodes, targets in edge order.
fn csr(n: usize, edges: &[(u32, u32)]) -> (Vec<u32>, Vec<u32>) {
let mut start = vec![0u32; n + 1];
for &(from, _) in edges {
start[from as usize + 1] += 1;
}
for v in 0..n {
start[v + 1] += start[v];
}
let mut cursor = start.clone();
let mut out = vec![0u32; edges.len()];
for &(from, to) in edges {
out[cursor[from as usize] as usize] = to;
cursor[from as usize] += 1;
}
(start, out)
}
fn run(n: usize, edges: &[(u32, u32)], root: u32) -> Dominators {
let (start, out) = csr(n, edges);
dominators(&start, &out, root, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap()
}
fn preds(n: usize, edges: &[(u32, u32)]) -> Vec<Vec<u32>> {
let mut preds = vec![Vec::new(); n];
for &(from, to) in edges {
preds[to as usize].push(from);
}
preds
}
/// Immediate dominators from iterated `Dom` sets (one `u64` bitset per node, `n <= 64`).
fn dataflow_idom(n: usize, edges: &[(u32, u32)], root: u32) -> Vec<u32> {
let (start, out) = csr(n, edges);
let mut reachable = vec![false; n];
let mut todo = vec![root];
reachable[root as usize] = true;
while let Some(v) = todo.pop() {
for &w in &out[start[v as usize] as usize..start[v as usize + 1] as usize] {
if !std::mem::replace(&mut reachable[w as usize], true) {
todo.push(w);
}
}
}
let preds = preds(n, edges);
let mut dom = vec![u64::MAX; n];
dom[root as usize] = 1 << root;
let mut changed = true;
while changed {
changed = false;
for v in (0..n).filter(|&v| reachable[v] && v != root as usize) {
let meet =
preds[v].iter().filter(|&&p| reachable[p as usize]).fold(u64::MAX, |acc, &p| acc & dom[p as usize]);
let next = meet | 1 << v;
if next != dom[v] {
dom[v] = next;
changed = true;
}
}
}
(0..n)
.map(|v| {
if !reachable[v] {
NONE
} else if v == root as usize {
root
} else {
// The strict dominator dominated by all the others has the most dominators itself.
let strict = dom[v] & !(1 << v);
(0..n as u32).filter(|&d| strict & 1 << d != 0).max_by_key(|&d| dom[d as usize].count_ones()).unwrap()
}
})
.collect()
}
/// Cooper–Harvey–Kennedy iteration over reverse postorder from an explicit-stack DFS.
fn chk_idom(n: usize, edges: &[(u32, u32)], root: u32) -> Vec<u32> {
let (start, out) = csr(n, edges);
let mut post_number = vec![NONE; n];
let mut postorder = Vec::new();
let mut seen = vec![false; n];
let mut stack = vec![(root, start[root as usize])];
seen[root as usize] = true;
while let Some(&mut (v, ref mut next)) = stack.last_mut() {
if *next < start[v as usize + 1] {
let w = out[*next as usize];
*next += 1;
if !std::mem::replace(&mut seen[w as usize], true) {
stack.push((w, start[w as usize]));
}
} else {
post_number[v as usize] = postorder.len() as u32;
postorder.push(v);
stack.pop();
}
}
let preds = preds(n, edges);
let mut idom = vec![NONE; n];
idom[root as usize] = root;
let intersect = |idom: &[u32], mut a: u32, mut b: u32| {
while a != b {
while post_number[a as usize] < post_number[b as usize] {
a = idom[a as usize];
}
while post_number[b as usize] < post_number[a as usize] {
b = idom[b as usize];
}
}
a
};
let mut changed = true;
while changed {
changed = false;
for &v in postorder.iter().rev().filter(|&&v| v != root) {
let mut new = NONE;
for &p in &preds[v as usize] {
if idom[p as usize] != NONE {
new = if new == NONE { p } else { intersect(&idom, p, new) };
}
}
if new != idom[v as usize] {
idom[v as usize] = new;
changed = true;
}
}
}
idom
}
#[test]
fn diamond() {
let d = run(5, &[(0, 1), (0, 2), (1, 3), (2, 3), (3, 4)], 0);
assert_eq!(&*d.idom, &[0, 0, 0, 0, 3]);
assert_eq!(d.reachable, 5);
assert_eq!(&*d.preorder, &[0, 1, 3, 4, 2]);
}
#[test]
fn unreachable_is_none() {
let d = run(6, &[(0, 1), (0, 2), (1, 3), (2, 3), (3, 4), (5, 3)], 0);
assert_eq!(&*d.idom, &[0, 0, 0, 0, 3, NONE]);
assert_eq!(d.reachable, 5);
assert_eq!(d.preorder.len(), 5);
}
#[test]
fn cycle_back_to_root() {
let d = run(3, &[(0, 1), (1, 2), (2, 1), (2, 0)], 0);
assert_eq!(&*d.idom, &[0, 0, 1]);
}
#[test]
fn root_need_not_be_node_zero() {
let d = run(4, &[(3, 1), (1, 0), (3, 2), (2, 0), (0, 3)], 3);
assert_eq!(&*d.idom, &[3, 3, 3, 3]);
assert_eq!(d.preorder[0], 3);
}
#[test]
fn sample_heap_dominators() {
let bytes = sample().bytes();
let g = build_graph(&bytes, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap();
let d = dominators(&g.out_start, &g.out, g.root, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap();
let node = |id: u64| g.ids.rank(id).unwrap();
let idom_of = |id: u64| d.idom[node(id) as usize];
assert_eq!(idom_of(0x1100), node(0x1000), "N1 → H");
assert_eq!(idom_of(0x1200), node(0x1000), "N2 → H");
assert_eq!(idom_of(0x1300), g.root, "N3 → root");
assert_eq!(idom_of(0x1400), node(0x1300), "L → N3");
assert_eq!(idom_of(0x1500), NONE, "R is unreachable");
assert_eq!(idom_of(0x1700), NONE, "N4 only through the weak referent");
assert_eq!(d.idom[g.root as usize], g.root);
}
#[test]
fn idom_is_written_to_the_store() {
let dir = tempfile::tempdir().unwrap();
let (start, out) = csr(3, &[(0, 1), (1, 2)]);
let store = Store::Dir(dir.path().to_path_buf());
let d = dominators(&start, &out, 0, &store, &no_progress, &CancellationToken::new()).unwrap();
drop(d);
assert_eq!(&*Column::<u32>::open(&dir.path().join("idom.u32")).unwrap(), &[0, 0, 1]);
let names: Vec<_> = std::fs::read_dir(dir.path()).unwrap().map(|e| e.unwrap().file_name()).collect();
assert_eq!(names, ["idom.u32"], "scratch columns leave no files");
}
#[test]
fn reports_dominators_progress_and_stops_on_cancel() {
let (start, out) = csr(3, &[(0, 1), (1, 2)]);
let calls = RefCell::new(Vec::new());
let record = |phase, done, total| {
calls.borrow_mut().push((phase, done, total));
Ok(())
};
dominators(&start, &out, 0, &Store::Ram, &record, &CancellationToken::new()).unwrap();
let calls = calls.into_inner();
let total = calls[0].2;
assert_eq!(calls.first(), Some(&(HeapPhase::Dominators, 0, total)));
assert_eq!(calls.last(), Some(&(HeapPhase::Dominators, total, total)));
assert!(calls.windows(2).all(|w| w[0].1 <= w[1].1), "progress never goes back: {calls:?}");
let cancel = CancellationToken::new();
let cancel_on_first = |_, _, _| {
cancel.cancel();
Ok(())
};
let result = dominators(&start, &out, 0, &Store::Ram, &cancel_on_first, &cancel);
assert!(matches!(result, Err(HeapError::Cancelled)));
let refuse = |_, _, _| Err(HeapError::TooLarge);
let result = dominators(&start, &out, 0, &Store::Ram, &refuse, &CancellationToken::new());
assert!(matches!(result, Err(HeapError::TooLarge)));
}
#[test]
fn cancel_during_idom_phase() {
// Long enough that the last phase alone passes a check interval.
const NODES: u32 = (CHECK_EVERY + CHECK_EVERY / 2) as u32;
let edges: Vec<_> = (1..NODES).map(|v| (v - 1, v)).collect();
let (start, out) = csr(NODES as usize, &edges);
let span = u64::from(NODES);
let cancel = CancellationToken::new();
let calls = RefCell::new(Vec::new());
let cancel_past_semidominators = |_, done, _| {
calls.borrow_mut().push(done);
if done > 2 * span {
cancel.cancel();
}
Ok(())
};
let result = dominators(&start, &out, 0, &Store::Ram, &cancel_past_semidominators, &cancel);
assert!(matches!(result, Err(HeapError::Cancelled)));
let calls = calls.into_inner();
assert!(calls.contains(&(2 * span)), "the semidominator phase finished: {calls:?}");
let last = *calls.last().unwrap();
assert!(last > 2 * span && last < 3 * span, "stopped inside the idom phase: {calls:?}");
}
proptest! {
#[test]
fn matches_dataflow_on_random_graphs(n in 2usize..64, edges in vec((0u32..64, 0u32..64), 0..192)) {
let edges: Vec<_> = edges.into_iter().map(|(a, b)| (a % n as u32, b % n as u32)).collect();
let d = run(n, &edges, 0);
prop_assert_eq!(d.idom.to_vec(), dataflow_idom(n, &edges, 0));
prop_assert_eq!(d.preorder.len(), d.reachable as usize);
}
}
#[test]
fn matches_chk_on_a_large_random_graph() {
const NODES: u32 = 200_000;
const EXTRA_EDGES: u32 = 100_000;
let mut rng = XorShift::new(0x5EED_D0D0);
let mut edges: Vec<_> = (1..NODES).map(|v| (rng.below(v), v)).collect();
edges.extend((0..EXTRA_EDGES).map(|_| (rng.below(NODES), rng.below(NODES))));
// Visit edges in a scrambled order so the DFS tree differs from the generating tree.
for i in (1..edges.len()).rev() {
edges.swap(i, rng.below(i as u32 + 1) as usize);
}
let started = Instant::now();
let d = run(NODES as usize, &edges, 0);
let elapsed = started.elapsed();
let expected = chk_idom(NODES as usize, &edges, 0);
let mismatches = d.idom.iter().zip(&expected).filter(|(a, b)| a != b).count();
assert_eq!(mismatches, 0);
assert_eq!(d.reachable, NODES);
eprintln!("dominators of {NODES} random nodes: {elapsed:?}");
}
/// A doubly linked list whose items an array list also holds: super-root 0 → list 1 → node 0,
/// 0 → array list 2 → element array 3 → every item; node j → item j, next, prev.
fn shared_list_and_array(k: u32) -> (usize, Vec<(u32, u32)>) {
let node = |j: u32| 4 + 2 * j;
let item = |j: u32| 5 + 2 * j;
let mut edges = vec![(0, 1), (0, 2), (2, 3), (1, node(0))];
for j in 0..k {
edges.push((node(j), item(j)));
if j + 1 < k {
edges.push((node(j), node(j + 1)));
}
if j > 0 {
edges.push((node(j), node(j - 1)));
}
edges.push((3, item(j)));
}
((4 + 2 * k) as usize, edges)
}
#[test]
fn shared_list_and_array_is_near_linear() {
let (n, edges) = shared_list_and_array(29);
assert_eq!(run(n, &edges, 0).idom.to_vec(), dataflow_idom(n, &edges, 0));
let (n, edges) = shared_list_and_array(3_000);
assert_eq!(run(n, &edges, 0).idom.to_vec(), chk_idom(n, &edges, 0));
const K: u32 = 200_000;
let (n, edges) = shared_list_and_array(K);
let (start, out) = csr(n, &edges);
let started = Instant::now();
let (d, steps) = compute(&start, &out, 0, &Store::Ram, &no_progress, &CancellationToken::new()).unwrap();
let elapsed = started.elapsed();
let item = |j: u32| (5 + 2 * j) as usize;
assert!((0..K).all(|j| d.idom[item(j)] == 0), "every item is held by both collections");
let m = edges.len() as u64;
let bound = (n as u64 + m) * u64::from(n.ilog2());
eprintln!("shared list k={K}: {steps} steps (n={n}, m={m}, bound {bound}), {elapsed:?}");
assert!(steps <= bound, "{steps} steps for n={n}, m={m}");
}
#[test]
fn deep_chain_has_linear_dominators() {
// A small stack proves the algorithm does not recurse along the chain.
std::thread::Builder::new()
.stack_size(256 << 10)
.spawn(|| {
const NODES: u32 = 1_000_000;
let mut edges: Vec<_> = (1..NODES).map(|v| (v - 1, v)).collect();
edges.push((0, NODES - 1));
// Makes `eval` compress the whole chain at once.
edges.push((NODES - 1, 1));
let started = Instant::now();
let d = run(NODES as usize, &edges, 0);
eprintln!("dominators of a {NODES}-node chain: {:?}", started.elapsed());
assert_eq!(d.reachable, NODES);
assert_eq!(d.idom[0], 0);
assert_eq!(d.idom[NODES as usize - 1], 0);
let wrong = (1..NODES - 1).filter(|&v| d.idom[v as usize] != v - 1).count();
assert_eq!(wrong, 0);
})
.unwrap()
.join()
.unwrap();
}
+26
Переглянути файл
@@ -0,0 +1,26 @@
/// Offsets (`at`) are byte positions in the `.hprof` file.
#[derive(Debug, thiserror::Error)]
pub enum HeapError {
#[error("the dump ends early at byte {at}")]
Truncated { at: u64 },
#[error("not a heap dump (no JAVA PROFILE header)")]
NotHprof,
#[error("unsupported identifier size {0}")]
UnsupportedIdSize(u32),
#[error("unknown heap sub-record tag {tag:#04x} at byte {at}")]
UnknownSubRecord { tag: u8, at: u64 },
#[error("unknown value type {ty} at byte {at}")]
UnknownType { ty: u8, at: u64 },
#[error("class {id:#x} used at byte {at} is not in the dump")]
MissingClass { id: u64, at: u64 },
#[error("the heap has too many objects to analyze")]
TooLarge,
#[error("the dump's class and field names are too long; it is not a dump a JVM wrote")]
NamesTooLarge,
#[error("not enough disk space: {needed} bytes needed, {available} free")]
NotEnoughDiskSpace { needed: u64, available: u64 },
#[error("cancelled")]
Cancelled,
#[error("{0}")]
Io(#[from] std::io::Error),
}
+34
Переглянути файл
@@ -0,0 +1,34 @@
//! Names of the files an analysis writes into its index directory. They live in one list so a
//! failed analysis removes exactly these and never the dump or anything else sharing the directory.
pub(crate) const IDS_BITS_FILE: &str = "ids.bits.u64";
pub(crate) const IDS_WRANK_FILE: &str = "ids.wrank.u32";
pub(crate) const IDS_TABLE_FILE: &str = "ids.table.u32";
pub(crate) const IDS_CHUNKS_FILE: &str = "ids.chunks.u32";
pub(crate) const IDS_SORTED_FILE: &str = "ids.sorted.u64";
pub(crate) const TYPES_FILE: &str = "types.u32";
pub(crate) const OFFSETS_FILE: &str = "offsets.u64";
pub(crate) const IDOM_FILE: &str = "idom.u32";
pub(crate) const SHALLOW8_FILE: &str = "shallow8.u32";
pub(crate) const RETAINED_FILE: &str = "retained.u64";
pub(crate) const KIDS_START_FILE: &str = "kids_start.u32";
pub(crate) const KIDS_FILE: &str = "kids.u32";
pub(crate) const GC_PARENT_FILE: &str = "gc_parent.u32";
pub(crate) const SUMMARY_FILE: &str = "summary.json";
pub(crate) const INDEX_FILES: &[&str] = &[
IDS_BITS_FILE,
IDS_WRANK_FILE,
IDS_TABLE_FILE,
IDS_CHUNKS_FILE,
IDS_SORTED_FILE,
TYPES_FILE,
OFFSETS_FILE,
IDOM_FILE,
SHALLOW8_FILE,
RETAINED_FILE,
KIDS_START_FILE,
KIDS_FILE,
GC_PARENT_FILE,
SUMMARY_FILE,
];
+274
Переглянути файл
@@ -0,0 +1,274 @@
//! The object graph of a heap dump in CSR form: one node per object, plus a super-root.
mod classes;
mod link;
use std::collections::hash_map::Entry;
use std::collections::{HashMap, HashSet};
use ide_core::CancellationToken;
use self::classes::{ClassTable, IdHash};
use self::link::Linker;
use crate::column::{Column, Store};
use crate::error::HeapError;
use crate::files::{OFFSETS_FILE, TYPES_FILE};
use crate::hprof::{BasicType, HeapRecord, HeapRootKind, Record, parse_header, scan, scan_segment};
use crate::ids::{IdMap, IdMapBuilder};
use crate::progress::{CHECK_EVERY, HeapPhase, Progress};
/// Number of primitive array type slots, one per non-object [`BasicType`].
pub const PRIM_SLOTS: u32 = 8;
pub struct ClassInfo {
pub id: u64,
/// The hprof name with `/` turned into `.` (`java.lang.String`, `[Ljava.lang.Object;`).
pub name: String,
/// Index in [`HeapGraph::classes`]. Chains always end: a super cycle in the dump is cut where it closes.
pub super_index: Option<u32>,
/// This class's own instance fields, in dump order.
pub fields: Vec<(String, BasicType)>,
/// `(name, object id)` of reference statics; a null static has object id 0.
pub statics: Vec<(String, u64)>,
pub static_bytes: u64,
/// Index in `fields` of `java.lang.ref.Reference.referent`, on that class only.
pub weak_field: Option<usize>,
}
pub struct HeapGraph {
pub ids: IdMap,
/// Type slot of each object: class index for instances and object arrays,
/// [`Self::class_slot`] for class objects, [`Self::prim_slot`] for primitive arrays.
pub types: Column<u32>,
/// Array length, 0 for instances and classes.
pub lens: Column<u32>,
/// File offset of each object's record, for [`crate::record_at`].
pub offsets: Column<u64>,
/// `n + 2` entries: node `v`'s targets are `out[out_start[v]..out_start[v + 1]]`.
pub out_start: Column<u32>,
pub out: Column<u32>,
/// The super-root, node `n`, with an edge to every rooted object.
pub root: u32,
/// `(object, kind, thread serial)` of every root record whose object is in the dump.
pub roots: Vec<(u32, HeapRootKind, Option<u32>)>,
/// Class dumps in dump order, then a synthetic `java.lang.Class` when the dump has none.
pub classes: Vec<ClassInfo>,
/// Type slot of class objects (`java.lang.Class`).
pub class_slot: u32,
/// Thread serial → thread object, from `ROOT THREAD OBJECT` records.
pub thread_objects: HashMap<u32, u32>,
/// `Reference.referent` edges left out of the graph.
pub weak_edges_skipped: u64,
pub id_size: u8,
pub timestamp_ms: u64,
}
impl HeapGraph {
pub fn prim_slot(&self, ty: BasicType) -> u32 {
self.classes.len() as u32 + (ty as u32 - BasicType::Boolean as u32)
}
/// The element type of primitive array slot `slot`, or `None` for a class slot.
pub fn prim_type(&self, slot: u32) -> Option<BasicType> {
let offset = slot.checked_sub(self.classes.len() as u32).filter(|&o| o < PRIM_SLOTS)?;
BasicType::from_u8(offset as u8 + BasicType::Boolean as u8)
}
pub fn out_edges(&self, v: u32) -> &[u32] {
let v = v as usize;
&self.out[self.out_start[v] as usize..self.out_start[v + 1] as usize]
}
}
/// Builds the graph of the dump in `buf`. Strong edges only: `Reference.referent` is counted in
/// [`HeapGraph::weak_edges_skipped`] instead. `progress` gets `Reading` by bytes, then
/// `Linking` by objects, starting with `(Linking, 0, objects)` once the object count is known;
/// an `Err` from it stops the build with that error.
pub fn build_graph(
buf: &[u8],
store: &Store,
progress: &dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &CancellationToken,
) -> Result<HeapGraph, HeapError> {
let progress = Progress::new(progress, cancel);
let header = parse_header(buf)?;
let id_size = header.id_size;
let len = buf.len() as u64;
// Pass 1: names, classes, roots and every object id.
progress.report(HeapPhase::Reading, 0, len)?;
let mut names: IdHash<&[u8]> = IdHash::default();
let mut class_names: IdHash<u64> = IdHash::default();
let mut dumps = Vec::new();
let mut class_by_id: IdHash<u32> = IdHash::default();
let mut raw_roots = Vec::new();
let mut segments = Vec::new();
let mut builder = IdMapBuilder::new();
let (mut step, mut pos) = (0, header.body_start as u64);
scan(buf, &header, &mut |record| {
progress.step(step, HeapPhase::Reading, pos, len)?;
step += 1;
match record {
Record::Utf8 { id, text } => {
names.insert(id, text);
}
Record::LoadClass { class_id, name_id } => {
class_names.insert(class_id, name_id);
}
Record::Segment(range) => {
pos = range.start as u64;
segments.push(range);
}
Record::Heap { at, record } => {
pos = at as u64;
match record {
HeapRecord::Root { kind, id, thread_serial } => raw_roots.push((id, kind, thread_serial)),
HeapRecord::Class(dump) => {
builder.insert(dump.id);
if let Entry::Vacant(slot) = class_by_id.entry(dump.id) {
slot.insert(dumps.len() as u32);
dumps.push(dump);
}
}
HeapRecord::Instance { id, .. }
| HeapRecord::ObjectArray { id, .. }
| HeapRecord::PrimitiveArray { id, .. } => builder.insert(id),
}
}
}
Ok(())
})?;
progress.report(HeapPhase::Reading, len, len)?;
let ids = builder.build(store, cancel)?;
let table = classes::resolve(dumps, class_by_id, &names, &class_names, id_size, buf.len())?;
drop((names, class_names));
let n = ids.len();
let objects = u64::from(n);
progress.report(HeapPhase::Linking, 0, objects)?;
let mut roots = Vec::new();
let mut root_targets = Vec::new();
let mut thread_objects = HashMap::new();
let (mut seen_records, mut seen_targets) = (HashSet::new(), HashSet::new());
for (i, &(id, kind, serial)) in raw_roots.iter().enumerate() {
progress.step(i as u64, HeapPhase::Linking, 0, objects)?;
let Some(object) = ids.rank(id).filter(|_| id != 0) else { continue };
if seen_records.insert((object, kind as u8, serial)) {
roots.push((object, kind, serial));
}
if seen_targets.insert(object) {
root_targets.push(object);
}
if let (HeapRootKind::ThreadObject, Some(serial)) = (kind, serial) {
thread_objects.entry(serial).or_insert(object);
}
}
drop((raw_roots, seen_records, seen_targets));
let linker = Linker::new(&ids, &table, id_size);
let nodes = n as usize;
let mut types = store.column::<u32>(TYPES_FILE, nodes, 0)?;
let mut lens = store.scratch::<u32>(nodes, 0)?;
let mut offsets = store.column::<u64>(OFFSETS_FILE, nodes, 0)?;
let mut out_start = store.scratch::<u32>(nodes + 2, 0)?;
// Both passes visit every object record once; report them as one walk over the objects.
let mut visited = 0u64;
let tick = |visited: &mut u64| {
let result = progress.step(*visited, HeapPhase::Linking, (*visited / 2).min(objects), objects);
*visited += 1;
result
};
// Pass 2: node columns and out-degrees, counted into `out_start[v + 1]`.
let mut weak_edges_skipped = 0;
for segment in &segments {
scan_segment(buf, segment.clone(), id_size, &mut |at, record| {
let Some(node) = linker.node(at, &record)? else { return Ok(()) };
tick(&mut visited)?;
let v = node.index as usize;
types[v] = node.ty;
lens[v] = node.len;
offsets[v] = at as u64;
let mut degree = 0u64;
linker.edges(&record, &mut |_, weak| {
if weak {
weak_edges_skipped += 1;
} else {
degree += 1;
}
});
let count = &mut out_start[v + 1];
*count = u32::try_from(u64::from(*count) + degree).map_err(|_| HeapError::TooLarge)?;
Ok(())
})?;
}
out_start[nodes + 1] = u32::try_from(root_targets.len()).map_err(|_| HeapError::TooLarge)?;
let edges = accumulate(&mut out_start, cancel)?;
// Pass 3: `out_start[v]` is node v's write cursor; once filled it holds v's end, so the
// column is shifted one slot right to become the starts again.
let mut out = store.scratch::<u32>(edges as usize, 0)?;
for segment in &segments {
scan_segment(buf, segment.clone(), id_size, &mut |at, record| {
let Some(node) = linker.node(at, &record)? else { return Ok(()) };
tick(&mut visited)?;
let v = node.index as usize;
let mut cursor = out_start[v] as usize;
linker.edges(&record, &mut |target, weak| {
if !weak && let Some(slot) = out.get_mut(cursor) {
*slot = target;
cursor += 1;
}
});
out_start[v] = cursor as u32;
Ok(())
})?;
}
let first_root_edge = out_start[nodes] as usize;
for (slot, &target) in out[first_root_edge..].iter_mut().zip(&root_targets) {
*slot = target;
}
out_start[nodes] = edges;
out_start.copy_within(0..nodes + 1, 1);
out_start[0] = 0;
progress.report(HeapPhase::Linking, objects, objects)?;
drop(linker);
let ClassTable { classes, class_slot, .. } = table;
Ok(HeapGraph {
ids,
types,
lens,
offsets,
out_start,
out,
root: n,
roots,
classes,
class_slot,
thread_objects,
weak_edges_skipped,
id_size,
timestamp_ms: header.timestamp_ms,
})
}
/// Turns per-node counts into running totals in place (`counts[k]` becomes the sum of
/// `counts[..=k]`) and returns the last one; past `u32::MAX` the graph is `TooLarge`.
pub(crate) fn accumulate(counts: &mut [u32], cancel: &CancellationToken) -> Result<u32, HeapError> {
let mut sum = 0u32;
for chunk in counts.chunks_mut(CHECK_EVERY as usize) {
if cancel.is_cancelled() {
return Err(HeapError::Cancelled);
}
for count in chunk {
sum = sum.checked_add(*count).ok_or(HeapError::TooLarge)?;
*count = sum;
}
}
Ok(sum)
}
#[cfg(test)]
mod tests;
+188
Переглянути файл
@@ -0,0 +1,188 @@
//! Class dumps resolved into [`ClassInfo`]s, plus where the references sit in instance data.
use std::collections::HashMap;
use std::hash::BuildHasherDefault;
use super::{ClassInfo, PRIM_SLOTS};
use crate::column::NONE;
use crate::error::HeapError;
use crate::hprof::{BasicType, ClassDump};
use crate::ids::ChunkHasher;
/// Keyed by hprof id (an address).
pub(super) type IdHash<V> = HashMap<u64, V, BuildHasherDefault<ChunkHasher>>;
const REFERENCE: &str = "java.lang.ref.Reference";
const JAVA_LANG_CLASS: &str = "java.lang.Class";
/// Not computed yet, in [`ClassTable::fielded`].
const UNSET: u32 = NONE - 1;
pub(super) struct ClassTable {
pub(super) classes: Vec<ClassInfo>,
/// Class id → class index.
pub(super) by_id: IdHash<u32>,
pub(super) class_slot: u32,
/// Per class: `(offset, is the referent)` of each reference field in the class's own part of
/// the instance data.
pub(super) refs: Vec<Vec<(usize, bool)>>,
/// Per class: bytes of its own part of the instance data.
pub(super) own_bytes: Vec<usize>,
/// Per class: the nearest class up its chain (itself included) whose own part is not empty, or
/// `NONE`. Walking these instead of every super keeps a walk bounded by the data it reads.
pub(super) fielded: Vec<u32>,
}
/// The longest name a JVM gives a class or field (a constant pool UTF8 has a `u2` length).
const MAX_NAME_BYTES: usize = 65_535;
/// Name bytes a dump may decode beyond 4 per byte of the file.
const NAME_BUDGET_FLOOR: usize = 64 << 20;
/// Names decoded from the UTF8 records within a budget: every field of every class may name the
/// same record, so a small crafted file could otherwise ask for terabytes and abort the IDE.
struct Names<'m, 'b> {
records: &'m IdHash<&'b [u8]>,
left: usize,
}
impl Names<'_, '_> {
/// Empty for an id with no record.
fn text(&mut self, id: u64) -> Result<String, HeapError> {
let Some(bytes) = self.records.get(&id) else { return Ok(String::new()) };
if bytes.len() > MAX_NAME_BYTES || bytes.len() > self.left {
return Err(HeapError::NamesTooLarge);
}
self.left -= bytes.len();
Ok(String::from_utf8_lossy(bytes).into_owned())
}
}
/// `dumps` holds one dump per class id, and `by_id` its position; `names` are the UTF8 records
/// and `class_names` the LOAD CLASS class id → name id pairs. `file_len` bounds the name bytes.
pub(super) fn resolve(
dumps: Vec<ClassDump>,
by_id: IdHash<u32>,
names: &IdHash<&[u8]>,
class_names: &IdHash<u64>,
id_size: u8,
file_len: usize,
) -> Result<ClassTable, HeapError> {
// Room for the synthetic java.lang.Class and the primitive slots in a `u32` type id.
if dumps.len() > (u32::MAX - PRIM_SLOTS - 1) as usize {
return Err(HeapError::TooLarge);
}
let mut names = Names { records: names, left: file_len.saturating_mul(4).saturating_add(NAME_BUDGET_FLOOR) };
let mut supers: Vec<Option<u32>> =
dumps.iter().map(|d| if d.super_id == 0 { None } else { by_id.get(&d.super_id).copied() }).collect();
cut_super_cycles(&mut supers);
let mut classes = Vec::with_capacity(dumps.len() + 1);
let mut refs = Vec::with_capacity(dumps.len() + 1);
let mut own_bytes = Vec::with_capacity(dumps.len() + 1);
for (dump, super_index) in dumps.into_iter().zip(supers) {
let name = match class_names.get(&dump.id) {
Some(&name_id) if names.records.contains_key(&name_id) => names.text(name_id)?.replace('/', "."),
_ => format!("{:#x}", dump.id),
};
let fields =
dump.fields.iter().map(|&(n, ty)| Ok((names.text(n)?, ty))).collect::<Result<Vec<_>, HeapError>>()?;
let weak_field = if name == REFERENCE {
fields.iter().position(|(n, ty)| n == "referent" && *ty == BasicType::Object)
} else {
None
};
let mut offset = 0;
let mut class_refs = Vec::new();
for (k, &(_, ty)) in fields.iter().enumerate() {
if ty == BasicType::Object {
class_refs.push((offset, weak_field == Some(k)));
}
offset += ty.hprof_size(id_size);
}
refs.push(class_refs);
own_bytes.push(offset);
classes.push(ClassInfo {
id: dump.id,
name,
super_index,
fields,
statics: dump
.statics
.iter()
.map(|&(n, object)| Ok((names.text(n)?, object)))
.collect::<Result<_, HeapError>>()?,
static_bytes: dump.static_bytes,
weak_field,
});
}
let class_slot = match classes.iter().position(|c| c.name == JAVA_LANG_CLASS) {
Some(slot) => slot as u32,
None => {
classes.push(ClassInfo {
id: 0,
name: JAVA_LANG_CLASS.to_owned(),
super_index: None,
fields: Vec::new(),
statics: Vec::new(),
static_bytes: 0,
weak_field: None,
});
refs.push(Vec::new());
own_bytes.push(0);
classes.len() as u32 - 1
}
};
let fielded = nearest_fielded(&classes, &own_bytes);
Ok(ClassTable { classes, by_id, class_slot, refs, own_bytes, fielded })
}
/// A dump may claim `A extends B extends A`; the edge that closes a cycle is dropped so every
/// walk up the chain ends. Iterative, so a long chain cannot overflow the stack.
fn cut_super_cycles(supers: &mut [Option<u32>]) {
const NEW: u8 = 0;
const ON_PATH: u8 = 1;
const DONE: u8 = 2;
let mut state = vec![NEW; supers.len()];
let mut path = Vec::new();
for start in 0..supers.len() {
let mut c = start;
while state[c] == NEW {
state[c] = ON_PATH;
path.push(c);
match supers[c] {
Some(s) if state[s as usize] == ON_PATH => {
supers[c] = None;
break;
}
Some(s) => c = s as usize,
None => break,
}
}
for p in path.drain(..) {
state[p] = DONE;
}
}
}
fn nearest_fielded(classes: &[ClassInfo], own_bytes: &[usize]) -> Vec<u32> {
let mut fielded: Vec<u32> =
own_bytes.iter().enumerate().map(|(c, &b)| if b > 0 { c as u32 } else { UNSET }).collect();
let mut path = Vec::new();
for start in 0..classes.len() {
let mut c = start;
let found = loop {
if fielded[c] != UNSET {
break fielded[c];
}
path.push(c);
match classes[c].super_index {
Some(s) => c = s as usize,
None => break NONE,
}
};
for p in path.drain(..) {
fielded[p] = found;
}
}
fielded
}
+107
Переглянути файл
@@ -0,0 +1,107 @@
//! Turns one heap sub-record into its node (index, type, length) and its outgoing edges. Both
//! linking passes go through here, so the degrees counted in the first match the edges written
//! in the second.
use super::classes::ClassTable;
use crate::column::NONE;
use crate::error::HeapError;
use crate::hprof::{BasicType, HeapRecord};
use crate::ids::IdMap;
pub(super) struct Node {
pub(super) index: u32,
pub(super) ty: u32,
pub(super) len: u32,
}
pub(super) struct Linker<'a> {
ids: &'a IdMap,
table: &'a ClassTable,
/// Per class: the node of its class object, `NONE` for the synthetic `java.lang.Class`.
class_objects: Vec<u32>,
id_size: u8,
}
impl<'a> Linker<'a> {
pub(super) fn new(ids: &'a IdMap, table: &'a ClassTable, id_size: u8) -> Self {
let class_objects =
table.classes.iter().map(|c| if c.id == 0 { NONE } else { ids.rank(c.id).unwrap_or(NONE) }).collect();
Self { ids, table, class_objects, id_size }
}
/// `None` for root records. `at` is the record's offset, for the error of an instance or array
/// whose class is not in the dump.
pub(super) fn node(&self, at: usize, record: &HeapRecord<'_>) -> Result<Option<Node>, HeapError> {
let class_index = |class_id: u64| {
self.table.by_id.get(&class_id).copied().ok_or(HeapError::MissingClass { id: class_id, at: at as u64 })
};
let (id, ty, len) = match record {
HeapRecord::Root { .. } => return Ok(None),
HeapRecord::Class(c) => (c.id, self.table.class_slot, 0),
HeapRecord::Instance { id, class_id, .. } => (*id, class_index(*class_id)?, 0),
HeapRecord::ObjectArray { id, class_id, len, .. } => (*id, class_index(*class_id)?, *len),
HeapRecord::PrimitiveArray { id, ty, len, .. } => {
let prim_base = self.table.classes.len() as u32;
(*id, prim_base + (*ty as u32 - BasicType::Boolean as u32), *len)
}
};
Ok(self.ids.rank(id).map(|index| Node { index, ty, len }))
}
/// Calls `edge(target, is_referent)` for every reference to an object in the dump, in field
/// order; `is_referent` marks `Reference.referent`, which callers leave out of the graph.
pub(super) fn edges(&self, record: &HeapRecord<'_>, edge: &mut impl FnMut(u32, bool)) {
let mut to = |id: u64, weak: bool| {
if let Some(target) = self.target(id) {
edge(target, weak);
}
};
match record {
HeapRecord::Root { .. } | HeapRecord::PrimitiveArray { .. } => {}
HeapRecord::Class(c) => {
to(c.super_id, false);
to(c.loader_id, false);
for &(_, object) in &c.statics {
to(object, false);
}
}
HeapRecord::ObjectArray { data, .. } => {
for item in data.chunks_exact(usize::from(self.id_size)) {
to(read_id(item, 0, self.id_size).unwrap_or(0), false);
}
}
HeapRecord::Instance { class_id, data, .. } => {
let Some(&class) = self.table.by_id.get(class_id) else { return };
let mut base = 0;
let mut next = self.table.fielded[class as usize];
'chain: while next != NONE && base < data.len() {
let c = next as usize;
for &(offset, weak) in &self.table.refs[c] {
let Some(id) = read_id(data, base + offset, self.id_size) else { break 'chain };
to(id, weak);
}
base += self.table.own_bytes[c];
next = self.table.classes[c].super_index.map_or(NONE, |s| self.table.fielded[s as usize]);
}
let object = self.class_objects[class as usize];
if object != NONE {
edge(object, false);
}
}
}
}
/// Id 0 is null; ids not in the dump have no node.
fn target(&self, id: u64) -> Option<u32> {
if id == 0 { None } else { self.ids.rank(id) }
}
}
/// The big-endian id at `at`, or `None` when `data` ends first.
fn read_id(data: &[u8], at: usize, id_size: u8) -> Option<u64> {
let bytes = data.get(at..at.checked_add(usize::from(id_size))?)?;
Some(match *bytes {
[a, b, c, d] => u64::from(u32::from_be_bytes([a, b, c, d])),
_ => u64::from_be_bytes(bytes.try_into().ok()?),
})
}
+479
Переглянути файл
@@ -0,0 +1,479 @@
use std::cell::RefCell;
use proptest::prelude::*;
use super::*;
use crate::hprof::{HeapRecord, record_at};
use crate::testing::{HprofWriter, MiniHeap, sample};
const H: u64 = 0x1000;
const N1: u64 = 0x1100;
const N2: u64 = 0x1200;
const N3: u64 = 0x1300;
const L: u64 = 0x1400;
const R: u64 = 0x1500;
const W: u64 = 0x1600;
const N4: u64 = 0x1700;
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn build(bytes: &[u8]) -> Result<HeapGraph, HeapError> {
build_graph(bytes, &Store::Ram, &no_progress, &CancellationToken::new())
}
fn index(g: &HeapGraph, id: u64) -> u32 {
g.ids.rank(id).unwrap_or_else(|| panic!("{id:#x} is not in the graph"))
}
fn class(g: &HeapGraph, name: &str) -> (u32, u64) {
let slot = g.classes.iter().position(|c| c.name == name).unwrap_or_else(|| panic!("no class {name}"));
(slot as u32, g.classes[slot].id)
}
/// Targets of `id`'s edges, as ids, in edge order.
fn out_of(g: &HeapGraph, id: u64) -> Vec<u64> {
g.out_edges(index(g, id)).iter().map(|&t| g.ids.id_at(t)).collect()
}
fn root_targets(g: &HeapGraph) -> Vec<u64> {
g.out_edges(g.root).iter().map(|&t| g.ids.id_at(t)).collect()
}
fn assert_csr(g: &HeapGraph) {
let n = g.ids.len();
assert_eq!(g.root, n);
assert_eq!(g.out_start.len(), n as usize + 2);
assert_eq!(g.out_start[0], 0);
assert!(g.out_start.windows(2).all(|w| w[0] <= w[1]), "out_start is not monotone");
assert_eq!(g.out_start[n as usize + 1] as usize, g.out.len());
assert!(g.out.iter().all(|&t| t < n), "edge to a node outside the objects");
assert_eq!((g.types.len(), g.lens.len(), g.offsets.len()), (n as usize, n as usize, n as usize));
let slots = g.classes.len() as u32 + PRIM_SLOTS;
assert!(g.types.iter().all(|&t| t < slots), "type outside the slots");
for c in &g.classes {
let mut steps = 0;
let mut cursor = c.super_index;
while let Some(s) = cursor {
steps += 1;
assert!(steps <= g.classes.len(), "super chain of {} does not end", c.name);
cursor = g.classes[s as usize].super_index;
}
}
}
#[test]
fn edges_follow_fields_and_skip_weak_referents() {
let g = build(&sample().bytes()).unwrap();
assert_csr(&g);
let (_, holder) = class(&g, "Holder");
let (_, node) = class(&g, "Node");
let (_, weak) = class(&g, "java.lang.ref.WeakReference");
assert_eq!(out_of(&g, H), [N1, N2, holder]);
assert_eq!(out_of(&g, N1), [N3, node]);
assert_eq!(out_of(&g, N3), [L, node]);
assert_eq!(out_of(&g, W), [weak]);
assert_eq!(out_of(&g, N4), [node]);
assert_eq!(out_of(&g, L), Vec::<u64>::new());
assert_eq!(g.weak_edges_skipped, 1);
let (reference, _) = class(&g, "java.lang.ref.Reference");
assert_eq!(g.classes[reference as usize].weak_field, Some(0));
assert!(g.classes.iter().filter(|c| c.name != "java.lang.ref.Reference").all(|c| c.weak_field.is_none()));
}
#[test]
fn object_arrays_skip_null_and_unknown() {
let mut heap = sample();
heap.array_of(0x1800, "[Ljava.lang.Object;", &[N2, 0, 0x9990, N1, 0xDEAD_BEEF]);
let g = build(&heap.bytes()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, R), [N1]);
assert_eq!(out_of(&g, 0x1800), [N2, N1]);
}
#[test]
fn super_root_points_at_each_root_once() {
let mut heap = sample();
heap.root(HeapRootKind::JniGlobal, H).root(HeapRootKind::MonitorUsed, H).root(HeapRootKind::JniLocal, 0x9990);
let g = build(&heap.bytes()).unwrap();
assert_csr(&g);
let targets = root_targets(&g);
assert_eq!(targets.iter().filter(|&&t| t == H).count(), 1);
assert!(targets.contains(&N3) && targets.contains(&W));
assert!(!targets.contains(&N1) && !targets.contains(&R) && !targets.contains(&N4));
let classes = g.classes.iter().filter(|c| c.id != 0).count();
assert_eq!(targets.len(), 3 + classes, "H, N3, W and every sticky class");
let mut sorted = targets.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), targets.len());
let h = index(&g, H);
let of_h: Vec<_> = g.roots.iter().filter(|r| r.0 == h).map(|r| (r.1, r.2)).collect();
assert_eq!(of_h, [(HeapRootKind::JniGlobal, None), (HeapRootKind::MonitorUsed, None)]);
assert!(g.roots.contains(&(index(&g, N3), HeapRootKind::JavaFrame, Some(1))));
assert!(g.roots.iter().all(|r| r.0 < g.root));
}
#[test]
fn class_dump_edges_cover_statics() {
let mut w = HprofWriter::new();
w.string(1, "java/lang/Object")
.load_class(0x100, 1)
.class(0x100, 0, &[], &[])
.string(2, "Config")
.string(3, "INSTANCE")
.string(4, "EMPTY")
.string(5, "LOST")
.load_class(0x200, 2)
.class_with_loader(0x200, 0x100, 0x1200, &[], &[(3, N1), (4, 0), (5, 0x9990)])
.instance(N1, 0x100, &[])
.instance(0x1200, 0x100, &[]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x200), [0x100, 0x1200, N1]);
assert_eq!(out_of(&g, 0x100), Vec::<u64>::new());
let (config, _) = class(&g, "Config");
let info = &g.classes[config as usize];
let statics: Vec<_> = info.statics.iter().map(|(n, id)| (n.as_str(), *id)).collect();
assert_eq!(statics, [("INSTANCE", N1), ("EMPTY", 0), ("LOST", 0x9990)]);
assert_eq!(info.super_index, Some(class(&g, "java.lang.Object").0));
}
#[test]
fn missing_class_is_an_error() {
let mut w = HprofWriter::new();
w.string(1, "java/lang/Object").load_class(0x100, 1).class(0x100, 0, &[], &[]).instance(N1, 0x999, &[]);
let bytes = w.finish();
let at = instance_offset(&bytes, N1);
let err = build(&bytes).err().unwrap();
assert!(matches!(err, HeapError::MissingClass { id: 0x999, at: a } if a == at as u64), "{err:?}");
let mut w = HprofWriter::new();
w.obj_array(0x10, 0x999, &[]);
assert!(matches!(build(&w.finish()), Err(HeapError::MissingClass { id: 0x999, .. })));
}
fn instance_offset(bytes: &[u8], id: u64) -> usize {
let header = crate::hprof::parse_header(bytes).unwrap();
let mut found = None;
crate::hprof::scan(bytes, &header, &mut |r| {
if let crate::hprof::Record::Heap { at, record: HeapRecord::Instance { id: i, .. } } = r
&& i == id
{
found = Some(at);
}
Ok(())
})
.unwrap();
found.unwrap()
}
#[test]
fn cancel_during_linking_returns_cancelled() {
let bytes = sample().bytes();
let cancelled = CancellationToken::new();
cancelled.cancel();
let result = build_graph(&bytes, &Store::Ram, &no_progress, &cancelled);
assert!(matches!(result, Err(HeapError::Cancelled)));
let cancel = CancellationToken::new();
let phases = RefCell::new(Vec::new());
let result = build_graph(
&bytes,
&Store::Ram,
&|phase, done, total| {
phases.borrow_mut().push((phase, done, total));
if phase == HeapPhase::Linking {
cancel.cancel();
}
Ok(())
},
&cancel,
);
assert!(matches!(result, Err(HeapError::Cancelled)));
let phases = phases.into_inner();
assert_eq!(phases.first(), Some(&(HeapPhase::Reading, 0, bytes.len() as u64)));
assert_eq!(phases.last(), Some(&(HeapPhase::Linking, 0, 15)), "stops at the first Linking report");
}
#[test]
fn progress_error_stops_the_build() {
let bytes = sample().bytes();
let fail = |phase, _, total| {
if phase == HeapPhase::Linking {
Err(HeapError::NotEnoughDiskSpace { needed: total * 90, available: 0 })
} else {
Ok(())
}
};
let result = build_graph(&bytes, &Store::Ram, &fail, &CancellationToken::new());
assert!(matches!(result, Err(HeapError::NotEnoughDiskSpace { needed: 1350, available: 0 })));
}
#[test]
fn progress_reads_bytes_then_links_objects() {
let bytes = sample().bytes();
let phases = RefCell::new(Vec::new());
let g = build_graph(
&bytes,
&Store::Ram,
&|phase, done, total| {
phases.borrow_mut().push((phase, done, total));
Ok(())
},
&CancellationToken::new(),
)
.unwrap();
let phases = phases.into_inner();
let len = bytes.len() as u64;
let objects = u64::from(g.root);
assert_eq!(objects, 15, "7 classes and 8 objects");
assert_eq!(phases.first(), Some(&(HeapPhase::Reading, 0, len)));
assert!(phases.contains(&(HeapPhase::Reading, len, len)));
let linking: Vec<_> = phases.iter().filter(|p| p.0 == HeapPhase::Linking).collect();
assert_eq!(linking.first(), Some(&&(HeapPhase::Linking, 0, objects)));
assert_eq!(linking.iter().filter(|p| ***p == (HeapPhase::Linking, 0, objects)).count(), 1, "{linking:?}");
assert_eq!(linking.last(), Some(&&(HeapPhase::Linking, objects, objects)));
assert!(phases.iter().all(|&(_, done, total)| done <= total));
}
#[test]
fn types_lengths_and_offsets() {
let bytes = sample().bytes();
let g = build(&bytes).unwrap();
let (holder, holder_id) = class(&g, "Holder");
let (node, _) = class(&g, "Node");
let (array, _) = class(&g, "[Ljava.lang.Object;");
let (java_lang_class, _) = class(&g, "java.lang.Class");
assert_eq!(g.class_slot, java_lang_class);
let ty = |id| g.types[index(&g, id) as usize];
let len = |id| g.lens[index(&g, id) as usize];
assert_eq!([ty(H), ty(N1), ty(R), ty(holder_id)], [holder, node, array, java_lang_class]);
assert_eq!(ty(L), g.prim_slot(BasicType::Long));
assert_eq!(g.prim_type(ty(L)), Some(BasicType::Long));
assert_eq!(g.prim_type(holder), None);
assert_eq!(g.prim_type(g.classes.len() as u32 + PRIM_SLOTS), None);
assert_eq!(g.prim_slot(BasicType::Boolean), g.classes.len() as u32);
assert_eq!(g.prim_slot(BasicType::Long), g.classes.len() as u32 + PRIM_SLOTS - 1);
assert_eq!([len(H), len(L), len(R), len(holder_id)], [0, 2, 3, 0]);
for i in 0..g.root {
let id = match record_at(&bytes, g.offsets[i as usize] as usize, g.id_size).unwrap() {
HeapRecord::Class(c) => c.id,
HeapRecord::Instance { id, .. } | HeapRecord::ObjectArray { id, .. } => id,
HeapRecord::PrimitiveArray { id, .. } => id,
other => panic!("offset of object {i} points at {other:?}"),
};
assert_eq!(id, g.ids.id_at(i));
}
assert_eq!((g.id_size, g.timestamp_ms), (8, 1_700_000_000_000));
let holder_info = &g.classes[holder as usize];
let fields: Vec<_> = holder_info.fields.iter().map(|(n, t)| (n.as_str(), *t)).collect();
assert_eq!(fields, [("a", BasicType::Object), ("b", BasicType::Object)]);
assert_eq!(holder_info.super_index, Some(class(&g, "java.lang.Object").0));
assert_eq!(g.classes[class(&g, "java.lang.Object").0 as usize].super_index, None);
}
#[test]
fn dump_without_java_lang_class_gets_a_synthetic_slot() {
let mut w = HprofWriter::new();
w.string(1, "java/lang/Object").load_class(0x100, 1).class(0x100, 0, &[], &[]).instance(N1, 0x100, &[]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(g.classes.len(), 2);
let synthetic = &g.classes[g.class_slot as usize];
assert_eq!((synthetic.id, synthetic.name.as_str(), synthetic.super_index), (0, "java.lang.Class", None));
assert_eq!(g.types[index(&g, 0x100) as usize], g.class_slot);
assert_eq!(g.types[index(&g, N1) as usize], 0);
}
#[test]
fn classes_without_a_name_record_get_their_address() {
let mut w = HprofWriter::new();
w.class(0x100, 0, &[(9, BasicType::Object)], &[])
.instance(N1, 0x100, &N2.to_be_bytes())
.instance(N2, 0x100, &[0; 8]);
let g = build(&w.finish()).unwrap();
assert_eq!(g.classes[0].name, "0x100");
assert_eq!(g.classes[0].fields, [(String::new(), BasicType::Object)]);
assert_eq!(out_of(&g, N1), [N2, 0x100]);
}
#[test]
fn thread_objects_map_serials_to_threads() {
let mut heap = sample();
heap.class("java.lang.Thread", Some("java.lang.Object"), &[])
.object(0x2000, "java.lang.Thread", &[])
.object(0x2100, "java.lang.Thread", &[])
.root_in_thread(HeapRootKind::ThreadObject, 0x2000, 1)
.root_in_thread(HeapRootKind::ThreadObject, 0x2100, 7)
.root_in_thread(HeapRootKind::ThreadObject, 0x9990, 9);
let g = build(&heap.bytes()).unwrap();
let expected: HashMap<u32, u32> = [(1, index(&g, 0x2000)), (7, index(&g, 0x2100))].into();
assert_eq!(g.thread_objects, expected);
assert!(g.roots.contains(&(index(&g, 0x2100), HeapRootKind::ThreadObject, Some(7))));
}
#[test]
fn inherited_fields_follow_the_class_first_layout() {
let mut heap = MiniHeap::new();
heap.class("A", Some("java.lang.Object"), &[("a", BasicType::Object), ("n", BasicType::Int)])
.class("Empty", Some("A"), &[])
.class("B", Some("Empty"), &[("flag", BasicType::Byte), ("b", BasicType::Object)])
.object(0x10, "B", &[("a", 0x20), ("b", 0x30), ("flag", 1)])
.object(0x20, "A", &[])
.object(0x30, "A", &[]);
let g = build(&heap.bytes()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x10), [0x30, 0x20, class(&g, "B").1]);
}
#[test]
fn super_class_cycles_are_cut() {
let mut w = HprofWriter::new();
w.string(1, "A")
.string(2, "B")
.string(3, "S")
.string(4, "f")
.load_class(0x100, 1)
.load_class(0x200, 2)
.load_class(0x300, 3)
.class(0x100, 0x200, &[(4, BasicType::Object)], &[])
.class(0x200, 0x100, &[(4, BasicType::Object)], &[])
.class(0x300, 0x300, &[], &[])
.instance(0x10, 0x100, &[[0; 7].as_slice(), &[0x20], &[0; 7], &[0x30]].concat())
.instance(0x20, 0x300, &[])
.instance(0x30, 0x300, &[0; 64]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x10), [0x20, 0x30, 0x100]);
assert_eq!(out_of(&g, 0x30), [0x300]);
}
#[test]
fn short_instance_data_stops_at_the_last_whole_field() {
let mut w = HprofWriter::new();
w.string(1, "P")
.string(2, "x")
.load_class(0x100, 1)
.class(0x100, 0, &[(2, BasicType::Object), (2, BasicType::Object)], &[])
.instance(0x10, 0x100, &[[0; 7].as_slice(), &[0x20], &[0; 5]].concat())
.instance(0x20, 0x100, &[]);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(out_of(&g, 0x10), [0x20, 0x100]);
}
#[test]
fn id_size_4_dump_links() {
let mut w = HprofWriter::with_id_size(4);
w.string(1, "Node")
.string(2, "next")
.load_class(0x100, 1)
.class(0x100, 0, &[(2, BasicType::Object)], &[(2, 0x20)])
.instance(0x10, 0x100, &0x20u32.to_be_bytes())
.instance(0x20, 0x100, &[0; 4])
.obj_array(0x30, 0x100, &[0x10, 0, 0x20])
.root(HeapRootKind::JniGlobal, 0x30);
let g = build(&w.finish()).unwrap();
assert_csr(&g);
assert_eq!(g.id_size, 4);
assert_eq!(out_of(&g, 0x10), [0x20, 0x100]);
assert_eq!(out_of(&g, 0x30), [0x10, 0x20]);
assert_eq!(out_of(&g, 0x100), [0x20]);
assert_eq!(root_targets(&g), [0x30]);
}
#[test]
fn dir_store_keeps_types_and_offsets_only() {
let dir = tempfile::tempdir().unwrap();
// The id map's own files (`ids.*`) depend on its layout; MiniHeap's class ids are unaligned.
let names = || {
let mut names: Vec<_> = std::fs::read_dir(dir.path())
.unwrap()
.map(|e| e.unwrap().file_name().into_string().unwrap())
.filter(|name| !name.starts_with("ids."))
.collect();
names.sort();
names
};
let g =
build_graph(&sample().bytes(), &Store::Dir(dir.path().to_path_buf()), &no_progress, &CancellationToken::new())
.unwrap();
assert_eq!(out_of(&g, H).len(), 3);
let kept = ["offsets.u64", "types.u32"];
assert!(dir.path().join("ids.sorted.u64").exists());
#[cfg(unix)]
assert_eq!(names(), kept);
drop(g);
assert_eq!(names(), kept);
}
#[test]
fn empty_dump_has_only_the_super_root() {
let g = build(&HprofWriter::new().finish()).unwrap();
assert_csr(&g);
assert_eq!((g.root, g.out.len(), g.classes.len()), (0, 0, 1));
assert_eq!(g.class_slot, 0);
}
#[test]
fn edge_count_past_u32_is_too_large() {
let cancel = CancellationToken::new();
let mut counts = [0, 2, 0, 3, 1];
assert_eq!(accumulate(&mut counts, &cancel).unwrap(), 6);
assert_eq!(counts, [0, 2, 2, 5, 6]);
let mut full = [0, u32::MAX - 1, 1];
assert_eq!(accumulate(&mut full, &cancel).unwrap(), u32::MAX);
let mut over = [0, u32::MAX, 1];
assert!(matches!(accumulate(&mut over, &cancel), Err(HeapError::TooLarge)));
cancel.cancel();
assert!(matches!(accumulate(&mut [0, 1], &cancel), Err(HeapError::Cancelled)));
}
/// A class with `fields` instance fields, every one named by the same `name_len`-byte UTF8 record.
fn shared_long_field_names(name_len: usize, fields: usize) -> Vec<u8> {
let mut w = HprofWriter::new();
let long = "f".repeat(name_len);
w.string(1, "java/lang/Object").load_class(0x100, 1).class(0x100, 0, &[], &[]);
w.string(2, "Wide").string(3, &long).load_class(0x200, 2);
w.class(0x200, 0x100, &vec![(3, BasicType::Int); fields], &[]);
w.finish()
}
#[test]
fn names_that_would_decode_far_larger_than_the_file_are_refused() {
// 128 MiB of names from a file of about 70 KB.
let bytes = shared_long_field_names(32 << 10, 4000);
assert!(bytes.len() < 100 << 10, "{}", bytes.len());
let err = build(&bytes).err().expect("refused");
assert!(matches!(err, HeapError::NamesTooLarge), "{err:?}");
}
#[test]
fn a_name_longer_than_the_jvm_allows_is_refused() {
let mut w = HprofWriter::new();
w.string(1, &"C".repeat(70_000)).load_class(0x100, 1).class(0x100, 0, &[], &[]);
assert!(matches!(build(&w.finish()), Err(HeapError::NamesTooLarge)));
}
#[test]
fn real_sized_names_shared_by_many_fields_are_fine() {
let g = build(&shared_long_field_names(40, 4000)).unwrap();
assert_eq!(g.classes[class(&g, "Wide").0 as usize].fields.len(), 4000);
}
proptest! {
#[test]
fn build_graph_never_panics_on_mutated_dumps(flips in proptest::collection::vec((0usize..4096, any::<u8>()), 1..12)) {
let mut bytes = sample().bytes();
for (pos, value) in flips {
let at = pos % bytes.len();
bytes[at] = value;
}
if let Ok(g) = build(&bytes) {
assert_csr(&g);
}
}
}
+945
Переглянути файл
@@ -0,0 +1,945 @@
//! Reader for HotSpot `.hprof` heap dumps (format 1.0.1 / 1.0.2, 4- or 8-byte ids).
use std::ops::Range;
use serde::{Deserialize, Serialize};
use crate::error::HeapError;
use crate::reader::Reader;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[repr(u8)]
pub enum BasicType {
Object = 2,
Boolean = 4,
Char = 5,
Float = 6,
Double = 7,
Byte = 8,
Short = 9,
Int = 10,
Long = 11,
}
impl BasicType {
pub fn from_u8(v: u8) -> Option<Self> {
Some(match v {
2 => Self::Object,
4 => Self::Boolean,
5 => Self::Char,
6 => Self::Float,
7 => Self::Double,
8 => Self::Byte,
9 => Self::Short,
10 => Self::Int,
11 => Self::Long,
_ => return None,
})
}
/// Bytes one value takes in the dump; `id_size` is the dump's identifier size.
pub fn hprof_size(self, id_size: u8) -> usize {
match self {
Self::Object => usize::from(id_size),
Self::Boolean | Self::Byte => 1,
Self::Char | Self::Short => 2,
Self::Float | Self::Int => 4,
Self::Double | Self::Long => 8,
}
}
/// Bytes one value takes in the JVM heap; `ref_size` is 4 with compressed oops, else 8.
pub fn heap_size(self, ref_size: u8) -> u64 {
match self {
Self::Object => u64::from(ref_size),
other => other.hprof_size(ref_size) as u64,
}
}
/// JVM class name of an array of this type (`[I`); `Object` arrays have no single name, so
/// they report the most general one.
pub fn array_name(self) -> &'static str {
match self {
Self::Object => "[Ljava.lang.Object;",
Self::Boolean => "[Z",
Self::Char => "[C",
Self::Float => "[F",
Self::Double => "[D",
Self::Byte => "[B",
Self::Short => "[S",
Self::Int => "[I",
Self::Long => "[J",
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub enum HeapRootKind {
Unknown,
JniGlobal,
JniLocal,
JavaFrame,
NativeStack,
StickyClass,
ThreadBlock,
MonitorUsed,
ThreadObject,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct HprofHeader {
pub version: String,
pub id_size: u8,
pub timestamp_ms: u64,
/// Offset of the first record.
pub body_start: usize,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ClassDump {
pub id: u64,
pub super_id: u64,
pub loader_id: u64,
/// `(name id, object id)` of reference statics; a null static has object id 0.
pub statics: Vec<(u64, u64)>,
/// Heap bytes taken by the primitive statics.
pub static_bytes: u64,
/// Instance fields in dump order, `(name id, type)`.
pub fields: Vec<(u64, BasicType)>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum HeapRecord<'a> {
/// `thread_serial` is present for the kinds that carry one.
Root {
kind: HeapRootKind,
id: u64,
thread_serial: Option<u32>,
},
Class(ClassDump),
/// `data` is the field values, class first, then each superclass.
Instance {
id: u64,
class_id: u64,
data: &'a [u8],
},
/// `data` is `len` ids.
ObjectArray {
id: u64,
class_id: u64,
len: u32,
data: &'a [u8],
},
/// `data` is `len` big-endian elements.
PrimitiveArray {
id: u64,
ty: BasicType,
len: u32,
data: &'a [u8],
},
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Record<'a> {
Utf8 {
id: u64,
text: &'a [u8],
},
LoadClass {
class_id: u64,
name_id: u64,
},
/// Byte range of a heap-dump segment's sub-records; its contents follow as `Heap` records.
Segment(Range<usize>),
/// `at` is the offset of the sub-record's tag, for [`record_at`].
Heap {
at: usize,
record: HeapRecord<'a>,
},
}
const MAGIC_1_0_2: &[u8] = b"JAVA PROFILE 1.0.2\0";
const MAGIC_1_0_1: &[u8] = b"JAVA PROFILE 1.0.1\0";
const TAG_UTF8: u8 = 0x01;
const TAG_LOAD_CLASS: u8 = 0x02;
const TAG_HEAP_DUMP: u8 = 0x0C;
const TAG_HEAP_DUMP_SEGMENT: u8 = 0x1C;
const TAG_HEAP_DUMP_END: u8 = 0x2C;
/// Tag (u1), timestamp (u4) and body length (u4).
const RECORD_HEADER_LEN: usize = 9;
fn check_id_size(id_size: u8) -> Result<(), HeapError> {
match id_size {
4 | 8 => Ok(()),
other => Err(HeapError::UnsupportedIdSize(u32::from(other))),
}
}
fn truncated(buf: &[u8]) -> HeapError {
HeapError::Truncated { at: buf.len() as u64 }
}
pub fn parse_header(buf: &[u8]) -> Result<HprofHeader, HeapError> {
let magic =
[MAGIC_1_0_2, MAGIC_1_0_1].into_iter().find(|magic| buf.starts_with(magic)).ok_or(HeapError::NotHprof)?;
let mut r = Reader::new(buf, magic.len(), 8);
let id_size = r.u4()?;
let id_size = u8::try_from(id_size).map_err(|_| HeapError::UnsupportedIdSize(id_size))?;
check_id_size(id_size)?;
let timestamp_ms = r.u8()?;
let version = String::from_utf8_lossy(&magic[..magic.len() - 1]).into_owned();
Ok(HprofHeader { version, id_size, timestamp_ms, body_start: r.pos() })
}
/// Walks every top-level record and, for heap-dump segments, the sub-records inside them.
/// A record whose length runs past the end of `buf` is `Truncated`, and so are segments that no
/// HEAP DUMP END follows: HotSpot writes whole segments, so a dump it did not finish ends after one.
pub fn scan<'a>(
buf: &'a [u8],
header: &HprofHeader,
visit: &mut dyn FnMut(Record<'a>) -> Result<(), HeapError>,
) -> Result<(), HeapError> {
let id_size = header.id_size;
check_id_size(id_size)?;
if header.body_start > buf.len() {
return Err(truncated(buf));
}
let mut pos = header.body_start;
let mut unended_segments = false;
while pos < buf.len() {
let mut head = Reader::new(buf, pos, id_size);
let tag = head.u1()?;
head.skip(4)?; // timestamp
let len = head.u4()? as usize;
let start = pos + RECORD_HEADER_LEN;
let end = start.checked_add(len).filter(|&end| end <= buf.len()).ok_or_else(|| truncated(buf))?;
let body = &buf[..end];
let mut r = Reader::new(body, start, id_size);
match tag {
TAG_UTF8 => {
let id = r.id()?;
let text = r.bytes(end - r.pos())?;
visit(Record::Utf8 { id, text })?;
}
TAG_LOAD_CLASS => {
r.skip(4)?; // class serial
let class_id = r.id()?;
r.skip(4)?; // stack trace serial
let name_id = r.id()?;
visit(Record::LoadClass { class_id, name_id })?;
}
TAG_HEAP_DUMP | TAG_HEAP_DUMP_SEGMENT => {
// A single HEAP DUMP (JDK 8, under 2 GB) has no end record.
unended_segments |= tag == TAG_HEAP_DUMP_SEGMENT;
visit(Record::Segment(start..end))?;
walk_segment(buf, start..end, id_size, &mut |at, record| visit(Record::Heap { at, record }))?;
}
TAG_HEAP_DUMP_END => unended_segments = false,
_ => {}
}
pos = end;
}
if unended_segments {
return Err(truncated(buf));
}
Ok(())
}
/// Visits the sub-records of one segment (the range from [`Record::Segment`]) with their offsets.
/// The last sub-record must end exactly at the segment's end.
pub fn scan_segment<'a>(
buf: &'a [u8],
segment: Range<usize>,
id_size: u8,
visit: &mut dyn FnMut(usize, HeapRecord<'a>) -> Result<(), HeapError>,
) -> Result<(), HeapError> {
check_id_size(id_size)?;
walk_segment(buf, segment, id_size, visit)
}
fn walk_segment<'a>(
buf: &'a [u8],
segment: Range<usize>,
id_size: u8,
visit: &mut dyn FnMut(usize, HeapRecord<'a>) -> Result<(), HeapError>,
) -> Result<(), HeapError> {
if segment.start > segment.end || segment.end > buf.len() {
return Err(truncated(buf));
}
let mut r = Reader::new(&buf[..segment.end], segment.start, id_size);
while r.pos() < segment.end {
let at = r.pos();
let record = read_sub_record(&mut r)?;
visit(at, record)?;
}
Ok(())
}
/// Reads the sub-record whose tag is at `at`, as given by [`Record::Heap`].
pub fn record_at(buf: &[u8], at: usize, id_size: u8) -> Result<HeapRecord<'_>, HeapError> {
check_id_size(id_size)?;
read_sub_record(&mut Reader::new(buf, at, id_size))
}
fn read_sub_record<'a>(r: &mut Reader<'a>) -> Result<HeapRecord<'a>, HeapError> {
let at = r.pos();
let tag = r.u1()?;
// (kind, carries a thread serial, bytes after it)
let root = match tag {
0xFF => Some((HeapRootKind::Unknown, false, 0)),
0x01 => Some((HeapRootKind::JniGlobal, false, usize::from(r.id_size()))),
0x02 => Some((HeapRootKind::JniLocal, true, 4)),
0x03 => Some((HeapRootKind::JavaFrame, true, 4)),
0x04 => Some((HeapRootKind::NativeStack, true, 0)),
0x05 => Some((HeapRootKind::StickyClass, false, 0)),
0x06 => Some((HeapRootKind::ThreadBlock, true, 0)),
0x07 => Some((HeapRootKind::MonitorUsed, false, 0)),
0x08 => Some((HeapRootKind::ThreadObject, true, 4)),
_ => None,
};
if let Some((kind, has_serial, trailing)) = root {
let id = r.id()?;
let thread_serial = if has_serial { Some(r.u4()?) } else { None };
r.skip(trailing)?;
return Ok(HeapRecord::Root { kind, id, thread_serial });
}
match tag {
0x20 => read_class_dump(r).map(HeapRecord::Class),
0x21 => {
let id = r.id()?;
r.skip(4)?; // stack trace serial
let class_id = r.id()?;
let len = r.u4()? as usize;
Ok(HeapRecord::Instance { id, class_id, data: r.bytes(len)? })
}
0x22 => {
let id = r.id()?;
r.skip(4)?;
let len = r.u4()?;
let class_id = r.id()?;
let size = (len as usize).checked_mul(usize::from(r.id_size())).ok_or_else(|| truncated(r.buf()))?;
Ok(HeapRecord::ObjectArray { id, class_id, len, data: r.bytes(size)? })
}
0x23 => {
let id = r.id()?;
r.skip(4)?;
let len = r.u4()?;
let ty = read_type(r)?;
if ty == BasicType::Object {
return Err(HeapError::UnknownType { ty: ty as u8, at: (r.pos() - 1) as u64 });
}
let size = (len as usize).checked_mul(ty.hprof_size(r.id_size())).ok_or_else(|| truncated(r.buf()))?;
Ok(HeapRecord::PrimitiveArray { id, ty, len, data: r.bytes(size)? })
}
tag => Err(HeapError::UnknownSubRecord { tag, at: at as u64 }),
}
}
fn read_type(r: &mut Reader<'_>) -> Result<BasicType, HeapError> {
let at = r.pos();
let ty = r.u1()?;
BasicType::from_u8(ty).ok_or(HeapError::UnknownType { ty, at: at as u64 })
}
fn read_class_dump(r: &mut Reader<'_>) -> Result<ClassDump, HeapError> {
let id_size = r.id_size();
let id = r.id()?;
r.skip(4)?; // stack trace serial
let super_id = r.id()?;
let loader_id = r.id()?;
r.skip(4 * usize::from(id_size))?; // signers, protection domain, 2 reserved
r.skip(4)?; // instance size
for _ in 0..r.u2()? {
r.skip(2)?; // constant pool index
let ty = read_type(r)?;
r.skip(ty.hprof_size(id_size))?;
}
let mut statics = Vec::new();
let mut static_bytes = 0;
for _ in 0..r.u2()? {
let name = r.id()?;
let ty = read_type(r)?;
if ty == BasicType::Object {
statics.push((name, r.id()?));
} else {
r.skip(ty.hprof_size(id_size))?;
static_bytes += ty.heap_size(id_size);
}
}
let mut fields = Vec::new();
for _ in 0..r.u2()? {
let name = r.id()?;
fields.push((name, read_type(r)?));
}
Ok(ClassDump { id, super_id, loader_id, statics, static_bytes, fields })
}
#[cfg(test)]
mod tests {
use super::*;
use crate::testing::{HprofWriter, MiniHeap};
use proptest::prelude::*;
fn collect(bytes: &[u8]) -> Result<Vec<Record<'_>>, HeapError> {
let header = parse_header(bytes)?;
let mut out = Vec::new();
scan(bytes, &header, &mut |r| {
out.push(r);
Ok(())
})?;
Ok(out)
}
fn segment_of(bytes: &[u8]) -> Range<usize> {
collect(bytes)
.unwrap()
.into_iter()
.find_map(|r| if let Record::Segment(s) = r { Some(s) } else { None })
.expect("a segment")
}
fn all_kinds(w: &mut HprofWriter) {
w.string(1, "Node")
.string(2, "next")
.load_class(0x100, 1)
.class(0x100, 0, &[(2, BasicType::Object), (3, BasicType::Long)], &[(2, 0x10)])
.instance(0x10, 0x100, &[0u8; 16])
.obj_array(0x20, 0x100, &[0x10, 0])
.prim_array(0x30, BasicType::Int, 3)
.root(HeapRootKind::JavaFrame, 0x10)
.root(HeapRootKind::StickyClass, 0x100);
}
#[test]
fn parses_header() {
let header = parse_header(&MiniHeap::new().bytes()).unwrap();
assert_eq!((header.version.as_str(), header.id_size, header.body_start), ("JAVA PROFILE 1.0.2", 8, 31));
let mut old = b"JAVA PROFILE 1.0.1\0".to_vec();
old.extend_from_slice(&[0, 0, 0, 4, 0, 0, 0, 1, 0, 0, 0, 2]);
let header = parse_header(&old).unwrap();
assert_eq!(
(header.version.as_str(), header.id_size, header.timestamp_ms),
("JAVA PROFILE 1.0.1", 4, (1 << 32) + 2)
);
let bad_size = b"JAVA PROFILE 1.0.2\0\0\0\0\x03\0\0\0\0\0\0\0\0";
assert!(matches!(parse_header(bad_size), Err(HeapError::UnsupportedIdSize(3))));
assert!(matches!(parse_header(b"GIF89a"), Err(HeapError::NotHprof)));
assert!(matches!(parse_header(b""), Err(HeapError::NotHprof)));
assert!(matches!(parse_header(b"JAVA PROFILE 1.0.3\0\0\0\0\x08"), Err(HeapError::NotHprof)));
assert!(matches!(parse_header(b"JAVA PROFILE 1.0.2\0\0\0"), Err(HeapError::Truncated { at: 21 })));
}
#[test]
fn scans_every_record_kind() {
let mut w = HprofWriter::new();
all_kinds(&mut w);
let bytes = w.finish();
let records = collect(&bytes).unwrap();
let (mut utf8, mut loads, mut segments) = (0, 0, 0);
let (mut classes, mut instances, mut obj_arrays, mut prim_arrays, mut roots) = (0, 0, 0, 0, 0);
for record in &records {
match record {
Record::Utf8 { .. } => utf8 += 1,
Record::LoadClass { class_id, name_id } => {
loads += 1;
assert_eq!((*class_id, *name_id), (0x100, 1));
}
Record::Segment(_) => segments += 1,
Record::Heap { record, .. } => match record {
HeapRecord::Class(c) => {
classes += 1;
assert_eq!((c.id, c.super_id, c.loader_id), (0x100, 0, 0));
assert_eq!(c.statics, [(2, 0x10)]);
assert_eq!(c.fields, [(2, BasicType::Object), (3, BasicType::Long)]);
}
HeapRecord::Instance { id, class_id, data } => {
instances += 1;
assert_eq!((*id, *class_id, data.len()), (0x10, 0x100, 16));
}
HeapRecord::ObjectArray { id, class_id, len, data } => {
obj_arrays += 1;
assert_eq!((*id, *class_id, *len, data.len()), (0x20, 0x100, 2, 16));
}
HeapRecord::PrimitiveArray { id, ty, len, data } => {
prim_arrays += 1;
assert_eq!((*id, *ty, *len, data.len()), (0x30, BasicType::Int, 3, 12));
}
HeapRecord::Root { kind, id, thread_serial } => {
roots += 1;
match kind {
HeapRootKind::JavaFrame => assert_eq!((*id, *thread_serial), (0x10, Some(1))),
HeapRootKind::StickyClass => assert_eq!((*id, *thread_serial), (0x100, None)),
other => panic!("unexpected root {other:?}"),
}
}
},
}
}
assert_eq!((utf8, loads, segments), (2, 1, 1));
assert_eq!((classes, instances, obj_arrays, prim_arrays, roots), (1, 1, 1, 1, 2));
assert!(matches!(&records[3], Record::Segment(s) if s.end <= bytes.len()), "segment precedes its contents");
}
#[test]
fn root_kinds_carry_the_thread_serial_they_have() {
let kinds = [
(HeapRootKind::Unknown, false),
(HeapRootKind::JniGlobal, false),
(HeapRootKind::JniLocal, true),
(HeapRootKind::JavaFrame, true),
(HeapRootKind::NativeStack, true),
(HeapRootKind::StickyClass, false),
(HeapRootKind::ThreadBlock, true),
(HeapRootKind::MonitorUsed, false),
(HeapRootKind::ThreadObject, true),
];
let mut w = HprofWriter::new();
for (i, (kind, _)) in kinds.iter().enumerate() {
w.root_in_thread(*kind, 0x40 + i as u64, 7);
w.root(*kind, 0x80 + i as u64);
}
let bytes = w.finish();
let mut seen = Vec::new();
for r in collect(&bytes).unwrap() {
if let Record::Heap { record: HeapRecord::Root { kind, id, thread_serial }, .. } = r {
seen.push((kind, id, thread_serial));
}
}
assert_eq!(seen.len(), kinds.len() * 2);
for (i, (kind, carries)) in kinds.iter().enumerate() {
let explicit = seen[2 * i];
let default = seen[2 * i + 1];
assert_eq!(explicit, (*kind, 0x40 + i as u64, carries.then_some(7)));
assert_eq!(default, (*kind, 0x80 + i as u64, carries.then_some(1)));
}
}
#[test]
fn primitive_arrays_expose_their_contents() {
let mut w = HprofWriter::new();
w.prim_bytes(0x1, BasicType::Char, &[0, b'h', 0, 0xE9]).prim_bytes(0x2, BasicType::Byte, b"v42");
let bytes = w.finish();
let arrays: Vec<_> = collect(&bytes)
.unwrap()
.into_iter()
.filter_map(|r| match r {
Record::Heap { record: HeapRecord::PrimitiveArray { ty, len, data, .. }, .. } => {
Some((ty, len, data.to_vec()))
}
_ => None,
})
.collect();
assert_eq!(arrays, [(BasicType::Char, 2, vec![0, b'h', 0, 0xE9]), (BasicType::Byte, 3, b"v42".to_vec())]);
}
#[test]
fn mini_heap_builds_strings_and_defines_object_and_class() {
let mut heap = MiniHeap::new();
heap.string(0x10, "v42", true).string(0x20, "hé", false);
let bytes = heap.bytes();
let records = collect(&bytes).unwrap();
let names: Vec<_> = records
.iter()
.filter_map(|r| {
if let Record::Utf8 { text, .. } = r { Some(String::from_utf8_lossy(text).into_owned()) } else { None }
})
.collect();
assert!(names.iter().any(|n| n == "java/lang/Object") && names.iter().any(|n| n == "java/lang/Class"));
let arrays: Vec<_> = records
.iter()
.filter_map(|r| match r {
Record::Heap { record: HeapRecord::PrimitiveArray { data, .. }, .. } => Some(data.to_vec()),
_ => None,
})
.collect();
assert_eq!(arrays, [b"v42".to_vec(), vec![b'h', 0, 0xE9, 0]]);
let coders: Vec<_> = records
.iter()
.filter_map(|r| match r {
Record::Heap { record: HeapRecord::Instance { id, data, .. }, .. } => Some((*id, data[8])),
_ => None,
})
.collect();
assert_eq!(coders, [(0x10, 0), (0x20, 1)]);
}
#[test]
fn mini_heap_lays_out_fields_class_first_then_super() {
let mut heap = MiniHeap::new();
heap.class("A", Some("java.lang.Object"), &[("a", BasicType::Int)])
.class("B", Some("A"), &[("b", BasicType::Object), ("c", BasicType::Byte)])
.object(0x10, "B", &[("a", 7), ("b", 0x99), ("c", 0x1FF)]);
let bytes = heap.bytes();
let data = collect(&bytes)
.unwrap()
.into_iter()
.find_map(|r| match r {
Record::Heap { record: HeapRecord::Instance { data, .. }, .. } => Some(data.to_vec()),
_ => None,
})
.unwrap();
assert_eq!(data, [0, 0, 0, 0, 0, 0, 0, 0x99, 0xFF, 0, 0, 0, 7]);
}
#[test]
fn record_at_reads_back_an_instance() {
let mut w = HprofWriter::new();
all_kinds(&mut w);
let bytes = w.finish();
let at = collect(&bytes)
.unwrap()
.into_iter()
.find_map(|r| match r {
Record::Heap { at, record: HeapRecord::Instance { .. } } => Some(at),
_ => None,
})
.unwrap();
match record_at(&bytes, at, 8).unwrap() {
HeapRecord::Instance { id, class_id, data } => assert_eq!((id, class_id, data.len()), (0x10, 0x100, 16)),
other => panic!("expected an instance, got {other:?}"),
}
assert!(matches!(record_at(&bytes, bytes.len(), 8), Err(HeapError::Truncated { .. })));
assert!(matches!(record_at(&bytes, at, 3), Err(HeapError::UnsupportedIdSize(3))));
}
#[test]
fn scan_segment_visits_the_same_records_scan_does() {
let mut w = HprofWriter::new();
all_kinds(&mut w);
let bytes = w.finish();
let from_scan: Vec<_> = collect(&bytes)
.unwrap()
.into_iter()
.filter_map(|r| if let Record::Heap { at, record } = r { Some((at, record)) } else { None })
.collect();
let mut from_segment = Vec::new();
scan_segment(&bytes, segment_of(&bytes), 8, &mut |at, record| {
from_segment.push((at, record));
Ok(())
})
.unwrap();
assert_eq!(from_scan.len(), 6);
assert_eq!(from_scan, from_segment);
}
#[test]
fn visitor_errors_stop_the_scan() {
let bytes = MiniHeap::new().bytes();
let header = parse_header(&bytes).unwrap();
let mut calls = 0;
let result = scan(&bytes, &header, &mut |_| {
calls += 1;
Err(HeapError::Cancelled)
});
assert!(matches!(result, Err(HeapError::Cancelled)));
assert_eq!(calls, 1);
}
#[test]
fn both_heap_dump_tags_are_segments() {
let bytes = MiniHeap::new().bytes();
let segment = segment_of(&bytes);
let mut old_style = bytes.clone();
old_style[segment.start - 9] = 0x0C;
assert_eq!(segment_of(&old_style), segment);
}
#[test]
fn id_size_4_dump_parses() {
let mut w = HprofWriter::with_id_size(4);
w.string(1, "Node")
.load_class(0x100, 1)
.class(0x100, 0, &[(2, BasicType::Object), (3, BasicType::Long)], &[(2, 0x10)])
.instance(0x10, 0x100, &[0u8; 12])
.obj_array(0x20, 0x100, &[0x10, 0, 0xFFFF_FFFF])
.prim_array(0x30, BasicType::Short, 5)
.root(HeapRootKind::ThreadObject, 0x10);
let bytes = w.finish();
let header = parse_header(&bytes).unwrap();
assert_eq!((header.id_size, header.body_start), (4, 31));
let records: Vec<_> = collect(&bytes)
.unwrap()
.into_iter()
.filter_map(|r| if let Record::Heap { record, .. } = r { Some(record) } else { None })
.collect();
assert_eq!(records.len(), 5);
match &records[0] {
HeapRecord::Class(c) => {
assert_eq!((c.id, c.statics.as_slice(), c.fields.len()), (0x100, &[(2, 0x10)][..], 2));
}
other => panic!("expected the class, got {other:?}"),
}
assert!(matches!(&records[1], HeapRecord::Instance { id: 0x10, class_id: 0x100, data } if data.len() == 12));
assert!(
matches!(&records[2], HeapRecord::ObjectArray { len: 3, data, .. } if data.len() == 12 && data[8..] == [0xFF; 4])
);
assert!(
matches!(&records[3], HeapRecord::PrimitiveArray { ty: BasicType::Short, len: 5, data, .. } if data.len() == 10)
);
assert!(matches!(
&records[4],
HeapRecord::Root { kind: HeapRootKind::ThreadObject, id: 0x10, thread_serial: Some(1) }
));
}
fn push_id(out: &mut Vec<u8>, v: u64) {
out.extend_from_slice(&v.to_be_bytes());
}
/// A class dump written by hand: loader, a constant-pool entry, and statics of each shape.
fn class_dump_bytes(last_field_type: u8) -> Vec<u8> {
let mut b = vec![0x20];
push_id(&mut b, 0x500);
b.extend_from_slice(&0u32.to_be_bytes());
for id in [0x400, 0x77, 0, 0, 0, 0] {
push_id(&mut b, id); // super, loader, signers, domain, 2 reserved
}
b.extend_from_slice(&24u32.to_be_bytes());
b.extend_from_slice(&[0, 1, 0, 7, 10, 0, 0, 0, 99]); // constant pool: one int
b.extend_from_slice(&[0, 3]);
push_id(&mut b, 0xA);
b.push(2);
push_id(&mut b, 0xBEEF); // object static
push_id(&mut b, 0xB);
b.push(10);
b.extend_from_slice(&5u32.to_be_bytes()); // int static
push_id(&mut b, 0xC);
b.push(11);
b.extend_from_slice(&6u64.to_be_bytes()); // long static
b.extend_from_slice(&[0, 2]);
push_id(&mut b, 0xD);
b.push(2);
push_id(&mut b, 0xE);
b.push(last_field_type);
b
}
#[test]
fn class_dump_reads_loader_statics_and_skips_the_constant_pool() {
let bytes = class_dump_bytes(11);
let HeapRecord::Class(c) = record_at(&bytes, 0, 8).unwrap() else { panic!("expected a class") };
assert_eq!((c.id, c.super_id, c.loader_id), (0x500, 0x400, 0x77));
assert_eq!(c.statics, [(0xA, 0xBEEF)]);
assert_eq!(c.static_bytes, 12);
assert_eq!(c.fields, [(0xD, BasicType::Object), (0xE, BasicType::Long)]);
}
#[test]
fn unknown_value_type_reports_its_offset() {
let bytes = class_dump_bytes(0x63);
let err = record_at(&bytes, 0, 8).unwrap_err();
assert!(matches!(err, HeapError::UnknownType { ty: 0x63, at } if at == bytes.len() as u64 - 1), "{err:?}");
}
#[test]
fn truncated_dump_reports_offset() {
let mut bytes = MiniHeap::new().bytes();
let body_start = parse_header(&bytes).unwrap().body_start as u64;
bytes.truncate(bytes.len() - 5);
let err = collect(&bytes).unwrap_err();
assert!(matches!(err, HeapError::Truncated { at } if at > body_start && at == bytes.len() as u64), "{err:?}");
assert!(err.to_string().contains(&bytes.len().to_string()));
}
/// HotSpot flushes whole segments, so a JVM killed while dumping leaves a file that ends
/// right after one, without HEAP DUMP END.
#[test]
fn a_dump_the_jvm_did_not_finish_ends_early() {
let mut bytes = MiniHeap::new().bytes();
bytes.truncate(bytes.len() - RECORD_HEADER_LEN);
let err = collect(&bytes).unwrap_err();
assert!(matches!(err, HeapError::Truncated { at } if at == bytes.len() as u64), "{err:?}");
assert!(err.to_string().starts_with("the dump ends early"), "{err}");
}
#[test]
fn every_prefix_is_ok_or_truncated_at_its_length() {
let mut heap = MiniHeap::new();
heap.class("N", Some("java.lang.Object"), &[("next", BasicType::Object)])
.object(0x10, "N", &[("next", 0x20)])
.string(0x20, "x", true)
.root(HeapRootKind::ThreadObject, 0x10);
let bytes = heap.bytes();
let header = parse_header(&bytes).unwrap();
let segment = segment_of(&bytes);
for cut in header.body_start..bytes.len() {
let prefix = &bytes[..cut];
match scan(prefix, &header, &mut |_| Ok(())) {
// Only a prefix without heap data is a complete (empty) dump.
Ok(()) => assert!(cut < segment.start, "cut at {cut} has heap data but no end"),
Err(HeapError::Truncated { at }) => assert_eq!(at, cut as u64, "cut at {cut}"),
Err(other) => panic!("cut at {cut}: {other:?}"),
}
}
let past_end = HprofHeader { body_start: bytes.len() + 1, ..header };
assert!(matches!(scan(&bytes, &past_end, &mut |_| Ok(())), Err(HeapError::Truncated { .. })));
}
#[test]
fn sub_record_overrunning_its_segment_is_truncated() {
let mut bytes = MiniHeap::new().bytes();
let segment = segment_of(&bytes);
let len_at = segment.start - 4;
let shrunk = (segment.len() as u32 - 3).to_be_bytes();
bytes[len_at..segment.start].copy_from_slice(&shrunk);
let err = collect(&bytes).unwrap_err();
assert!(matches!(err, HeapError::Truncated { at } if at == segment.end as u64 - 3), "{err:?}");
}
#[test]
fn unknown_sub_record_is_an_error() {
let mut bytes = MiniHeap::new().bytes();
let segment = segment_of(&bytes);
bytes[segment.start] = 0x99;
let err = collect(&bytes).unwrap_err();
assert!(matches!(err, HeapError::UnknownSubRecord { tag: 0x99, at } if at == segment.start as u64), "{err:?}");
assert!(err.to_string().contains("0x99"));
}
#[test]
fn array_lengths_larger_than_the_dump_do_not_allocate_or_panic() {
let mut b = vec![0x23];
push_id(&mut b, 1);
b.extend_from_slice(&0u32.to_be_bytes());
b.extend_from_slice(&u32::MAX.to_be_bytes());
b.push(11);
assert!(matches!(record_at(&b, 0, 8), Err(HeapError::Truncated { .. })));
b[0] = 0x22;
assert!(matches!(record_at(&b, 0, 8), Err(HeapError::Truncated { .. })));
}
#[test]
fn primitive_array_of_objects_is_an_unknown_type() {
let mut b = vec![0x23];
push_id(&mut b, 1);
b.extend_from_slice(&[0, 0, 0, 0, 0, 0, 0, 0]);
b.push(2);
assert!(matches!(record_at(&b, 0, 8), Err(HeapError::UnknownType { ty: 2, .. })));
}
#[test]
fn basic_type_sizes_and_names() {
assert_eq!(BasicType::from_u8(10), Some(BasicType::Int));
assert_eq!([0, 1, 3, 12, 255].map(BasicType::from_u8), [None; 5]);
assert_eq!((BasicType::Object.hprof_size(4), BasicType::Object.hprof_size(8)), (4, 8));
assert_eq!((BasicType::Object.heap_size(4), BasicType::Long.heap_size(4)), (4, 8));
assert_eq!((BasicType::Char.hprof_size(8), BasicType::Boolean.heap_size(8)), (2, 1));
assert_eq!(
(BasicType::Int.array_name(), BasicType::Boolean.array_name(), BasicType::Long.array_name()),
("[I", "[Z", "[J")
);
}
fn sample() -> Vec<u8> {
let mut heap = MiniHeap::new();
heap.class("Node", Some("java.lang.Object"), &[("next", BasicType::Object), ("v", BasicType::Long)])
.class("[LNode;", Some("java.lang.Object"), &[])
.object(0x10, "Node", &[("next", 0x20), ("v", 5)])
.object(0x20, "Node", &[])
.array_of(0x30, "[LNode;", &[0x10, 0x20, 0])
.prim(0x40, BasicType::Int, 3)
.string(0x50, "hello", true)
.root(HeapRootKind::ThreadObject, 0x10)
.root(HeapRootKind::JniGlobal, 0x30);
heap.bytes()
}
fn with_header(id_size: u8, body: &[u8]) -> Vec<u8> {
let mut bytes = MAGIC_1_0_2.to_vec();
bytes.extend_from_slice(&u32::from(id_size).to_be_bytes());
bytes.extend_from_slice(&1_700_000_000_000u64.to_be_bytes());
bytes.extend_from_slice(body);
bytes
}
proptest! {
#[test]
fn parser_never_panics(bytes in proptest::collection::vec(any::<u8>(), 0..4096)) {
let _ = parse_header(&bytes).and_then(|h| scan(&bytes, &h, &mut |_| Ok(())));
}
/// Random bytes alone fail the magic check, so this puts a valid header in front to make
/// `scan` and `scan_segment` run on arbitrary record streams.
#[test]
fn random_bytes_after_a_valid_header_never_panic(
id_size in prop_oneof![Just(4u8), Just(8u8)],
body in proptest::collection::vec(any::<u8>(), 0..4096),
) {
let bytes = with_header(id_size, &body);
let header = parse_header(&bytes).unwrap();
prop_assert_eq!(header.id_size, id_size);
let _ = scan(&bytes, &header, &mut |_| Ok(()));
let _ = scan_segment(&bytes, header.body_start..bytes.len(), id_size, &mut |_, _| Ok(()));
for at in header.body_start..bytes.len() {
let _ = record_at(&bytes, at, id_size);
}
}
/// Same, but every record has a correct length so the random bytes reach the heap-dump
/// segments and the sub-record parsers instead of stopping at the first bad length.
#[test]
fn random_records_after_a_valid_header_never_panic(
id_size in prop_oneof![Just(4u8), Just(8u8)],
records in proptest::collection::vec(
(
prop_oneof![Just(0x01u8), Just(0x02), Just(0x0C), Just(0x1C), Just(0x2C), any::<u8>()],
prop_oneof![
Just(0x20u8), Just(0x21), Just(0x22), Just(0x23), Just(0x01), Just(0x02), Just(0x03),
Just(0x04), Just(0x05), Just(0x06), Just(0x07), Just(0x08), Just(0xFF), any::<u8>()
],
proptest::collection::vec(any::<u8>(), 0..96),
),
0..12,
),
) {
let mut body = Vec::new();
for (tag, first, rest) in records {
let mut record_body = vec![first];
record_body.extend_from_slice(&rest);
body.push(tag);
body.extend_from_slice(&0u32.to_be_bytes());
body.extend_from_slice(&(record_body.len() as u32).to_be_bytes());
body.extend_from_slice(&record_body);
}
let bytes = with_header(id_size, &body);
let header = parse_header(&bytes).unwrap();
let mut segments = Vec::new();
let _ = scan(&bytes, &header, &mut |r| {
if let Record::Segment(s) = r {
segments.push(s);
}
Ok(())
});
for s in segments {
let _ = scan_segment(&bytes, s, id_size, &mut |_, _| Ok(()));
}
}
#[test]
fn mutated_dump_never_panics(flips in proptest::collection::vec((0usize..2048, any::<u8>()), 1..16)) {
let pristine = sample();
let header = parse_header(&pristine).unwrap();
let segment = segment_of(&pristine);
let mut bytes = pristine.clone();
for (pos, value) in flips {
let at = pos % bytes.len();
bytes[at] = value;
}
let _ = scan(&bytes, &header, &mut |_| Ok(()));
for id_size in [4, 8] {
let _ = scan_segment(&bytes, segment.clone(), id_size, &mut |_, _| Ok(()));
}
for at in 0..bytes.len() {
let _ = record_at(&bytes, at, 8);
}
}
}
}
+327
Переглянути файл
@@ -0,0 +1,327 @@
//! Maps hprof object ids (addresses) to dense `u32` indices in ascending address order.
//!
//! Object ids are addresses and objects are at least 8 bytes apart, so one bit per 8-byte slot
//! ranks ids at about a byte per object. Addresses are grouped in 1 MiB chunks of
//! [`WORDS_PER_CHUNK`] words; only chunks holding an id get words, and a dense table maps a chunk
//! to its first word. The rank of an id is the number of set bits before its word plus the set
//! bits below it in the word. Ids that are not multiples of 8, or that span too wide an address
//! range for the table, use a sorted array and binary search instead.
mod builder;
pub(crate) use builder::ChunkHasher;
pub use builder::IdMapBuilder;
use crate::column::{Column, NONE};
const SLOT_SHIFT: u32 = 3;
const CHUNK_SLOT_BITS: u32 = 17;
const WORD_BITS: u64 = 64;
const WORDS_PER_CHUNK: usize = (1 << CHUNK_SLOT_BITS) / WORD_BITS as usize;
// The bit table is matched in hot paths; boxing the big variant would only add
// an indirection to save stack space `IdMap` never holds by value in bulk.
#[allow(clippy::large_enum_variant)]
enum Repr {
Bits {
/// Chunk number of the first table entry.
min_chunk: u64,
words: Column<u64>,
/// Set bits before each word.
wrank: Column<u32>,
/// Per chunk from `min_chunk` up: index of its first word, or `NONE` when it has no id.
table: Column<u32>,
/// Per chunk that has words, in word order: its offset from `min_chunk`.
chunk_offsets: Column<u32>,
len: u32,
},
Sorted(Column<u64>),
}
/// Ids ranked in ascending address order: the index of an object is its position in the order.
pub struct IdMap {
repr: Repr,
}
impl IdMap {
pub fn len(&self) -> u32 {
match &self.repr {
Repr::Bits { len, .. } => *len,
Repr::Sorted(ids) => ids.len() as u32,
}
}
pub fn is_empty(&self) -> bool {
self.len() == 0
}
/// The index of `id`, or `None` when it is not in the map.
pub fn rank(&self, id: u64) -> Option<u32> {
match &self.repr {
Repr::Sorted(ids) => ids.binary_search(&id).ok().map(|r| r as u32),
Repr::Bits { min_chunk, words, wrank, table, .. } => {
if id & 7 != 0 {
return None;
}
let slot = id >> SLOT_SHIFT;
let offset = (slot >> CHUNK_SLOT_BITS).checked_sub(*min_chunk)?;
let first = *table.get(usize::try_from(offset).ok()?)?;
if first == NONE {
return None;
}
let w = first as usize + (slot >> 6) as usize % WORDS_PER_CHUNK;
let bit = slot & 63;
let word = words[w];
if word >> bit & 1 == 0 {
return None;
}
Some(wrank[w] + (word & ((1u64 << bit) - 1)).count_ones())
}
}
}
/// The id with index `rank`. Panics when `rank >= len()`.
pub fn id_at(&self, rank: u32) -> u64 {
match &self.repr {
Repr::Sorted(ids) => ids[rank as usize],
Repr::Bits { words, wrank, len, .. } => {
assert!(rank < *len, "rank {rank} out of range for {len} ids");
let w = wrank.partition_point(|&before| before <= rank) - 1;
let mut word = words[w];
for _ in 0..rank - wrank[w] {
word &= word - 1;
}
(self.word_first_slot(w) + u64::from(word.trailing_zeros())) << SLOT_SHIFT
}
}
}
/// All ids in ascending order.
pub fn ids(&self) -> impl Iterator<Item = u64> + '_ {
Ids { map: self, next: 0, slot_base: 0, rest: 0 }
}
/// Bytes of all the columns, in RAM or on disk.
pub fn column_bytes(&self) -> u64 {
match &self.repr {
Repr::Sorted(ids) => std::mem::size_of_val(&**ids) as u64,
Repr::Bits { words, wrank, table, chunk_offsets, .. } => {
(std::mem::size_of_val(&**words)
+ std::mem::size_of_val(&**wrank)
+ std::mem::size_of_val(&**table)
+ std::mem::size_of_val(&**chunk_offsets)) as u64
}
}
}
/// Slot number of bit 0 of word `w` (bitmap layout only).
fn word_first_slot(&self, w: usize) -> u64 {
let Repr::Bits { min_chunk, chunk_offsets, .. } = &self.repr else {
unreachable!("sorted layout has no words")
};
let chunk = min_chunk + u64::from(chunk_offsets[w / WORDS_PER_CHUNK]);
(chunk << CHUNK_SLOT_BITS) + (w % WORDS_PER_CHUNK) as u64 * WORD_BITS
}
}
struct Ids<'a> {
map: &'a IdMap,
/// Next word (bitmap layout) or next id (sorted layout).
next: usize,
slot_base: u64,
/// Bits of the current word not yielded yet.
rest: u64,
}
impl Iterator for Ids<'_> {
type Item = u64;
fn next(&mut self) -> Option<u64> {
match &self.map.repr {
Repr::Sorted(ids) => {
let id = *ids.get(self.next)?;
self.next += 1;
Some(id)
}
Repr::Bits { words, .. } => loop {
if self.rest != 0 {
let bit = u64::from(self.rest.trailing_zeros());
self.rest &= self.rest - 1;
return Some((self.slot_base + bit) << SLOT_SHIFT);
}
self.rest = *words.get(self.next)?;
self.slot_base = self.map.word_first_slot(self.next);
self.next += 1;
},
}
}
}
#[cfg(test)]
mod tests {
use ide_core::CancellationToken;
use super::*;
use crate::column::Store;
/// Deterministic generator so failures replay; the crate has no `rand`.
struct XorShift(u64);
impl XorShift {
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
}
fn build(ids: &[u64], store: &Store) -> IdMap {
let mut builder = IdMapBuilder::new();
for &id in ids {
builder.insert(id);
}
builder.build(store, &CancellationToken::new()).unwrap()
}
fn assert_matches_model(map: &IdMap, model: &[u64]) {
assert_eq!(map.len() as usize, model.len());
assert_eq!(map.ids().collect::<Vec<_>>(), model);
for (rank, &id) in model.iter().enumerate() {
assert_eq!(map.rank(id), Some(rank as u32), "rank of {id:#x}");
assert_eq!(map.id_at(rank as u32), id, "id at {rank}");
for near in [id.wrapping_sub(8), id.wrapping_add(8), id.wrapping_add(1), id.wrapping_add(4)] {
if model.binary_search(&near).is_err() {
assert_eq!(map.rank(near), None, "{near:#x} was never inserted");
}
}
}
}
#[test]
fn rank_follows_address_order() {
let map = build(&[0x2000, 0x1000, 0x1010, 0x7_0000_0000], &Store::Ram);
assert_eq!(map.len(), 4);
assert_eq!(
[0x1000, 0x1010, 0x2000, 0x7_0000_0000].map(|id| map.rank(id)),
[Some(0), Some(1), Some(2), Some(3)]
);
assert_eq!(map.rank(0x1008), None);
assert_eq!(map.rank(0), None);
assert_eq!(map.id_at(2), 0x2000);
assert_eq!(map.ids().collect::<Vec<_>>(), [0x1000, 0x1010, 0x2000, 0x7_0000_0000]);
}
#[test]
fn unaligned_ids_use_sorted_fallback() {
let map = build(&[0x2000, 0x1004, 0x1000, 0x1010, 0x7_0000_0000], &Store::Ram);
assert_eq!(
[0x1000, 0x1004, 0x1010, 0x2000, 0x7_0000_0000].map(|id| map.rank(id)),
[Some(0), Some(1), Some(2), Some(3), Some(4)]
);
assert_eq!(map.rank(0x1008), None);
assert_eq!(map.id_at(1), 0x1004);
}
#[test]
fn sparse_wide_span_uses_fallback() {
let map = build(&[0x1000 + (1 << 50), 0x1000], &Store::Ram);
assert_eq!(map.rank(0x1000), Some(0));
assert_eq!(map.rank(0x1000 + (1 << 50)), Some(1));
assert_eq!(map.rank(0x1008), None);
assert!(map.column_bytes() < 64, "a two-id map must not allocate a chunk table");
}
#[test]
fn ten_million_ids_cost_about_a_byte_each() {
let mut builder = IdMapBuilder::new();
for i in 0..10_000_000u64 {
builder.insert(0x7_0000_0000 + i * 32);
}
let map = builder.build(&Store::Ram, &CancellationToken::new()).unwrap();
assert_eq!(map.len(), 10_000_000);
assert!(map.column_bytes() < 16_000_000, "{} bytes", map.column_bytes());
assert_eq!(map.rank(0x7_0000_0000 + 9_999_999 * 32), Some(9_999_999));
assert_eq!(map.id_at(5_000_000), 0x7_0000_0000 + 5_000_000 * 32);
}
#[test]
fn empty_map_and_duplicates() {
let map = build(&[], &Store::Ram);
assert!(map.is_empty());
assert_eq!(map.rank(0x1000), None);
assert_eq!(map.ids().count(), 0);
let map = build(&[0x1000, 0x1000, 0x1008, 0x1000], &Store::Ram);
assert_eq!(map.len(), 2);
assert_eq!(map.rank(0x1008), Some(1));
}
#[test]
fn chunk_boundaries_and_gaps_between_chunks() {
let ids = [0, 0xF_FFF8, 0x10_0000, 0x10_0008, 0x50_0000, 0x1_0000_0000];
let mut model = ids.to_vec();
model.sort_unstable();
let map = build(&ids, &Store::Ram);
assert_matches_model(&map, &model);
}
#[test]
fn ids_outside_the_populated_chunks_are_absent() {
let ids = [0x30_0000, 0x30_0008, 0x70_0000];
let map = build(&ids, &Store::Ram);
assert_matches_model(&map, &ids);
// Chunk 5 lies between the lowest and highest chunk but holds nothing.
assert_eq!(map.rank(0x50_0000), None);
// Chunk 1 is below the lowest populated chunk.
assert_eq!(map.rank(0x10_0000), None);
// Chunk 9 is past the last table entry.
assert_eq!(map.rank(0x900_0000), None);
assert_eq!(map.rank(u64::MAX - 7), None);
}
#[test]
fn highest_address_round_trips() {
let map = build(&[0xFFFF_FFFF_FFFF_FFF8], &Store::Ram);
assert_eq!(map.rank(0xFFFF_FFFF_FFFF_FFF8), Some(0));
assert_eq!(map.id_at(0), 0xFFFF_FFFF_FFFF_FFF8);
let map = build(&[0, 0xFFFF_FFFF_FFFF_FFF8], &Store::Ram);
assert_matches_model(&map, &[0, 0xFFFF_FFFF_FFFF_FFF8]);
}
#[test]
fn random_ids_match_a_sorted_model_in_ram_and_on_disk() {
for seed in 1..=8u64 {
let mut rng = XorShift(seed.wrapping_mul(0x9E37_79B9_7F4A_7C15) | 1);
let aligned = seed % 4 != 0;
let wide = seed % 3 == 0;
let ids: Vec<u64> = (0..3000)
.map(|_| {
let r = rng.next();
let base = if wide { (r >> 20) << 12 } else { 0x6_0000_0000 + (r % (6 << 20)) };
if aligned { base & !7 } else { base }
})
.collect();
let mut model = ids.clone();
model.sort_unstable();
model.dedup();
let dir = tempfile::tempdir().unwrap();
for store in [Store::Ram, Store::Dir(dir.path().to_path_buf())] {
assert_matches_model(&build(&ids, &store), &model);
}
}
}
#[test]
fn dir_store_keeps_the_index_columns_under_stable_names() {
let dir = tempfile::tempdir().unwrap();
let map = build(&[0x1000, 0x2000], &Store::Dir(dir.path().to_path_buf()));
let mut names: Vec<_> =
std::fs::read_dir(dir.path()).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect();
names.sort();
assert_eq!(names, ["ids.bits.u64", "ids.chunks.u32", "ids.table.u32", "ids.wrank.u32"]);
assert_eq!(map.rank(0x2000), Some(1));
let dir = tempfile::tempdir().unwrap();
build(&[0x1004], &Store::Dir(dir.path().to_path_buf()));
assert!(dir.path().join("ids.sorted.u64").exists());
}
}
+409
Переглянути файл
@@ -0,0 +1,409 @@
//! Collects object ids in any order and lays them out as an [`IdMap`].
use std::cmp::Reverse;
use std::collections::{BinaryHeap, HashMap};
use std::hash::{BuildHasherDefault, Hasher};
use ide_core::CancellationToken;
use super::{CHUNK_SLOT_BITS, IdMap, Repr, SLOT_SHIFT, WORD_BITS, WORDS_PER_CHUNK};
use crate::column::{NONE, Store};
use crate::error::HeapError;
use crate::files::{IDS_BITS_FILE, IDS_CHUNKS_FILE, IDS_SORTED_FILE, IDS_TABLE_FILE, IDS_WRANK_FILE};
/// Widest chunk range the dense table covers (256 MiB of table at most).
const MAX_CHUNK_SPAN: u64 = 1 << 26;
/// Chunks whose first word index still fits in the table's `u32`.
const MAX_CHUNKS: usize = (NONE as usize - 1) / WORDS_PER_CHUNK;
/// Largest object count: `NONE` and the one after it stay free as sentinels for the columns above.
const MAX_IDS: u64 = NONE as u64 - 2;
const CHUNK_BYTES: u64 = (WORDS_PER_CHUNK * size_of::<u64>()) as u64;
/// The bitmaps may take this much before any id is inserted, then this many bytes per distinct id.
/// A heap that fits in 4 GiB of address space never reaches the base allowance whatever the
/// insertion order; sparse input (one id per chunk) is moved to plain `u64` ids, 8 bytes each,
/// after at most `BITMAP_BASE_BYTES + BITMAP_BYTES_PER_ID * ids` of bitmaps, so builder memory is
/// bounded by the number of ids rather than by the address range they spread over.
const BITMAP_BASE_BYTES: u64 = 64 << 20;
const BITMAP_BYTES_PER_ID: u64 = 32;
/// Longest stretch of work between two looks at the cancellation token.
const CHECK_EVERY: usize = 1 << 20;
/// Chunk numbers are already well spread by a multiplicative hash; SipHash would dominate the
/// per-insert cost on a 100M-object heap. The product keeps the low zero bits of its input, so
/// the high half is folded down for the bucket index. The graph builder keys class and string
/// ids (addresses) with it for the same reason.
#[derive(Default)]
pub(crate) struct ChunkHasher(u64);
impl Hasher for ChunkHasher {
fn finish(&self) -> u64 {
self.0 ^ (self.0 >> 32)
}
fn write(&mut self, bytes: &[u8]) {
for &b in bytes {
self.write_u64(self.0.rotate_left(8) ^ u64::from(b));
}
}
fn write_u64(&mut self, n: u64) {
self.0 = n.wrapping_mul(0x9E37_79B9_7F4A_7C15);
}
}
struct BitChunks {
/// Chunk number → position in `words`.
index: HashMap<u64, usize, BuildHasherDefault<ChunkHasher>>,
words: Vec<Box<[u64]>>,
min: u64,
max: u64,
/// The chunk of the previous insert; ids usually arrive in runs.
last: Option<(u64, usize)>,
distinct: u64,
}
impl BitChunks {
fn new() -> Self {
Self { index: HashMap::default(), words: Vec::new(), min: u64::MAX, max: 0, last: None, distinct: 0 }
}
/// `false` when the id's chunk does not fit the dense table or the bitmap budget; the id is
/// then not recorded and the caller moves to plain ids.
fn insert(&mut self, id: u64) -> bool {
let slot = id >> SLOT_SHIFT;
let chunk = slot >> CHUNK_SLOT_BITS;
let at = match self.last {
Some((c, at)) if c == chunk => at,
_ => match self.index.get(&chunk) {
Some(&at) => at,
None => {
let (min, max) = (self.min.min(chunk), self.max.max(chunk));
let bitmap_bytes = (self.words.len() as u64 + 1) * CHUNK_BYTES;
if self.words.len() >= MAX_CHUNKS
|| max - min >= MAX_CHUNK_SPAN
|| bitmap_bytes > BITMAP_BASE_BYTES + BITMAP_BYTES_PER_ID * self.distinct
{
return false;
}
(self.min, self.max) = (min, max);
self.words.push(vec![0u64; WORDS_PER_CHUNK].into_boxed_slice());
self.index.insert(chunk, self.words.len() - 1);
self.words.len() - 1
}
},
};
self.last = Some((chunk, at));
let word = &mut self.words[at][(slot >> 6) as usize % WORDS_PER_CHUNK];
let bit = 1u64 << (slot & 63);
if *word & bit == 0 {
*word |= bit;
self.distinct += 1;
}
true
}
/// `(chunk number, position in words)` in ascending address order.
fn sorted_chunks(&self) -> Vec<(u64, usize)> {
let mut chunks: Vec<_> = self.index.iter().map(|(&chunk, &at)| (chunk, at)).collect();
chunks.sort_unstable();
chunks
}
/// The recorded ids, ascending.
fn ids(&self) -> BitIds<'_> {
BitIds { chunks: self.sorted_chunks(), words: &self.words, chunk: 0, word: 0, base: 0, rest: 0 }
}
}
struct BitIds<'a> {
chunks: Vec<(u64, usize)>,
words: &'a [Box<[u64]>],
chunk: usize,
word: usize,
/// Slot number of bit 0 of the word `rest` came from.
base: u64,
rest: u64,
}
impl Iterator for BitIds<'_> {
type Item = u64;
fn next(&mut self) -> Option<u64> {
loop {
if self.rest != 0 {
let bit = u64::from(self.rest.trailing_zeros());
self.rest &= self.rest - 1;
return Some((self.base + bit) << SLOT_SHIFT);
}
let &(chunk, at) = self.chunks.get(self.chunk)?;
self.rest = self.words[at][self.word];
self.base = (chunk << CHUNK_SLOT_BITS) + self.word as u64 * WORD_BITS;
self.word += 1;
if self.word == WORDS_PER_CHUNK {
self.word = 0;
self.chunk += 1;
}
}
}
}
pub struct IdMapBuilder {
bits: BitChunks,
/// Set once an id cannot go into `bits`; every later id goes here, and `bits` keeps what it
/// had. Unsorted, with duplicates.
plain: Option<Vec<u64>>,
max_ids: u64,
}
impl Default for IdMapBuilder {
fn default() -> Self {
Self::new()
}
}
impl IdMapBuilder {
pub fn new() -> Self {
Self::with_limit(MAX_IDS)
}
/// `max_ids` is a parameter so tests can reach the too-large path without billions of inserts.
fn with_limit(max_ids: u64) -> Self {
Self { bits: BitChunks::new(), plain: None, max_ids }
}
/// Inserting an id twice is the same as once.
pub fn insert(&mut self, id: u64) {
match &mut self.plain {
Some(plain) => plain.push(id),
None => {
if id & 7 != 0 || !self.bits.insert(id) {
self.plain = Some(vec![id]);
}
}
}
}
/// Fails with [`HeapError::TooLarge`] when there are more than `u32::MAX - 2` distinct ids,
/// and with [`HeapError::Cancelled`] when `cancel` fires (checked at least every `1 << 20`
/// steps).
pub fn build(self, store: &Store, cancel: &CancellationToken) -> Result<IdMap, HeapError> {
match self.plain {
None if self.bits.words.is_empty() => sorted_map(store, &[]),
None => bit_map(&self.bits, store, self.max_ids, cancel),
Some(plain) => merged_map(&self.bits, plain, store, self.max_ids, cancel),
}
}
#[cfg(test)]
fn allocated_bytes(&self) -> u64 {
self.bits.words.len() as u64 * CHUNK_BYTES
+ self.plain.as_ref().map_or(0, |p| (p.capacity() * size_of::<u64>()) as u64)
}
}
fn check(cancel: &CancellationToken) -> Result<(), HeapError> {
if cancel.is_cancelled() { Err(HeapError::Cancelled) } else { Ok(()) }
}
fn sorted_map(store: &Store, ids: &[u64]) -> Result<IdMap, HeapError> {
let mut column = store.column(IDS_SORTED_FILE, ids.len(), 0u64)?;
column.copy_from_slice(ids);
Ok(IdMap { repr: Repr::Sorted(column) })
}
fn bit_map(bits: &BitChunks, store: &Store, max_ids: u64, cancel: &CancellationToken) -> Result<IdMap, HeapError> {
if bits.distinct > max_ids {
return Err(HeapError::TooLarge);
}
let chunks = bits.sorted_chunks();
let span = (bits.max - bits.min + 1) as usize;
let mut words = store.column(IDS_BITS_FILE, chunks.len() * WORDS_PER_CHUNK, 0u64)?;
let mut wrank = store.column(IDS_WRANK_FILE, chunks.len() * WORDS_PER_CHUNK, 0u32)?;
let mut table = store.column(IDS_TABLE_FILE, span, NONE)?;
let mut chunk_offsets = store.column(IDS_CHUNKS_FILE, chunks.len(), 0u32)?;
let mut before = 0u32;
for (ordinal, &(chunk, at)) in chunks.iter().enumerate() {
check(cancel)?;
let first = ordinal * WORDS_PER_CHUNK;
let offset = (chunk - bits.min) as usize;
table[offset] = first as u32;
chunk_offsets[ordinal] = offset as u32;
for (w, &word) in bits.words[at].iter().enumerate() {
words[first + w] = word;
wrank[first + w] = before;
before += word.count_ones();
}
}
Ok(IdMap { repr: Repr::Bits { min_chunk: bits.min, words, wrank, table, chunk_offsets, len: before } })
}
enum Source<'a> {
Bits(BitIds<'a>),
Run(std::iter::Copied<std::slice::Iter<'a, u64>>),
}
impl Iterator for Source<'_> {
type Item = u64;
fn next(&mut self) -> Option<u64> {
match self {
Source::Bits(it) => it.next(),
Source::Run(it) => it.next(),
}
}
}
/// Sorts `plain` in runs and merges the runs with the ids already in `bits` (itself a sorted
/// source), dropping duplicates. A single `sort_unstable` of a very large vector could not be
/// interrupted.
fn merged_map(
bits: &BitChunks,
mut plain: Vec<u64>,
store: &Store,
max_ids: u64,
cancel: &CancellationToken,
) -> Result<IdMap, HeapError> {
for run in plain.chunks_mut(CHECK_EVERY) {
check(cancel)?;
run.sort_unstable();
}
let mut sources = vec![Source::Bits(bits.ids())];
sources.extend(plain.chunks(CHECK_EVERY).map(|run| Source::Run(run.iter().copied())));
let mut heap = BinaryHeap::with_capacity(sources.len());
for (i, source) in sources.iter_mut().enumerate() {
if let Some(id) = source.next() {
heap.push(Reverse((id, i)));
}
}
let upper_bound = bits.distinct as usize + plain.len();
let mut merged = store.scratch::<u64>(upper_bound, 0)?;
let mut count = 0usize;
let mut steps = 0usize;
while let Some(Reverse((id, i))) = heap.pop() {
steps += 1;
if steps.is_multiple_of(CHECK_EVERY) {
check(cancel)?;
}
if let Some(next) = sources[i].next() {
heap.push(Reverse((next, i)));
}
if count > 0 && merged[count - 1] == id {
continue;
}
if count as u64 >= max_ids {
return Err(HeapError::TooLarge);
}
merged[count] = id;
count += 1;
}
let mut column = store.column(IDS_SORTED_FILE, count, 0u64)?;
for (to, from) in column.chunks_mut(CHECK_EVERY).zip(merged[..count].chunks(CHECK_EVERY)) {
check(cancel)?;
to.copy_from_slice(from);
}
Ok(IdMap { repr: Repr::Sorted(column) })
}
#[cfg(test)]
mod tests {
use super::*;
fn insert_all(builder: &mut IdMapBuilder, ids: &[u64]) {
for &id in ids {
builder.insert(id);
}
}
#[test]
fn too_many_ids_is_too_large() {
// Dense layout, then the merged layout (one unaligned id); the duplicate does not count.
for unaligned in [false, true] {
let filled = |limit| {
let mut builder = IdMapBuilder::with_limit(limit);
insert_all(&mut builder, &[0x1000, 0x1008, 0x1010, 0x1010]);
if unaligned {
builder.insert(0x1_0004);
}
builder
};
let distinct = if unaligned { 4 } else { 3 };
let map = filled(distinct).build(&Store::Ram, &CancellationToken::new()).unwrap();
assert_eq!(u64::from(map.len()), distinct);
let err = filled(distinct - 1).build(&Store::Ram, &CancellationToken::new()).err().unwrap();
assert!(matches!(err, HeapError::TooLarge), "{err}");
}
}
#[test]
fn cancelled_build_returns_cancelled() {
let cancelled = CancellationToken::new();
cancelled.cancel();
let mut dense = IdMapBuilder::new();
insert_all(&mut dense, &[0x1000, 0x1008, 0x20_0000]);
assert!(matches!(dense.build(&Store::Ram, &cancelled), Err(HeapError::Cancelled)));
let mut merged = IdMapBuilder::new();
insert_all(&mut merged, &[0x1000, 0x1004, 0x20_0000]);
assert!(matches!(merged.build(&Store::Ram, &cancelled), Err(HeapError::Cancelled)));
let mut ok = IdMapBuilder::new();
insert_all(&mut ok, &[0x1000, 0x1004]);
assert_eq!(ok.build(&Store::Ram, &CancellationToken::new()).unwrap().len(), 2);
}
#[test]
fn sparse_ids_do_not_amplify_builder_memory() {
const IDS: u64 = 2_000_000;
let mut builder = IdMapBuilder::new();
let mut peak = 0;
for i in 0..IDS {
builder.insert(0x1_0000_0000 + (i << 20));
if i % 1024 == 0 {
peak = peak.max(builder.allocated_bytes());
}
}
peak = peak.max(builder.allocated_bytes());
// Bitmaps up to the budget, plus the plain ids' vector, which may have doubled its capacity.
let budget = BITMAP_BASE_BYTES + (BITMAP_BYTES_PER_ID + 2 * size_of::<u64>() as u64) * IDS;
assert!(peak <= budget, "peak {peak} bytes, budget {budget}");
// One bitmap per id would have taken 16 KiB each.
assert!(peak < IDS * CHUNK_BYTES / 100, "peak {peak} bytes");
assert!(builder.plain.is_some(), "sparse input must have left the bitmaps");
let map = builder.build(&Store::Ram, &CancellationToken::new()).unwrap();
assert_eq!(map.len() as u64, IDS);
assert_eq!(map.rank(0x1_0000_0000 + (1_234_567 << 20)), Some(1_234_567));
assert_eq!(map.rank(0x1_0000_0000 + (1_234_567 << 20) + 8), None);
}
#[test]
fn dense_input_in_any_order_stays_in_bitmaps() {
// 4 GiB of address space is the base allowance; random order must not trip the budget.
let mut state = 0x2545_F491_4F6C_DD1Du64;
let mut builder = IdMapBuilder::new();
for _ in 0..300_000 {
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
builder.insert((0x7_0000_0000 + (state % (4 << 30))) & !7);
}
assert!(builder.plain.is_none());
assert!(builder.allocated_bytes() <= BITMAP_BASE_BYTES);
}
#[test]
fn chunk_hasher_spreads_chunks_that_share_low_zero_bits() {
let buckets: std::collections::HashSet<u64> = (0..1024u64)
.map(|i| {
let mut hasher = ChunkHasher::default();
hasher.write_u64(i << 12);
hasher.finish() & 1023
})
.collect();
assert!(buckets.len() > 500, "{} buckets for 1024 keys", buckets.len());
}
}
+109
Переглянути файл
@@ -0,0 +1,109 @@
//! The result of analyzing a heap dump: per-object columns, the class table and the summary.
mod fields;
mod path;
mod queries;
mod rows;
use std::collections::HashMap;
use std::path::Path;
use memmap2::Mmap;
use serde::{Deserialize, Serialize};
use crate::column::Column;
use crate::error::HeapError;
use crate::files::SUMMARY_FILE;
use crate::graph::ClassInfo;
use crate::hprof::HeapRootKind;
use crate::ids::IdMap;
use crate::shallow::SizeSource;
pub(crate) use self::fields::{id_value, instance_fields};
pub use self::rows::{
HeapClassRow, HeapEdge, HeapObjectPage, HeapObjectRef, HeapObjectRow, HeapPathStep, HeapRootPath, MAX_PAGE,
};
/// Counts over instances and arrays; class objects (class dumps) are left out.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapSummary {
/// `reachable_objects + unreachable_objects`.
pub objects: u64,
pub reachable_objects: u64,
pub reachable_bytes: u64,
pub unreachable_objects: u64,
pub unreachable_bytes: u64,
/// Class dumps in the dump.
pub classes: u32,
pub weak_edges_skipped: u64,
pub size_source: SizeSource,
pub analysis_ms: u32,
}
/// One row of the class table, for one type slot.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ClassStat {
/// [`crate::display_name`] of the class (`long[]` for a primitive array slot).
pub name: String,
/// Reachable instances.
pub count: u64,
/// Shallow bytes of the reachable instances.
pub shallow: u64,
/// Bytes the reachable instances retain together: the sum of the retained sizes of those
/// with no instance of the same type above them in the dominator tree.
pub retained: u64,
pub unreachable_count: u64,
}
/// Columns are indexed by node: objects `0..root`, then the super-root `root`. Columns of the
/// objects only (`types`, `offsets`) have no entry for the root.
pub struct HeapIndex {
/// The dump, for [`crate::record_at`] at `offsets[v]`.
pub hprof: Mmap,
pub ids: IdMap,
/// Type slot of each object, as in [`crate::HeapGraph::types`].
pub types: Column<u32>,
pub offsets: Column<u64>,
/// Shallow size in units of 8 bytes, rounded up.
pub shallow8: Column<u32>,
/// Retained bytes; 0 for unreachable objects, every reachable byte for the root.
pub retained: Column<u64>,
/// Immediate dominator; [`crate::NONE`] for unreachable objects, `root` for the root.
pub idom: Column<u32>,
/// Children of node `v` in the dominator tree are `kids[kids_start[v]..kids_start[v + 1]]`,
/// by retained size descending, then by node.
pub kids_start: Column<u32>,
pub kids: Column<u32>,
/// Parent on a shortest path from the root (BFS over strong edges); [`crate::NONE`] for
/// unreachable objects, `root` for the root.
pub gc_parent: Column<u32>,
/// By type slot: class slots, then the primitive array slots.
pub classes: Vec<ClassStat>,
/// By class slot, as in [`crate::HeapGraph::classes`].
pub class_info: Vec<ClassInfo>,
/// Type slot of class objects.
pub class_slot: u32,
/// Object → kind and thread serial of its first root record.
pub roots: HashMap<u32, (HeapRootKind, Option<u32>)>,
/// Thread serial → thread object.
pub thread_objects: HashMap<u32, u32>,
pub id_size: u8,
pub summary: HeapSummary,
}
impl HeapIndex {
/// The super-root, whose children in both trees are the rooted objects.
pub fn root(&self) -> u32 {
self.ids.len()
}
}
pub(crate) fn write_summary(dir: &Path, summary: &HeapSummary) -> Result<(), HeapError> {
let json = serde_json::to_vec_pretty(summary).map_err(std::io::Error::from)?;
std::fs::write(dir.join(SUMMARY_FILE), json)?;
Ok(())
}
#[cfg(test)]
mod tests;
+41
Переглянути файл
@@ -0,0 +1,41 @@
//! Named field values of an instance record.
use super::HeapIndex;
use crate::hprof::BasicType;
pub(crate) struct FieldValue<'a> {
pub(crate) name: &'a str,
pub(crate) ty: BasicType,
/// Big-endian, as in the dump.
pub(crate) bytes: &'a [u8],
/// `java.lang.ref.Reference.referent`, which the graph leaves out.
pub(crate) weak: bool,
}
/// The fields of an instance of class slot `class` whose record holds `data`: the class's own,
/// then each superclass's, as HotSpot lays them out. Stops where `data` ends.
pub(crate) fn instance_fields<'a>(index: &'a HeapIndex, class: u32, data: &'a [u8]) -> Vec<FieldValue<'a>> {
let mut out = Vec::new();
let mut base = 0usize;
let mut cursor = Some(class);
// Super chains end (their cycles are cut), so this is only a bound on corrupt input.
for _ in 0..index.class_info.len() {
let Some(info) = cursor.and_then(|c| index.class_info.get(c as usize)) else { break };
for (k, (name, ty)) in info.fields.iter().enumerate() {
let size = ty.hprof_size(index.id_size);
let Some(bytes) = base.checked_add(size).and_then(|end| data.get(base..end)) else { return out };
out.push(FieldValue { name, ty: *ty, bytes, weak: info.weak_field == Some(k) });
base += size;
}
cursor = info.super_index;
}
out
}
/// A 4- or 8-byte big-endian id.
pub(crate) fn id_value(bytes: &[u8]) -> Option<u64> {
match *bytes {
[a, b, c, d] => Some(u64::from(u32::from_be_bytes([a, b, c, d]))),
_ => Some(u64::from_be_bytes(bytes.try_into().ok()?)),
}
}
+105
Переглянути файл
@@ -0,0 +1,105 @@
//! Paths to GC roots, along the BFS tree of `gc_parent`.
use super::{HeapEdge, HeapIndex, HeapObjectRef, HeapPathStep, HeapRootPath, id_value, instance_fields};
use crate::column::NONE;
use crate::hprof::{BasicType, HeapRecord, HeapRootKind};
use crate::strings::{char_array_text, string_preview};
const THREAD_NAME_CHARS: usize = 100;
/// Steps a long path keeps at each end: one deep in a long linked list would otherwise send
/// every node over IPC.
const PATH_END_STEPS: u32 = 500;
impl HeapIndex {
/// From `object` up to its GC root, each step with how the next one holds it; past 1000
/// steps, only the 500 at each end. `None` for an unreachable object or a ref that names no
/// object.
pub fn path_to_root(&self, object: HeapObjectRef) -> Option<HeapRootPath> {
let root = self.root();
let mut v = object.0;
if v >= root || self.gc_parent[v as usize] == NONE {
return None;
}
let len = self.path_len(v)?;
let end = PATH_END_STEPS;
let cut = len > 2 * end;
let mut steps = Vec::with_capacity(len.min(2 * end) as usize);
for i in 0..len {
let parent = self.gc_parent[v as usize];
if !cut || i < end || i >= len - end {
let via = if parent == root { self.root_edge(v) } else { self.edge(parent, v) };
steps.push(HeapPathStep { object: self.object_row(HeapObjectRef(v))?, via });
}
v = parent;
}
let omitted = if cut { len - 2 * end } else { 0 };
Some(HeapRootPath { steps, omitted, omitted_after: if cut { end } else { 0 } })
}
/// Steps from `v` to the root; `None` when `gc_parent` leads nowhere.
fn path_len(&self, mut v: u32) -> Option<u32> {
let root = self.root();
// A BFS tree has no cycles; the bound only stops a corrupt column from looping.
for len in 1..=root {
let parent = self.gc_parent[v as usize];
if parent == root {
return Some(len);
}
if parent > root {
return None;
}
v = parent;
}
None
}
fn root_edge(&self, v: u32) -> HeapEdge {
let (kind, serial) = self.roots.get(&v).copied().unwrap_or((HeapRootKind::Unknown, None));
let thread = serial.and_then(|s| self.thread_objects.get(&s)).and_then(|&t| self.thread_name(t));
HeapEdge::Root { root: kind, thread }
}
/// `Thread.name`: a String, or a `char[]` before JDK 9.
fn thread_name(&self, thread: u32) -> Option<String> {
let Some(HeapRecord::Instance { data, .. }) = self.record(thread) else { return None };
let fields = instance_fields(self, *self.types.get(thread as usize)?, data);
let name = fields.iter().find(|f| f.name == "name" && f.ty == BasicType::Object)?;
let target = self.ids.rank(id_value(name.bytes)?)?;
string_preview(self, target, THREAD_NAME_CHARS).or_else(|| char_array_text(self, target, THREAD_NAME_CHARS))
}
/// How `parent` refers to `child`, checked in the order the graph's edges were made. A strong
/// edge from `parent` always exists; the empty field name only covers a dump that changed.
fn edge(&self, parent: u32, child: u32) -> HeapEdge {
self.find_edge(parent, self.ids.id_at(child)).unwrap_or(HeapEdge::Field { name: String::new() })
}
fn find_edge(&self, parent: u32, child_id: u64) -> Option<HeapEdge> {
match self.record(parent)? {
HeapRecord::Instance { data, .. } => {
let class = self.types[parent as usize];
let field = instance_fields(self, class, data)
.into_iter()
.find(|f| !f.weak && f.ty == BasicType::Object && id_value(f.bytes) == Some(child_id));
match field {
Some(f) => Some(HeapEdge::Field { name: f.name.to_owned() }),
None => (self.class_info.get(class as usize)?.id == child_id).then_some(HeapEdge::ClassOf),
}
}
HeapRecord::ObjectArray { data, .. } => {
let index =
data.chunks_exact(usize::from(self.id_size)).position(|item| id_value(item) == Some(child_id))?;
Some(HeapEdge::Element { index: u32::try_from(index).ok()? })
}
HeapRecord::Class(dump) if dump.super_id == child_id => Some(HeapEdge::SuperClass),
HeapRecord::Class(dump) if dump.loader_id == child_id => Some(HeapEdge::ClassLoader),
HeapRecord::Class(_) => {
let info = self.class_info.get(self.class_of_object(parent)?)?;
let (name, _) = info.statics.iter().find(|(_, object)| *object == child_id)?;
Some(HeapEdge::Static { name: name.clone() })
}
HeapRecord::Root { .. } | HeapRecord::PrimitiveArray { .. } => None,
}
}
}
+164
Переглянути файл
@@ -0,0 +1,164 @@
//! Class table rows, pages of the dominator tree and single object rows.
use std::cmp::Reverse;
use std::collections::{BinaryHeap, HashMap};
use super::{HeapClassRow, HeapIndex, HeapObjectPage, HeapObjectRef, HeapObjectRow, HeapSummary, MAX_PAGE};
use crate::column::NONE;
use crate::hprof::HeapRecord;
use crate::strings::string_preview;
const PREVIEW_CHARS: usize = 100;
impl HeapIndex {
pub fn summary(&self) -> &HeapSummary {
&self.summary
}
/// Types that have objects, by retained size descending, then by name.
pub fn class_rows(&self) -> Vec<HeapClassRow> {
let class_slots = &self.classes[..self.class_info.len().min(self.classes.len())];
let mut names: HashMap<&str, u32> = HashMap::with_capacity(class_slots.len());
for class in class_slots {
*names.entry(class.name.as_str()).or_default() += 1;
}
let mut rows: Vec<HeapClassRow> = self
.classes
.iter()
.enumerate()
.filter(|(_, c)| c.count > 0 || c.unreachable_count > 0)
.map(|(slot, c)| HeapClassRow {
class_index: slot as u32,
name: c.name.clone(),
count: c.count,
shallow: c.shallow,
retained: c.retained,
unreachable_count: c.unreachable_count,
// Primitive array slots have no class dump and so no second loader.
ambiguous: slot < class_slots.len() && names[c.name.as_str()] > 1,
})
.collect();
rows.sort_by(|a, b| b.retained.cmp(&a.retained).then_with(|| a.name.cmp(&b.name)));
rows
}
/// A page of objects by retained size descending, then by index:
///
/// - no `parent`, no `class_index`: the children of the super-root, class objects left out;
/// - `parent`: its children in the dominator tree, of type `class_index` if given;
/// - `class_index` only: every reachable object of that type.
///
/// `limit` is clamped to [`MAX_PAGE`]. A `parent` or `class_index` that names nothing gives an
/// empty page.
pub fn dominated(
&self,
parent: Option<HeapObjectRef>,
class_index: Option<u32>,
offset: u32,
limit: u32,
) -> HeapObjectPage {
let limit = limit.min(MAX_PAGE);
let root = self.root();
if class_index.is_some_and(|c| c as usize >= self.classes.len()) {
return HeapObjectPage { rows: Vec::new(), total: 0 };
}
match (parent, class_index) {
(Some(HeapObjectRef(p)), _) if p >= root => HeapObjectPage { rows: Vec::new(), total: 0 },
(Some(HeapObjectRef(p)), class) => self.page(
self.children(p).filter(|&v| class.is_none_or(|c| self.types[v as usize] == c)),
offset,
limit,
),
(None, None) => {
self.page(self.children(root).filter(|&v| self.types[v as usize] != self.class_slot), offset, limit)
}
(None, Some(class)) => self.largest_of(class, offset, limit),
}
}
/// `None` for a ref that names no object, the super-root included.
pub fn object_row(&self, object: HeapObjectRef) -> Option<HeapObjectRow> {
let v = object.0;
if v >= self.root() {
return None;
}
let i = v as usize;
let slot = self.types[i];
let preview = if slot == self.class_slot {
self.class_of_object(v).and_then(|c| self.classes.get(c)).map(|c| c.name.clone())
} else {
string_preview(self, v, PREVIEW_CHARS)
};
Some(HeapObjectRow {
object,
class_name: self.classes.get(slot as usize).map(|c| c.name.clone()).unwrap_or_default(),
address: format!("{:#x}", self.ids.id_at(v)),
shallow: u64::from(self.shallow8[i]) * 8,
retained: self.retained[i],
has_children: self.kids_start[i + 1] > self.kids_start[i],
preview,
})
}
/// The record of object `v`, or `None` when it cannot be read.
pub(crate) fn record(&self, v: u32) -> Option<HeapRecord<'_>> {
let at = usize::try_from(*self.offsets.get(v as usize)?).ok()?;
crate::hprof::record_at(&self.hprof, at, self.id_size).ok()
}
/// The class slot a class object stands for.
pub(crate) fn class_of_object(&self, v: u32) -> Option<usize> {
if v >= self.root() {
return None;
}
let id = self.ids.id_at(v);
self.class_info.iter().position(|c| c.id == id && c.id != 0)
}
/// Children of node `v` in the dominator tree, already in page order.
fn children(&self, v: u32) -> impl Iterator<Item = u32> + '_ {
let v = v as usize;
self.kids[self.kids_start[v] as usize..self.kids_start[v + 1] as usize].iter().copied()
}
fn page(&self, objects: impl Iterator<Item = u32>, offset: u32, limit: u32) -> HeapObjectPage {
let mut rows = Vec::new();
let mut total = 0u32;
for v in objects {
if total >= offset && total - offset < limit {
rows.extend(self.object_row(HeapObjectRef(v)));
}
total = total.saturating_add(1);
}
HeapObjectPage { rows, total }
}
/// Scans every object: the top `offset + limit` stay in a heap whose top is the smallest.
fn largest_of(&self, class: u32, offset: u32, limit: u32) -> HeapObjectPage {
let count = self.classes[class as usize].count;
let total = u32::try_from(count).unwrap_or(u32::MAX);
let keep = match offset.checked_add(limit) {
Some(keep) if offset < total && limit > 0 => keep.min(total) as usize,
_ => return HeapObjectPage { rows: Vec::new(), total },
};
let mut top: BinaryHeap<Reverse<(u64, Reverse<u32>)>> = BinaryHeap::with_capacity(keep + 1);
for v in 0..self.root() {
let i = v as usize;
if self.types[i] != class || self.idom[i] == NONE {
continue;
}
top.push(Reverse((self.retained[i], Reverse(v))));
if top.len() > keep {
top.pop();
}
}
// Ascending `Reverse` order is descending retained, then ascending index.
let rows = top
.into_sorted_vec()
.into_iter()
.skip(offset as usize)
.filter_map(|Reverse((_, Reverse(v)))| self.object_row(HeapObjectRef(v)))
.collect();
HeapObjectPage { rows, total }
}
}
+95
Переглянути файл
@@ -0,0 +1,95 @@
//! What the index queries return over IPC.
use serde::{Deserialize, Serialize};
use crate::hprof::HeapRootKind;
/// Most rows one page of [`crate::HeapIndex::dominated`] returns.
pub const MAX_PAGE: u32 = 500;
/// An object of one snapshot: its index in the [`crate::HeapIndex`] columns. Any value may
/// arrive over IPC; queries answer an index that names no object with nothing.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize, specta::Type)]
#[serde(transparent)]
pub struct HeapObjectRef(pub u32);
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapClassRow {
/// Type slot, as `classIndex` in [`crate::HeapIndex::dominated`].
pub class_index: u32,
pub name: String,
pub count: u64,
pub shallow: u64,
pub retained: u64,
pub unreachable_count: u64,
/// Another class has the same name (several class loaders): nothing joined by name, such as a
/// second snapshot, can tell them apart.
pub ambiguous: bool,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapObjectRow {
pub object: HeapObjectRef,
pub class_name: String,
/// The object's hprof id in hex, `0x1300`.
pub address: String,
pub shallow: u64,
pub retained: u64,
/// Has children in the dominator tree.
pub has_children: bool,
/// The text of a `java.lang.String`, or the name of the class a class object stands for.
pub preview: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapObjectPage {
pub rows: Vec<HeapObjectRow>,
/// Rows of every page together.
pub total: u32,
}
/// How an object is held by the next step of a path to a GC root.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum HeapEdge {
/// An instance field; the name is empty when the dump does not show which one.
Field {
name: String,
},
Element {
index: u32,
},
Static {
name: String,
},
/// From an instance to its class.
ClassOf,
SuperClass,
ClassLoader,
/// `thread` is the name of the thread whose root it is, when the dump has it.
Root {
root: HeapRootKind,
thread: Option<String>,
},
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapPathStep {
pub object: HeapObjectRow,
/// How the next step holds this object; on the last step, the root.
pub via: HeapEdge,
}
/// A path from an object up to its GC root. A long one keeps only the steps nearest each end.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapRootPath {
pub steps: Vec<HeapPathStep>,
/// Steps left out between `steps[omitted_after - 1]` and `steps[omitted_after]`; 0 for none.
pub omitted: u32,
pub omitted_after: u32,
}
+336
Переглянути файл
@@ -0,0 +1,336 @@
use super::*;
use crate::hprof::BasicType;
use crate::test_index::{analyzed, analyzed_bytes, obj, slot};
use crate::testing::{HprofWriter, MiniHeap, sample, sample_with_string};
const H: u64 = 0x1000;
const N1: u64 = 0x1100;
const N3: u64 = 0x1300;
const L: u64 = 0x1400;
const W: u64 = 0x1600;
const N4: u64 = 0x1700;
fn addresses(page: &HeapObjectPage) -> Vec<&str> {
page.rows.iter().map(|r| r.address.as_str()).collect()
}
fn retained(page: &HeapObjectPage) -> Vec<u64> {
page.rows.iter().map(|r| r.retained).collect()
}
fn path(index: &HeapIndex, addr: u64) -> Option<Vec<(String, HeapEdge)>> {
let path = index.path_to_root(obj(index, addr))?;
assert_eq!((path.omitted, path.omitted_after), (0, 0));
Some(path.steps.into_iter().map(|s| (s.object.address, s.via)).collect())
}
fn field(name: &str) -> HeapEdge {
HeapEdge::Field { name: name.to_owned() }
}
fn root(root: HeapRootKind, thread: Option<&str>) -> HeapEdge {
HeapEdge::Root { root, thread: thread.map(str::to_owned) }
}
#[test]
fn top_level_dominators_sorted_by_retained() {
let index = analyzed(&sample());
let page = index.dominated(None, None, 0, 10);
assert_eq!(addresses(&page), ["0x1000", "0x1300", "0x1600"]);
assert_eq!(retained(&page), [72, 56, 24]);
// The class objects, rooted as sticky classes, are left out.
assert_eq!(page.total, 3);
assert_eq!(addresses(&index.dominated(None, None, 1, 1)), ["0x1300"]);
}
#[test]
fn children_of_holder() {
let index = analyzed(&sample());
let page = index.dominated(Some(obj(&index, H)), None, 0, 10);
assert_eq!(addresses(&page), ["0x1100", "0x1200"]);
assert_eq!(retained(&page), [24, 24]);
assert_eq!(page.total, 2);
assert!(page.rows.iter().all(|r| !r.has_children && r.class_name == "Node"));
let n3 = index.object_row(obj(&index, N3)).unwrap();
assert!(n3.has_children);
assert_eq!(index.dominated(Some(n3.object), None, 0, 10).rows[0].class_name, "long[]");
}
#[test]
fn largest_of_class() {
let index = analyzed(&sample());
let node = slot(&index, "Node");
let page = index.dominated(None, Some(node), 0, 2);
assert_eq!(addresses(&page), ["0x1300", "0x1100"]);
assert_eq!(retained(&page), [56, 24]);
assert_eq!(page.total, 3);
assert_eq!(addresses(&index.dominated(None, Some(node), 2, 2)), ["0x1200"]);
assert!(index.dominated(None, Some(node), 3, 2).rows.is_empty());
// Within a parent's children.
assert_eq!(addresses(&index.dominated(Some(obj(&index, H)), Some(node), 1, 5)), ["0x1200"]);
assert_eq!(index.dominated(Some(obj(&index, H)), Some(slot(&index, "Holder")), 0, 5).total, 0);
}
#[test]
fn object_row_describes_object() {
let index = analyzed(&sample());
let row = index.object_row(obj(&index, N3)).unwrap();
assert_eq!(
row,
HeapObjectRow {
object: obj(&index, N3),
class_name: "Node".to_owned(),
address: "0x1300".to_owned(),
shallow: 24,
retained: 56,
has_children: true,
preview: None,
}
);
let unreachable = index.object_row(obj(&index, N4)).unwrap();
assert_eq!((unreachable.retained, unreachable.has_children), (0, false));
let holder = &index.class_info[slot(&index, "Holder") as usize];
let class_object = obj(&index, holder.id);
let row = index.object_row(class_object).unwrap();
assert_eq!((row.class_name.as_str(), row.preview.as_deref()), ("java.lang.Class", Some("Holder")));
}
#[test]
fn refs_that_name_no_object_answer_nothing() {
let index = analyzed(&sample());
let node = slot(&index, "Node");
for bad in [index.root(), index.root() + 1, u32::MAX] {
let bad = HeapObjectRef(bad);
assert_eq!(index.object_row(bad), None);
assert_eq!(index.path_to_root(bad), None);
assert_eq!(index.dominated(Some(bad), None, 0, 10), HeapObjectPage { rows: Vec::new(), total: 0 });
assert_eq!(index.dominated(Some(bad), Some(node), 0, 10).total, 0);
}
for bad in [index.classes.len() as u32, u32::MAX] {
assert_eq!(index.dominated(None, Some(bad), 0, 10).total, 0);
}
}
#[test]
fn path_to_root_names_fields_and_root() {
let index = analyzed(&sample());
assert_eq!(
path(&index, L),
Some(vec![("0x1400".to_owned(), field("next")), ("0x1300".to_owned(), root(HeapRootKind::JavaFrame, None))])
);
assert_eq!(
path(&index, N1),
Some(vec![("0x1100".to_owned(), field("a")), ("0x1000".to_owned(), root(HeapRootKind::JniGlobal, None))])
);
assert_eq!(path(&index, W), Some(vec![("0x1600".to_owned(), root(HeapRootKind::JniGlobal, None))]));
assert_eq!(path(&index, N4), None);
}
/// A rooted linked list of `len` Nodes from `0x100000`, each held by the next one's `next`.
fn chain(len: u64) -> HeapIndex {
let mut heap = sample();
for i in 0..len {
let next = if i == 0 { 0 } else { 0x10_0000 + (i - 1) * 0x20 };
heap.object(0x10_0000 + i * 0x20, "Node", &[("next", next)]);
}
heap.root(HeapRootKind::JniGlobal, 0x10_0000 + (len - 1) * 0x20);
analyzed(&heap)
}
#[test]
fn path_of_an_object_deep_in_a_linked_list_keeps_both_ends() {
let index = chain(3000);
let path = index.path_to_root(obj(&index, 0x10_0000)).unwrap();
assert_eq!((path.steps.len(), path.omitted, path.omitted_after), (1000, 2000, 500));
let address = |i: usize| path.steps[i].object.address.clone();
assert_eq!([address(0), address(499)], ["0x100000", "0x103e60"]);
// The step after the gap is the one 2000 nodes further up.
assert_eq!([address(500), address(999)], ["0x113880", "0x1176e0"]);
assert_eq!(path.steps[999].via, root(HeapRootKind::JniGlobal, None));
assert!(path.steps[..999].iter().all(|s| s.via == field("next")));
}
#[test]
fn path_of_exactly_a_thousand_steps_is_whole() {
let index = chain(1000);
let path = index.path_to_root(obj(&index, 0x10_0000)).unwrap();
assert_eq!((path.steps.len(), path.omitted, path.omitted_after), (1000, 0, 0));
}
#[test]
fn path_skips_the_weak_referent() {
let mut heap = sample();
heap.object(0x1c00, "java.lang.ref.WeakReference", &[("referent", 0x1d00), ("queue", 0x1d00)])
.object(0x1d00, "Node", &[])
.root(HeapRootKind::JniGlobal, 0x1c00);
let index = analyzed(&heap);
assert_eq!(path(&index, 0x1d00).unwrap()[0].1, field("queue"));
}
/// Kid extends Base, loaded by loader `0x300`; Base's static `cache` holds X; K is a rooted Kid;
/// `0x700` is a rooted `Object[] { null, Y }`. No class is rooted on its own.
fn class_edges_heap() -> Vec<u8> {
let mut w = HprofWriter::new();
let names = [(1, "Base"), (2, "Kid"), (3, "Ldr"), (4, "[Ljava/lang/Object;"), (5, "cache")];
for (id, name) in names {
w.string(id, name);
}
for (class, name) in [(0x100, 1), (0x200, 2), (0x400, 3), (0x800, 4)] {
w.load_class(class, name);
}
w.class(0x100, 0, &[], &[(5, 0x500)])
.class_with_loader(0x200, 0x100, 0x300, &[], &[])
.class(0x400, 0, &[], &[])
.class(0x800, 0, &[], &[])
.instance(0x300, 0x400, &[])
.instance(0x500, 0x400, &[])
.instance(0x600, 0x200, &[])
.obj_array(0x700, 0x800, &[0, 0x900])
.instance(0x900, 0x400, &[])
.root(HeapRootKind::JniGlobal, 0x600)
.root(HeapRootKind::JniGlobal, 0x700);
w.finish()
}
#[test]
fn path_names_statics_supers_loaders_classes_and_elements() {
let index = analyzed_bytes(&class_edges_heap());
let k = ("0x600".to_owned(), root(HeapRootKind::JniGlobal, None));
assert_eq!(
path(&index, 0x500),
Some(vec![
("0x500".to_owned(), HeapEdge::Static { name: "cache".to_owned() }),
("0x100".to_owned(), HeapEdge::SuperClass),
("0x200".to_owned(), HeapEdge::ClassOf),
k.clone(),
])
);
assert_eq!(
path(&index, 0x300),
Some(vec![("0x300".to_owned(), HeapEdge::ClassLoader), ("0x200".to_owned(), HeapEdge::ClassOf), k])
);
assert_eq!(
path(&index, 0x900),
Some(vec![
("0x900".to_owned(), HeapEdge::Element { index: 1 }),
("0x700".to_owned(), root(HeapRootKind::JniGlobal, None))
])
);
}
fn with_main_thread(mut heap: MiniHeap, name: impl FnOnce(&mut MiniHeap, u64)) -> MiniHeap {
heap.class("java.lang.Thread", Some("java.lang.Object"), &[("name", BasicType::Object)]);
name(&mut heap, 0x2000);
heap.object(0x2100, "java.lang.Thread", &[("name", 0x2000)]).root_in_thread(HeapRootKind::ThreadObject, 0x2100, 1);
heap
}
#[test]
fn thread_name_on_java_frame_root() {
let heap = with_main_thread(sample_with_string(), |h, at| {
h.string(at, "main", true);
});
let index = analyzed(&heap);
let steps = path(&index, L).unwrap();
assert_eq!(steps.last().unwrap().1, root(HeapRootKind::JavaFrame, Some("main")));
// A JNI global carries no thread.
assert_eq!(path(&index, H).unwrap()[0].1, root(HeapRootKind::JniGlobal, None));
}
#[test]
fn jdk8_thread_name_is_a_char_array() {
let heap = with_main_thread(sample(), |h, at| {
let utf16: Vec<u8> = "main".encode_utf16().flat_map(u16::to_be_bytes).collect();
h.prim_bytes(at, BasicType::Char, &utf16);
});
let index = analyzed(&heap);
assert_eq!(path(&index, N3).unwrap()[0].1, root(HeapRootKind::JavaFrame, Some("main")));
}
#[test]
fn page_limit_is_clamped() {
let mut heap = MiniHeap::new();
heap.class("Node", Some("java.lang.Object"), &[("next", BasicType::Object)]);
for k in 0..600u64 {
let addr = 0x10_0000 + k * 0x100;
heap.object(addr, "Node", &[]).root(HeapRootKind::JniGlobal, addr);
}
let index = analyzed(&heap);
let node = slot(&index, "Node");
let page = index.dominated(None, Some(node), 0, 10_000);
assert_eq!((page.rows.len(), page.total), (MAX_PAGE as usize, 600));
let top = index.dominated(None, None, 0, 10_000);
assert_eq!((top.rows.len(), top.total), (MAX_PAGE as usize, 600));
assert_eq!(index.dominated(None, None, 550, 10_000).rows.len(), 50);
for (offset, limit) in [(u32::MAX, u32::MAX), (u32::MAX - 10, 100), (599, u32::MAX)] {
let page = index.dominated(None, Some(node), offset, limit);
assert_eq!((page.rows.len(), page.total), (usize::from(offset == 599), 600), "{offset} {limit}");
assert!(index.dominated(None, None, offset, limit).rows.len() <= 1);
}
}
#[test]
fn class_rows_skip_empty_slots() {
let index = analyzed(&sample());
let rows = index.class_rows();
let node = rows.iter().find(|r| r.name == "Node").unwrap();
assert_eq!(
(node.class_index, node.count, node.shallow, node.retained, node.unreachable_count, node.ambiguous),
(slot(&index, "Node"), 3, 72, 104, 1, false)
);
assert!(rows.iter().all(|r| r.count + r.unreachable_count > 0));
assert!(!rows.iter().any(|r| r.name == "boolean[]"));
assert!(rows.windows(2).all(|p| p[0].retained >= p[1].retained));
assert_eq!(index.summary().objects, 8);
}
#[test]
fn class_rows_mark_names_of_several_classes() {
let mut heap = sample();
// `object` takes the class defined last under a name, so each Foo gets one instance.
heap.class("Foo", Some("java.lang.Object"), &[])
.object(0x3000, "Foo", &[])
.class("Foo", Some("java.lang.Object"), &[])
.object(0x3100, "Foo", &[])
.object(0x3200, "Foo", &[]);
let index = analyzed(&heap);
let rows = index.class_rows();
let foos: Vec<_> = rows.iter().filter(|r| r.name == "Foo").collect();
assert_eq!(foos.len(), 2);
assert!(foos.iter().all(|r| r.ambiguous));
let mut counts: Vec<u64> = foos.iter().map(|r| r.unreachable_count).collect();
counts.sort_unstable();
assert_eq!(counts, [1, 2]);
assert!(rows.iter().filter(|r| r.name != "Foo").all(|r| !r.ambiguous));
}
#[test]
fn ipc_shapes() {
assert_eq!(json(&HeapObjectRef(7)), "7");
assert_eq!(json(&HeapEdge::ClassOf), r#"{"kind":"classOf"}"#);
assert_eq!(json(&HeapEdge::Element { index: 2 }), r#"{"kind":"element","index":2}"#);
assert_eq!(
json(&root(HeapRootKind::JavaFrame, Some("main"))),
r#"{"kind":"root","root":"javaFrame","thread":"main"}"#
);
let row = HeapClassRow {
class_index: 1,
name: "A".to_owned(),
count: 2,
shallow: 3,
retained: 4,
unreachable_count: 5,
ambiguous: false,
};
assert_eq!(
json(&row),
r#"{"classIndex":1,"name":"A","count":2,"shallow":3,"retained":4,"unreachableCount":5,"ambiguous":false}"#
);
let back: HeapObjectRef = serde_json::from_str("4294967295").unwrap();
assert_eq!(back, HeapObjectRef(u32::MAX));
}
fn json(value: &impl serde::Serialize) -> String {
serde_json::to_string(value).unwrap()
}
+44
Переглянути файл
@@ -0,0 +1,44 @@
//! HotSpot `.hprof` heap-dump reading and analysis for the debugger's JVM Memory view.
mod analysis;
mod column;
mod compare;
mod disk;
mod dominators;
mod error;
mod files;
mod graph;
mod hprof;
mod ids;
mod index;
mod names;
mod progress;
mod reader;
mod registry;
mod shallow;
mod strings;
#[cfg(test)]
mod test_index;
/// Fixture builders shared by this crate's tests and by dependants' tests and benches; not API.
#[doc(hidden)]
pub mod testing;
pub use analysis::{AnalyzeOptions, analyze};
pub use column::{Column, NONE, Pod, Store};
pub use compare::{HeapClassDiff, compare};
pub use disk::{DISK_MARGIN, capture_need, create_snapshot_dir, free_bytes, index_need, remove_stale, snapshots_root};
pub use dominators::{Dominators, dominators};
pub use error::HeapError;
pub use graph::{ClassInfo, HeapGraph, PRIM_SLOTS, build_graph};
pub use hprof::{
BasicType, ClassDump, HeapRecord, HeapRootKind, HprofHeader, Record, parse_header, record_at, scan, scan_segment,
};
pub use ids::{IdMap, IdMapBuilder};
pub use index::{
ClassStat, HeapClassRow, HeapEdge, HeapIndex, HeapObjectPage, HeapObjectRef, HeapObjectRow, HeapPathStep,
HeapRootPath, HeapSummary, MAX_PAGE,
};
pub use names::display_name;
pub use progress::HeapPhase;
pub use registry::{HeapOrigin, HeapRegistry, HeapSnapshotId, HeapSnapshotInfo, OpenSnapshot};
pub use shallow::{HistogramEntry, ShallowModel, SizeSource, VmLayout};
+66
Переглянути файл
@@ -0,0 +1,66 @@
//! Class names in the form the JVM's class histogram and the UI use.
/// The Java source form of an hprof class name (`java/util/HashMap$Node`, `[Ljava/lang/String;`),
/// as `ide_jvm_attach::display_class_name` writes it, plus `+0x` → `/0x` for hidden classes,
/// which the histogram spells with a slash. Names that already use dots pass through unchanged.
pub fn display_name(hprof_name: &str) -> String {
let dotted = hprof_name.replace('/', ".");
let dims = dotted.bytes().take_while(|&b| b == b'[').count();
let shown = if dims == 0 { dotted } else { array_name(&dotted, dims) };
// After the slash replacement, or the new slash would become a dot.
shown.replace("+0x", "/0x")
}
/// `[[I` → `int[][]`; a descriptor that is not one stays as it is.
fn array_name(dotted: &str, dims: usize) -> String {
let element = &dotted[dims..];
let base = match element {
"Z" => "boolean",
"B" => "byte",
"C" => "char",
"S" => "short",
"I" => "int",
"J" => "long",
"F" => "float",
"D" => "double",
_ => match element.strip_prefix('L').and_then(|e| e.strip_suffix(';')) {
Some(class) => class,
None => return dotted.to_owned(),
},
};
format!("{base}{}", "[]".repeat(dims))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn display_names() {
for (hprof, shown) in [
("[Z", "boolean[]"),
("[J", "long[]"),
("[[D", "double[][]"),
("[Ljava/lang/String;", "java.lang.String[]"),
("java/util/HashMap$Node", "java.util.HashMap$Node"),
("Gen$$Lambda+0x0000007800000a00", "Gen$$Lambda/0x0000007800000a00"),
] {
assert_eq!(display_name(hprof), shown, "{hprof}");
}
}
#[test]
fn dotted_names_and_hidden_class_arrays() {
for (hprof, shown) in [
("java.lang.String", "java.lang.String"),
("[Ljava.lang.Object;", "java.lang.Object[]"),
("[[Lpkg.Gen$$Lambda+0x0000007800000a00;", "pkg.Gen$$Lambda/0x0000007800000a00[][]"),
("[Ljava.lang.String", "[Ljava.lang.String"),
("[Q", "[Q"),
("0x7f00", "0x7f00"),
("", ""),
] {
assert_eq!(display_name(hprof), shown, "{hprof}");
}
}
}
+103
Переглянути файл
@@ -0,0 +1,103 @@
//! Phases of a heap analysis and the progress/cancellation checks every long pass makes.
use ide_core::CancellationToken;
use serde::{Deserialize, Serialize};
use crate::error::HeapError;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub enum HeapPhase {
Reading,
Linking,
Dominators,
Retained,
Paths,
Saving,
}
/// Longest stretch of work between two progress reports (and looks at the cancellation token).
pub(crate) const CHECK_EVERY: u64 = 1 << 20;
/// The caller's progress callback plus the token, looked at right after every report so a callback
/// that cancels stops the work before the next step.
pub(crate) struct Progress<'a> {
report: &'a dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &'a CancellationToken,
}
impl<'a> Progress<'a> {
pub(crate) fn new(
report: &'a dyn Fn(HeapPhase, u64, u64) -> Result<(), HeapError>,
cancel: &'a CancellationToken,
) -> Self {
Self { report, cancel }
}
pub(crate) fn report(&self, phase: HeapPhase, done: u64, total: u64) -> Result<(), HeapError> {
(self.report)(phase, done, total)?;
self.check()
}
pub(crate) fn token(&self) -> &'a CancellationToken {
self.cancel
}
/// Looks at the token without reporting.
pub(crate) fn check(&self) -> Result<(), HeapError> {
if self.cancel.is_cancelled() { Err(HeapError::Cancelled) } else { Ok(()) }
}
/// Looks at the token at step 0 and reports every [`CHECK_EVERY`] steps after it. Step 0 does
/// not report: a loop starts right after its phase's own report.
pub(crate) fn step(&self, step: u64, phase: HeapPhase, done: u64, total: u64) -> Result<(), HeapError> {
if step == 0 {
self.check()
} else if step.is_multiple_of(CHECK_EVERY) {
self.report(phase, done, total)
} else {
Ok(())
}
}
}
#[cfg(test)]
mod tests {
use std::cell::RefCell;
use super::*;
#[test]
fn steps_check_at_zero_and_report_every_check_interval() {
let calls = RefCell::new(Vec::new());
let report = |phase, done, total| {
calls.borrow_mut().push((phase, done, total));
Ok(())
};
let cancel = CancellationToken::new();
let progress = Progress::new(&report, &cancel);
for step in [0, 1, CHECK_EVERY - 1, CHECK_EVERY, CHECK_EVERY + 1, 3 * CHECK_EVERY] {
progress.step(step, HeapPhase::Linking, step, 9).unwrap();
}
assert_eq!(*calls.borrow(), [(HeapPhase::Linking, CHECK_EVERY, 9), (HeapPhase::Linking, 3 * CHECK_EVERY, 9)]);
cancel.cancel();
assert!(matches!(progress.step(0, HeapPhase::Linking, 0, 9), Err(HeapError::Cancelled)));
assert!(matches!(progress.report(HeapPhase::Saving, 1, 1), Err(HeapError::Cancelled)));
assert_eq!(calls.borrow().last(), Some(&(HeapPhase::Saving, 1, 1)), "reports before it looks at the token");
}
#[test]
fn report_errors_pass_through() {
let report = |_, _, _| Err(HeapError::NotEnoughDiskSpace { needed: 2, available: 1 });
let cancel = CancellationToken::new();
let progress = Progress::new(&report, &cancel);
assert!(matches!(progress.report(HeapPhase::Linking, 0, 1), Err(HeapError::NotEnoughDiskSpace { .. })));
}
#[test]
fn phases_serialize_in_camel_case() {
assert_eq!(serde_json::to_string(&HeapPhase::Dominators).unwrap(), "\"dominators\"");
assert_eq!(serde_json::from_str::<HeapPhase>("\"saving\"").unwrap(), HeapPhase::Saving);
}
}
+92
Переглянути файл
@@ -0,0 +1,92 @@
use crate::error::HeapError;
/// Big-endian cursor over a dump. Every read is bounds-checked; running off `buf` yields
/// `Truncated` at `buf.len()`, so callers bound a record by slicing `buf` to its end.
pub(crate) struct Reader<'a> {
buf: &'a [u8],
pos: usize,
id_size: u8,
}
impl<'a> Reader<'a> {
pub(crate) fn new(buf: &'a [u8], pos: usize, id_size: u8) -> Self {
Self { buf, pos, id_size }
}
pub(crate) fn pos(&self) -> usize {
self.pos
}
pub(crate) fn id_size(&self) -> u8 {
self.id_size
}
pub(crate) fn buf(&self) -> &'a [u8] {
self.buf
}
pub(crate) fn bytes(&mut self, n: usize) -> Result<&'a [u8], HeapError> {
let end = self.pos.checked_add(n).filter(|&end| end <= self.buf.len());
let Some(end) = end else {
return Err(HeapError::Truncated { at: self.buf.len() as u64 });
};
let slice = &self.buf[self.pos..end];
self.pos = end;
Ok(slice)
}
pub(crate) fn skip(&mut self, n: usize) -> Result<(), HeapError> {
self.bytes(n).map(|_| ())
}
fn array<const N: usize>(&mut self) -> Result<[u8; N], HeapError> {
let mut out = [0u8; N];
out.copy_from_slice(self.bytes(N)?);
Ok(out)
}
pub(crate) fn u1(&mut self) -> Result<u8, HeapError> {
Ok(self.array::<1>()?[0])
}
pub(crate) fn u2(&mut self) -> Result<u16, HeapError> {
Ok(u16::from_be_bytes(self.array()?))
}
pub(crate) fn u4(&mut self) -> Result<u32, HeapError> {
Ok(u32::from_be_bytes(self.array()?))
}
pub(crate) fn u8(&mut self) -> Result<u64, HeapError> {
Ok(u64::from_be_bytes(self.array()?))
}
/// `id_size` is 4 or 8; the public entry points reject anything else before building a reader.
pub(crate) fn id(&mut self) -> Result<u64, HeapError> {
if self.id_size == 4 { self.u4().map(u64::from) } else { self.u8() }
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_big_endian_and_ids_of_both_sizes() {
let buf = [1, 0, 2, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 5];
let mut r = Reader::new(&buf, 0, 8);
assert_eq!((r.u1().unwrap(), r.u2().unwrap(), r.u4().unwrap(), r.u8().unwrap()), (1, 2, 3, 4));
assert_eq!(r.pos(), 15);
assert_eq!(Reader::new(&buf[15..], 0, 4).id().unwrap(), 5);
}
#[test]
fn running_off_the_end_reports_the_buffer_end() {
let buf = [0u8; 6];
let mut r = Reader::new(&buf, 4, 8);
assert!(matches!(r.u4(), Err(HeapError::Truncated { at: 6 })));
assert_eq!(r.pos(), 4, "a failed read does not advance");
assert!(matches!(r.skip(usize::MAX), Err(HeapError::Truncated { at: 6 })));
assert!(matches!(Reader::new(&buf, 99, 8).u1(), Err(HeapError::Truncated { at: 6 })));
}
}
+163
Переглянути файл
@@ -0,0 +1,163 @@
//! The heap snapshots that are open in this process.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
use ide_core::process::process_alive;
use ide_core::{DebugSessionId, IpcPath};
use serde::{Deserialize, Serialize};
use crate::disk::remove_stale;
use crate::index::{HeapIndex, HeapSummary};
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize, specta::Type)]
#[serde(transparent)]
pub struct HeapSnapshotId(pub u32);
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(tag = "kind", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum HeapOrigin {
/// Dumped from a live debug session.
Captured { session: DebugSessionId, pid: u32, java_version: String, taken_while_suspended: bool, dump_ms: u32 },
/// An `.hprof` the user opened; it is never deleted.
File { path: IpcPath },
}
impl HeapOrigin {
/// Text identifying the snapshot in the UI: the process id of a capture, the file name of a
/// file. The capture time is `created_ms`, which the UI formats in local time.
pub fn label(&self) -> String {
match self {
HeapOrigin::Captured { pid, .. } => pid.to_string(),
HeapOrigin::File { path } => path.file_name().unwrap_or(path.as_os_str()).to_string_lossy().into_owned(),
}
}
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct HeapSnapshotInfo {
pub id: HeapSnapshotId,
pub label: String,
pub origin: HeapOrigin,
pub summary: HeapSummary,
/// Unix time in milliseconds when the snapshot was opened.
pub created_ms: u64,
}
/// An analyzed snapshot together with the directory that holds its files. Dropping it unmaps the
/// index and then removes the directory, dump included for a capture; a dump opened from a file
/// is outside the directory and stays.
pub struct OpenSnapshot {
pub info: HeapSnapshotInfo,
/// `None` only while dropping, so the mappings are gone before the directory is deleted
/// (Windows cannot delete a mapped file).
index: Option<HeapIndex>,
/// Holds the index files and the dump of a capture.
pub dir: PathBuf,
pub dump: PathBuf,
/// Set by [`HeapRegistry::insert`] once `dir` is known to be one of the registry's own.
remove_dir: bool,
}
impl OpenSnapshot {
/// The directory is not removed on drop until the snapshot is in a [`HeapRegistry`] whose
/// root holds it.
pub fn new(info: HeapSnapshotInfo, index: HeapIndex, dir: PathBuf, dump: PathBuf) -> Self {
Self { info, index: Some(index), dir, dump, remove_dir: false }
}
pub fn index(&self) -> &HeapIndex {
self.index.as_ref().expect("the index is only taken while dropping")
}
}
impl Drop for OpenSnapshot {
fn drop(&mut self) {
drop(self.index.take());
if !self.remove_dir {
return;
}
// The user's own dump that happens to sit inside `dir` (an opened file is analyzed from a
// private copy, which goes with the directory).
if matches!(&self.info.origin, HeapOrigin::File { path } if **path == self.dump)
&& self.dump.starts_with(&self.dir)
{
return;
}
// A link is not followed: only a real directory is deleted.
if std::fs::symlink_metadata(&self.dir).is_ok_and(|meta| meta.file_type().is_dir())
&& let Err(err) = std::fs::remove_dir_all(&self.dir)
{
tracing::warn!(dir = %self.dir.display(), %err, "could not remove a heap snapshot directory");
}
}
}
struct State {
next_id: u32,
open: BTreeMap<u32, Arc<OpenSnapshot>>,
}
/// The open snapshots, shared between threads. Ids are never reused within a process.
pub struct HeapRegistry {
root: PathBuf,
state: Mutex<State>,
}
impl HeapRegistry {
/// Registry for snapshot directories under `root`; first removes the directories there that
/// belong to processes that no longer exist.
pub fn new(root: PathBuf) -> Self {
remove_stale(&root, process_alive);
Self { root, state: Mutex::new(State { next_id: 1, open: BTreeMap::new() }) }
}
pub fn root(&self) -> &Path {
&self.root
}
fn state(&self) -> MutexGuard<'_, State> {
self.state.lock().unwrap_or_else(PoisonError::into_inner)
}
/// Registers the snapshot that `make` builds for a fresh id. `make` runs outside the lock.
pub fn insert(&self, make: impl FnOnce(HeapSnapshotId) -> OpenSnapshot) -> HeapSnapshotInfo {
let id = {
let mut state = self.state();
let id = state.next_id;
state.next_id = id.saturating_add(1);
HeapSnapshotId(id)
};
let mut snapshot = make(id);
snapshot.info.id = id;
snapshot.remove_dir = snapshot.dir.file_name().is_some() && snapshot.dir.parent() == Some(self.root.as_path());
if !snapshot.remove_dir {
tracing::warn!(dir = %snapshot.dir.display(), "heap snapshot directory is outside the registry root; keeping it");
}
let info = snapshot.info.clone();
self.state().open.insert(id.0, Arc::new(snapshot));
info
}
pub fn get(&self, id: HeapSnapshotId) -> Option<Arc<OpenSnapshot>> {
self.state().open.get(&id.0).cloned()
}
/// Forgets the snapshot; its directory goes once the last query using it lets go. `false` for
/// an id that is not open.
pub fn close(&self, id: HeapSnapshotId) -> bool {
// Bound first so the lock is released before the snapshot may be dropped: that deletes files.
let removed = self.state().open.remove(&id.0);
removed.is_some()
}
/// In the order the snapshots were opened.
pub fn list(&self) -> Vec<HeapSnapshotInfo> {
self.state().open.values().map(|snapshot| snapshot.info.clone()).collect()
}
}
#[cfg(test)]
mod tests;
+265
Переглянути файл
@@ -0,0 +1,265 @@
use std::sync::Arc;
use ide_core::{CancellationToken, DebugSessionId};
use super::*;
use crate::analysis::{AnalyzeOptions, analyze};
use crate::disk::create_snapshot_dir;
use crate::error::HeapError;
use crate::progress::HeapPhase;
use crate::testing::sample;
fn no_progress(_: HeapPhase, _: u64, _: u64) -> Result<(), HeapError> {
Ok(())
}
fn captured() -> HeapOrigin {
HeapOrigin::Captured {
session: DebugSessionId(7),
pid: 4242,
java_version: "21.0.2".into(),
taken_while_suspended: true,
dump_ms: 35,
}
}
fn external(path: &Path) -> HeapOrigin {
HeapOrigin::File { path: IpcPath(path.to_path_buf()) }
}
/// A snapshot whose index files are mapped from `dir`, like a real one: `dump` is analyzed in
/// place and `dir` receives the columns.
fn open_in(dir: &Path, dump: &Path, origin: HeapOrigin) -> impl FnOnce(HeapSnapshotId) -> OpenSnapshot {
let dir = dir.to_path_buf();
let dump = dump.to_path_buf();
move |id| {
std::fs::write(&dump, sample().bytes()).unwrap();
let index = analyze(&dump, &dir, AnalyzeOptions::default(), &no_progress, &CancellationToken::new()).unwrap();
let info = HeapSnapshotInfo {
id,
label: origin.label(),
origin,
summary: index.summary.clone(),
created_ms: 1_700_000_000_000,
};
OpenSnapshot::new(info, index, dir, dump)
}
}
/// Inserts a captured snapshot with its own directory under the registry's root.
fn insert_captured(registry: &HeapRegistry) -> (HeapSnapshotInfo, PathBuf) {
let dir = create_snapshot_dir(registry.root()).unwrap();
let dump = dir.join("dump.hprof");
let info = registry.insert(open_in(&dir, &dump, captured()));
(info, dir)
}
fn names(dir: &Path) -> Vec<String> {
let mut names: Vec<_> =
std::fs::read_dir(dir).unwrap().map(|e| e.unwrap().file_name().into_string().unwrap()).collect();
names.sort();
names
}
#[test]
fn closing_snapshot_removes_dir() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let (info, dir) = insert_captured(&registry);
assert!(dir.join("dump.hprof").exists() && dir.join("summary.json").exists());
assert!(registry.close(info.id));
assert!(!dir.exists());
assert!(!registry.close(info.id));
assert!(registry.get(info.id).is_none());
}
#[test]
fn external_dump_is_never_deleted() {
let root = tempfile::tempdir().unwrap();
let outside = tempfile::tempdir().unwrap();
let dump = outside.path().join("app.hprof");
let registry = HeapRegistry::new(root.path().to_path_buf());
let dir = create_snapshot_dir(registry.root()).unwrap();
let info = registry.insert(open_in(&dir, &dump, external(&dump)));
assert_eq!(info.label, "app.hprof");
assert!(dir.join("summary.json").exists());
assert!(registry.close(info.id));
assert!(!dir.exists());
assert!(dump.exists());
}
#[test]
fn a_dump_inside_the_dir_of_a_file_snapshot_keeps_the_dir() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let dir = create_snapshot_dir(registry.root()).unwrap();
let dump = dir.join("opened.hprof");
let info = registry.insert(open_in(&dir, &dump, external(&dump)));
assert!(registry.close(info.id));
assert!(dump.exists());
}
#[test]
fn a_dir_outside_the_root_is_never_deleted() {
let root = tempfile::tempdir().unwrap();
let elsewhere = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let dump = elsewhere.path().join("dump.hprof");
let info = registry.insert(open_in(elsewhere.path(), &dump, captured()));
assert!(registry.close(info.id));
assert!(dump.exists());
// Neither the root itself nor a path that climbs out of it counts as a snapshot dir.
let sneaky = root.path().join("..").join(elsewhere.path().file_name().unwrap());
let info = registry.insert(open_in(&sneaky, &dump, captured()));
assert!(registry.close(info.id));
assert!(dump.exists());
}
#[cfg(unix)]
#[test]
fn a_symlinked_dir_is_not_followed() {
let root = tempfile::tempdir().unwrap();
let outside = tempfile::tempdir().unwrap();
let dump = outside.path().join("dump.hprof");
let link = root.path().join("link");
std::os::unix::fs::symlink(outside.path(), &link).unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let info = registry.insert(open_in(&link, &dump, captured()));
assert!(registry.close(info.id));
assert!(dump.exists());
}
#[test]
fn ids_are_never_reused() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let (first, _) = insert_captured(&registry);
assert!(registry.close(first.id));
let (second, _) = insert_captured(&registry);
let (third, _) = insert_captured(&registry);
assert!(first.id.0 < second.id.0 && second.id.0 < third.id.0);
}
#[test]
fn the_registry_decides_the_id_in_the_info() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let dir = create_snapshot_dir(registry.root()).unwrap();
let dump = dir.join("dump.hprof");
let make = open_in(&dir, &dump, captured());
let info = registry.insert(|id| {
let mut snapshot = make(id);
snapshot.info.id = HeapSnapshotId(u32::MAX);
snapshot
});
assert_ne!(info.id, HeapSnapshotId(u32::MAX));
assert_eq!(registry.get(info.id).unwrap().info.id, info.id);
}
#[test]
fn list_is_in_opening_order_and_get_shares_the_snapshot() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let (a, _) = insert_captured(&registry);
let (b, _) = insert_captured(&registry);
let (c, _) = insert_captured(&registry);
assert!(registry.close(b.id));
assert_eq!(registry.list(), [a.clone(), c.clone()]);
let one = registry.get(a.id).unwrap();
let two = registry.get(a.id).unwrap();
assert!(Arc::ptr_eq(&one, &two));
assert_eq!(one.info, a);
assert_eq!(one.index().summary, a.summary);
}
#[test]
fn a_snapshot_in_use_keeps_its_dir_until_the_last_holder_lets_go() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let (info, dir) = insert_captured(&registry);
let held = registry.get(info.id).unwrap();
assert!(registry.close(info.id));
assert!(registry.get(info.id).is_none() && registry.list().is_empty());
assert!(dir.exists());
drop(held);
assert!(!dir.exists());
}
#[test]
fn dropping_the_registry_removes_the_open_snapshots() {
let root = tempfile::tempdir().unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
let (_, dir) = insert_captured(&registry);
drop(registry);
assert!(!dir.exists());
assert!(names(root.path()).is_empty());
}
#[cfg(unix)]
#[test]
fn new_clears_dirs_whose_owner_died() {
let root = tempfile::tempdir().unwrap();
let mut child = std::process::Command::new("true").spawn().unwrap();
let dead = child.id();
child.wait().unwrap();
let dead_dir = root.path().join("dead");
std::fs::create_dir(&dead_dir).unwrap();
std::fs::write(dead_dir.join("owner"), dead.to_string()).unwrap();
let live_dir = create_snapshot_dir(root.path()).unwrap();
let registry = HeapRegistry::new(root.path().to_path_buf());
assert!(!dead_dir.exists());
assert!(live_dir.exists());
assert_eq!(registry.root(), root.path());
}
#[test]
fn concurrent_inserts_get_distinct_ids() {
let root = tempfile::tempdir().unwrap();
let registry = Arc::new(HeapRegistry::new(root.path().to_path_buf()));
let handles: Vec<_> = (0..6)
.map(|_| {
let registry = registry.clone();
std::thread::spawn(move || (0..3).map(|_| insert_captured(&registry).0.id).collect::<Vec<_>>())
})
.collect();
let mut ids: Vec<_> = handles.into_iter().flat_map(|h| h.join().unwrap()).collect();
ids.sort_by_key(|id| id.0);
ids.dedup();
assert_eq!(ids.len(), 18);
assert_eq!(registry.list().len(), 18);
for id in ids {
assert!(registry.close(id));
}
assert!(names(root.path()).is_empty());
}
#[test]
fn labels_name_the_pid_or_the_file() {
assert_eq!(captured().label(), "4242");
assert_eq!(external(Path::new("/tmp/dumps/app.hprof")).label(), "app.hprof");
assert_eq!(external(Path::new("/")).label(), "/");
}
#[test]
fn origin_and_info_serialize_as_the_ui_expects() {
let value = serde_json::to_value(captured()).unwrap();
assert_eq!(
value,
serde_json::json!({
"kind": "captured", "session": 7, "pid": 4242, "javaVersion": "21.0.2",
"takenWhileSuspended": true, "dumpMs": 35
})
);
assert_eq!(serde_json::to_value(external(Path::new("/a/b.hprof"))).unwrap()["path"], "/a/b.hprof");
assert_eq!(serde_json::to_value(HeapSnapshotId(3)).unwrap(), serde_json::json!(3));
let back: HeapOrigin = serde_json::from_value(value).unwrap();
assert_eq!(back, captured());
}
// `HeapRegistry::get` hands out `Arc<OpenSnapshot>` to query threads.
const _: fn() = || {
fn check<T: Send + Sync>() {}
check::<HeapIndex>();
check::<OpenSnapshot>();
check::<HeapRegistry>();
};
+265
Переглянути файл
@@ -0,0 +1,265 @@
//! Shallow sizes: the bytes each object takes in the JVM heap, from the object layout rules, the
//! JVM's own class histogram, or the distances between object addresses in the dump.
mod layout;
mod mirrors;
use std::collections::HashMap;
use ide_core::CancellationToken;
use serde::{Deserialize, Serialize};
pub use self::layout::VmLayout;
use self::mirrors::{Mirrors, injected_by_table};
use crate::error::HeapError;
use crate::graph::{ClassInfo, HeapGraph};
use crate::hprof::BasicType;
use crate::names::display_name;
use crate::progress::CHECK_EVERY;
/// How far past the layout size an address gap may be and still count as an object size.
const GAP_SLACK: u64 = 64;
/// Share of the instance types that can have a gap (two or more instances), in tenths, that must
/// take their size from address gaps for the sizes to be reported as coming from them.
const GAP_TRUST_TENTHS: usize = 9;
/// Where the sizes of instances came from.
///
/// Class objects (mirrors) are approximate in every case: HotSpot leaves the fields it injects
/// into `java.lang.Class` out of the dump, so their layout size is short by a fixed amount per
/// mirror. With a histogram that amount is fitted to its `java.lang.Class` row; otherwise (or when
/// the fit is rejected) it comes from a table measured per JDK and reference size, within a few
/// bytes per mirror, or is 32 when the JDK is unknown.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub enum SizeSource {
/// The JVM's class histogram from the capture.
Jvm,
/// The smallest distance between neighbouring objects of a type, in a dump the IDE did not
/// capture.
AddressGaps,
Layout,
}
/// A row of the JVM's class histogram.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct HistogramEntry {
/// Display form: `java.lang.String`, `int[]`, `Gen$$Lambda/0x…`.
pub name: String,
pub instances: u64,
pub bytes: u64,
}
enum TypeSize {
Fixed(u64),
ObjectArray,
/// Class objects are sized one by one; this is for an instance of `java.lang.Class` that is
/// not a class object.
ClassObject(u64),
}
/// Shallow size of every object of a [`HeapGraph`]. Instances of a type share one size; arrays
/// follow the layout; class objects add their statics to the size of `java.lang.Class`.
pub struct ShallowModel {
pub source: SizeSource,
layout: VmLayout,
/// Per type slot of a class; primitive array slots follow and have no entry.
types: Vec<TypeSize>,
/// `(node, size)` of each class object, by node.
class_objects: Vec<(u32, u64)>,
/// Per class slot: another class has the same display name.
ambiguous: Vec<bool>,
}
impl ShallowModel {
/// `histogram` is the JVM's class histogram of the same stop as the dump, when the IDE captured
/// it. Without it, sizes come from address gaps where they agree with the layout.
///
/// A name shared by several classes (several class loaders) never takes a histogram size,
/// since the name is the only link between the dump and the histogram; see
/// [`Self::is_ambiguous`]. `java.lang.Class` never does either, because mirror sizes vary with
/// their statics; the histogram only tells how many bytes HotSpot adds to each mirror.
pub fn new(
graph: &HeapGraph,
layout: VmLayout,
histogram: Option<&[HistogramEntry]>,
cancel: &CancellationToken,
) -> Result<Self, HeapError> {
if cancel.is_cancelled() {
return Err(HeapError::Cancelled);
}
let classes = &graph.classes;
let field_bytes = field_bytes(classes, layout);
let by_layout: Vec<u64> = field_bytes.iter().map(|&bytes| layout.instance_size(bytes)).collect();
let names: Vec<String> = classes.iter().map(|c| display_name(&c.name)).collect();
let ambiguous = ambiguous_names(&names);
let class_slot = graph.class_slot as usize;
let mirrors = Mirrors::collect(graph, by_layout[class_slot], layout);
let mut fitted = None;
if let Some(rows) = histogram
&& !ambiguous[class_slot]
{
let mut class_rows = rows.iter().filter(|row| row.name == names[class_slot]);
if let (Some(row), None) = (class_rows.next(), class_rows.next()) {
fitted = mirrors.fit_injected(graph, layout, by_layout[class_slot], row, cancel)?;
}
}
let injected = fitted.unwrap_or_else(|| injected_by_table(layout));
let mut types: Vec<TypeSize> = classes
.iter()
.enumerate()
.map(|(slot, class)| {
if class.name.starts_with('[') {
TypeSize::ObjectArray
} else if slot == class_slot {
TypeSize::ClassObject(layout.align(by_layout[slot] + injected))
} else {
TypeSize::Fixed(by_layout[slot])
}
})
.collect();
let source = match histogram {
Some(rows) => {
apply_histogram(&mut types, &names, &ambiguous, rows);
SizeSource::Jvm
}
None => apply_address_gaps(&mut types, graph, &by_layout, cancel)?,
};
Ok(Self { source, layout, types, class_objects: mirrors.sizes(layout, injected), ambiguous })
}
/// Shallow size of `object` (a node of the graph); 0 for the super-root.
pub fn size_of(&self, graph: &HeapGraph, object: u32) -> u64 {
let Some(&slot) = graph.types.get(object as usize) else { return 0 };
match self.types.get(slot as usize) {
Some(&TypeSize::Fixed(size)) => size,
Some(TypeSize::ObjectArray) => self.array_size(BasicType::Object, graph.lens[object as usize]),
Some(&TypeSize::ClassObject(fallback)) => self
.class_objects
.binary_search_by_key(&object, |&(node, _)| node)
.map_or(fallback, |at| self.class_objects[at].1),
None => match graph.prim_type(slot) {
Some(ty) => self.array_size(ty, graph.lens[object as usize]),
None => 0,
},
}
}
/// Whether another class has the same display name as the class in `slot`. Sizes of such
/// classes come from the layout, and anything joined to them by name (histogram rows, a
/// second snapshot) cannot tell them apart.
pub fn is_ambiguous(&self, slot: u32) -> bool {
self.ambiguous.get(slot as usize).copied().unwrap_or(false)
}
fn array_size(&self, ty: BasicType, len: u32) -> u64 {
self.layout.array_size(ty, u64::from(len))
}
}
/// Bytes of instance fields per class, its superclasses included. Super chains always end
/// (`ClassInfo::super_index` has its cycles cut).
fn field_bytes(classes: &[ClassInfo], layout: VmLayout) -> Vec<u64> {
let mut total: Vec<Option<u64>> = vec![None; classes.len()];
let mut path = Vec::new();
for start in 0..classes.len() {
let mut cursor = Some(start);
let mut below = 0;
while let Some(c) = cursor {
if let Some(known) = total[c] {
below = known;
break;
}
path.push(c);
cursor = classes[c].super_index.map(|s| s as usize);
}
while let Some(c) = path.pop() {
let own: u64 = classes[c].fields.iter().map(|(_, ty)| ty.heap_size(layout.ref_size)).sum();
below += own;
total[c] = Some(below);
}
}
total.into_iter().map(Option::unwrap_or_default).collect()
}
fn ambiguous_names(names: &[String]) -> Vec<bool> {
let mut counts: HashMap<&str, u32> = HashMap::with_capacity(names.len());
for name in names {
*counts.entry(name).or_default() += 1;
}
names.iter().map(|name| counts[name.as_str()] > 1).collect()
}
/// `bytes / instances` for each plain class with a histogram row of its name. A name that is
/// ambiguous in the dump or repeated in the histogram, or a row that is not a whole number of
/// objects, keeps its layout size.
fn apply_histogram(types: &mut [TypeSize], names: &[String], ambiguous: &[bool], rows: &[HistogramEntry]) {
let mut per_object: HashMap<&str, Option<u64>> = HashMap::with_capacity(rows.len());
for row in rows {
let size = row.bytes.checked_div(row.instances).filter(|&s| s > 0 && row.bytes % row.instances == 0);
per_object.entry(&row.name).and_modify(|seen| *seen = None).or_insert(size);
}
for (slot, ty) in types.iter_mut().enumerate() {
if let TypeSize::Fixed(size) = ty
&& !ambiguous[slot]
&& let Some(&Some(from_histogram)) = per_object.get(names[slot].as_str())
{
*size = from_histogram;
}
}
}
/// Takes the smallest distance from an instance to the next object in address order as the size
/// of its type, for types with two or more instances where that distance is no smaller than the
/// layout size and at most [`GAP_SLACK`] bigger. The result says [`SizeSource::AddressGaps`] when
/// nine in ten of the types with two or more instances did; types with one instance have no gap
/// to measure, so they neither count for nor against.
fn apply_address_gaps(
types: &mut [TypeSize],
graph: &HeapGraph,
by_layout: &[u64],
cancel: &CancellationToken,
) -> Result<SizeSource, HeapError> {
let slots = types.len();
let mut instances = vec![0u32; slots];
let mut smallest_gap = vec![u64::MAX; slots];
let mut ids = graph.ids.ids();
let mut current = ids.next();
let mut object = 0usize;
while let Some(id) = current {
if (object as u64).is_multiple_of(CHECK_EVERY) && cancel.is_cancelled() {
return Err(HeapError::Cancelled);
}
let next = ids.next();
let slot = graph.types[object] as usize;
if matches!(types.get(slot), Some(TypeSize::Fixed(_))) {
instances[slot] = instances[slot].saturating_add(1);
if let Some(next) = next.filter(|&n| n > id) {
smallest_gap[slot] = smallest_gap[slot].min(next - id);
}
}
current = next;
object += 1;
}
let (mut measurable, mut by_gap) = (0usize, 0usize);
for (slot, ty) in types.iter_mut().enumerate() {
let TypeSize::Fixed(size) = ty else { continue };
if instances[slot] < 2 {
continue;
}
measurable += 1;
let gap = smallest_gap[slot];
if (by_layout[slot]..=by_layout[slot] + GAP_SLACK).contains(&gap) {
*size = gap;
by_gap += 1;
}
}
let trusted = measurable > 0 && by_gap * 10 >= measurable * GAP_TRUST_TENTHS;
Ok(if trusted { SizeSource::AddressGaps } else { SizeSource::Layout })
}
#[cfg(test)]
mod tests;
Loaded 100 of 853 files, more files were not shown because too many files have changed in this diff. Show more