Порівняти коміти

...
Автор SHA1 Повідомлення Дата
l0ng-ai 564f08b062 test(i18n): allow "Python {short}" to read the same in every locale
The conda Python version picker labels its entries "Python 3.12", which every locale writes like English.
2026-11-13 19:11:53 +08:00
Mr0Bread 401d7a4105 fix(database): Tab goes from a result tab to the grid's first cell (#3130)
Tab on a database result tab stepped through every toolbar control (7 to 11 stops) before reaching the grid, and the grid then had no active cell. Now:
- a new WAI-ARIA Toolbar (src/ui/Toolbar.tsx) makes the result, plan, Output and pager toolbars one Tab stop with roving focus on arrows/Home/End;
- Tabs gets an optional focusContent prop, called on plain Tab, so Tab from a result tab focuses the grid (or plan tree / Output log);
- the grid selects its first cell when focused from the keyboard, Tab / Shift+Tab move between cells row by row and leave the grid past either end, and Tab in the cell editor commits and moves on.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 19:05:47 +08:00
Mr0Bread 46bd1f6fb8 fix(ui): accent the selected tab only in the part that has focus (#3124)
The editor, tool windows and database results painted their selected tab with the same accent, so it wasn't clear which part had focus. A focus-scope tracker now marks the editor pane, each tool window and the results view that contain the focused element; outside the focused scope the selected tab uses the existing --bg-tab-selected-inactive / --border-tab-selected-inactive tokens. Scopes are opt-in, so tab strips in dialogs and popups stay accented, and focus moving into a popup, menu or dialog keeps the current marks. IntelliJ theme import maps the inactive underlined tab colours to the same tokens.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 19:05:26 +08:00
Mr0Bread 85f755591b feat(database): Expand All in the Database tool window header (#3140)
The Database tool window header had Collapse All but no Expand All. Add Expand All (db.expandAll) next to it, also reachable from Find Action. It opens every node whose children are already loaded and never connects to a data source or runs a query, so unconnected sources and tables without loaded columns stay closed.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 19:05:14 +08:00
Shohruh Ismoilovтаshohruh-genius 06b7513210 fix(vcs): keep Jump to Source current in focused diffs (#3137)
Pressing Cmd+Down or F4 in a focused revision diff did nothing when the working file was found after focus entered the pane: the keyboard dispatcher kept the navigator object registered before that lookup, while the diff replaced it. The diff now keeps the registered navigator and refreshes its handlers in place, so Jump to Source opens the working file at the mapped caret.

Co-authored-by: shohruh-genius <[email protected]>
2026-11-13 19:04:56 +08:00
l0ng-ai 09aa03bb86 fix(debugger): keep the frames | variables split, put the views after the toolbar, badge live sessions (#3144)
The divider between frames and variables is now remembered across debug starts; the Debug tool window's toolbar leads with the session actions, followed by the Threads & Variables / Console / Memory tabs; and the Debug stripe button shows how many debug sessions are live, as Run does.

Refs #3112
2026-11-13 18:58:27 +08:00
l0ng-ai 317988cfa7 fix(python): refresh the interpreter page after recreating an environment, pick a conda env's Python version (#3143)
Creating an environment at the path the project interpreter already pointed to left the Python Interpreter page saying it does not exist: the package list and the validity check reloaded only when the path changed. Both now reload after an environment is created. A new conda environment now asks for a Python version instead of a base interpreter chosen among existing environments, which read as copying one; the note says it starts with only Python.

Refs #3109
2026-11-13 18:58:22 +08:00
l0ng-ai 7db2e98269 fix(project-view): offer Git and Find Usages in workspace members outside the workspace folder (#3142)
The Project view's context menu counted a node as part of the project only inside the window's root, which in a workspace is the workspace folder. Members outside it lost the Git submenu and Find Usages, although their repositories are discovered and their changes listed. Those entries now accept nodes inside the root or any workspace member; Mark Directory as stays limited to the root, whose project structure it edits, and Scratches stay outside.

Fixes #3111
2026-11-13 18:55:42 +08:00
l0ng-ai fdc8bc2d4c fix(editor): leave reference counts to Code Vision's usages (#3139)
csharp-ls adds a "3 Reference(s)" lens above each declaration, next to Code Vision's own "3 usages". It duplicated the count, ignored the Code Vision settings, and was a second set of lens rows replaced after every edit. Language-server lenses that are only a reference count are now left out; Settings › Editor › Inlay Hints › Code Vision › Usages governs the count.

Refs #3107
2026-11-13 18:41:34 +08:00
l0ng-ai 497d4766d7 fix(vcs): lay out a merge commit's file side by side, each pane scrolling (#3138)
Opening a file of a merge commit from the Git log stacked its three panes in a column the editor could not scroll: only the first parent's text showed, without the changes, and neither the wheel nor the arrow keys moved it. The panes now sit side by side under a label row, each scrolls and they stay in step.

Fixes #3108
2026-11-13 18:38:56 +08:00
l0ng-ai 7e11d0729b fix(macos): keep full screen when Escape goes unhandled (#3134)
Escape the page leaves unhandled reaches the window as cancelOperation:, and NSWindow leaves full screen on it, so pressing Escape once more after closing a popup dropped the IDE out of full screen. The IDE's windows now answer cancelOperation: with nothing; View › Appearance › Toggle Full Screen (⌃⌘F) leaves full screen. A page-level preventDefault was not used because several windows close on an otherwise unhandled Escape.

Fixes #3104
2026-11-13 18:33:53 +08:00
l0ng-ai 2e78c42fac perf(java): find implementations in the sources through the index (#3132)
Go to Implementation on project code built a region type hierarchy over every source folder, which resolves every type in the region whatever the target; in a large multi-module project it took seconds. It now runs the operation IType.newTypeHierarchy uses, scoped to the source folders and the bridged Kotlin output, so the index narrows the subtypes to files that name a supertype. Results are unchanged, local and anonymous classes included (~230 ms -> ~7 ms on a 4,000-class test project).

Refs #3101
2026-11-13 18:27:46 +08:00
l0ng-ai 1d0449d430 fix(refactor): introduce a variable directly when the expression occurs once (#3128)
With a language server that offers both a single-occurrence and an all-occurrences Introduce Variable action, the "Multiple occurrences found" chooser opened even when the selected expression occurred only once. It now applies the single-occurrence action directly then, as the built-in path already did.
2026-11-13 18:05:20 +08:00
l0ng-ai 73f7fa0016 feat(editor): start a new line in a /* */ comment with a leading asterisk (#3127)
Enter on the first line of a /* */ or /** */ comment, or on a line that starts with *, starts the new line with * aligned under the comment's asterisks; before the closing */ the comment is closed on its own line. Strings and // comments are left alone.

Fixes #3099
2026-11-13 18:03:37 +08:00
l0ng-ai f3dfb384c9 fix(lsp): check again for a missing language server from the status popup (#3126)
A language server reported as not installed had its restart button disabled in the Language Servers popup, so one installed outside the IDE afterwards stayed "Not installed" until the project was reopened. The popup now offers "Check for <name> again" next to Install; restarting a missing server resolves its executable again and starts it when found, and says "<name> is still not installed" otherwise.

Refs #2925
2026-11-13 18:02:59 +08:00
Muuтаmuutot b53cfe8cc2 feat(color-scheme): Git page for inline blame and project view branch colors (#3090)
Adds an Editor › Color Scheme › Git page with attributes for the editor inline blame and the Project View branch decorations (changed, HEAD tags, local, master/main with and without remote, tracking), with a live preview. The attribute panel now disables the channels an attribute doesn't use and labels the effect selector "Effect style".

Co-authored-by: muutot <[email protected]>
2026-11-13 17:43:02 +08:00
Mr0Bread 7d146ef1b7 feat(database): reuse a console's result tabs on each run, pin tabs to keep them (#3113)
Running a console added a new Result N tab per run. As in IntelliJ, a run now replaces that console's unpinned result tabs (in place of the first of them); a run without result sets keeps them and shows Output. Pin Tab (tab menu or the result toolbar) keeps a result: its close button becomes Unpin and close gestures spare it; Close All but Pinned closes the rest. Explain Plan replaces only an unpinned plan tab, and the 12-tab limit never evicts pinned or just-created tabs. The shared Tabs component gets an opt-in pinned item flag.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 17:37:59 +08:00
Mr0Bread 4bde38ef9d fix(database): use the Database tab's table icon for result tabs and the Table view (#3115)
Result tabs and the result toolbar's Table button drew the db-table glyph while the Services Database tab above them shows the dataTables icon; as in IntelliJ all three now use the same icon. Table nodes in the Database tree and Edit Data keep db-table.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 17:36:45 +08:00
Mr0Bread c6212293e1 fix(database): close the active result tab, not the Services window, on Close Tab (#3118)
Close Tab (⌘W) with focus in the database results hid the whole Services window: toolwindow.closeContent only closes tabs of strips registered with the window's content-tab registry, which was provided only in tool window header rows. ToolWindowContentTabs registers tab strips rendered in a window's content; the result tabs use it, so Close Tab closes the selected result, and with Output selected the window hides as before.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 17:34:42 +08:00
l0ng-ai b8ba1256d2 fix(terminal): keep Nerd Font icons wider than their cell above the next cell's background (#3121)
Nerd Font / Powerline icons (Private Use Area characters) are often drawn wider than one cell. xterm's DOM renderer paints each span's background in order, so the next prompt segment's background covered the icon's right part and powerline prompts showed half icons. Spans holding such characters are now marked in the DOM row hook and lifted over the following spans.

Fixes #3094
2026-11-13 17:30:43 +08:00
langxpтаkradme 6fc6bf598c fix(java): use and cache the Maven runner launch classpath (#2929)
Delegated Java launches took their dependency classpath from jdtls, whose m2e import runs on the server JDK and can activate other JDK-activated Maven profiles or classifiers than the module build (a reactor switching between jdk8 and jdk21 artifacts). The launch now asks the Maven process and runner JDK that built the module for its runtime / test classpath (compile + dependency:build-classpath with -am, no package), cached by the launch fingerprint (POMs, sources, JDK, settings, profiles, environment). A failed probe falls back to the jdtls classpath with a hint in the console; cancellation still cancels. A fingerprint-verified snapshot is reused before JDT errors escalate a launch to Rebuild, and generated sources under the build directory no longer change the fingerprint after each compile.

Co-authored-by: kradme <[email protected]>
2026-11-13 17:27:38 +08:00
l0ng-ai 95235220b5 fix(lsp): escape unescaped Unicode and spaces in file URIs from any server (#3117)
lsp-types rejects a URI with raw non-ASCII characters or spaces, so one such file URI made a whole response fail to deserialize ("data did not match any variant of untagged enum DocumentChanges"). Java servers build some URIs with java.net.URI.toString(), which keeps non-ASCII characters, e.g. jdtls in Introduce Variable's edit for a project under a Chinese path. The csharp-ls normalisation now applies to file: URIs from every server, in responses, publishDiagnostics and workspace/applyEdit, including WorkspaceEdit.changes keys; opaque data and other schemes pass through unchanged.

Refs #3089
2026-11-13 17:24:17 +08:00
GuoYuтаguoyucode 620fa247ef feat(mcp): fixed port and kept token, focus only for confirmations, no refactoring notice for agent edits (#3062)
The IDE MCP server took a new port and token in every run, so configured HTTP clients and port forwards broke at each restart; every MCP request brought the window to the front; and an agent's edits were announced as refactorings.

- Settings › Agents › IDE MCP Server: Use a fixed port (on by default, 26371) with a fallback to a free port when it is in use or reserved, reported in Status; Keep the token (off by default) stores the bearer token in the app data, readable by this user only, with Regenerate Token. Changing either restarts the server. Access control is otherwise unchanged (127.0.0.1, bearer token, origin and host checks).
- Only confirmations, which wait for the user, raise the window.
- Agent edits stay undoable (Local History point for multi-file edits) but are not announced as refactorings.

Co-authored-by: guoyucode <[email protected]>
2026-11-13 17:19:02 +08:00
l0ng-ai 52e5a0c4af fix(vcs): leave a new branch from a remote branch with another name untracked (#3063)
Git's autoSetupMerge made the remote start point the upstream of a branch created from it whatever its name, so a branch 'feature' created from 'origin/main' proposed and pushed to 'main'. New Branch and New Worktree now create such a branch with --no-track, so the Push dialog targets its own name and the first push sets that upstream; a remote branch checked out under the same name is still tracked.
2026-11-13 17:11:46 +08:00
l0ng-ai 4acb265b90 fix(gradle): run the unpacked wrapper distribution when gradle-wrapper.jar is missing (#3116)
A *.jar rule in .gitignore often leaves gradle/wrapper/gradle-wrapper.jar out of the repository, and gradlew then fails with only "could not find or load main class org.gradle.wrapper.GradleWrapperMain". When the jar is missing, the distribution named by gradle-wrapper.properties runs from the Gradle user home's wrapper/dists if the wrapper already unpacked it; otherwise the wrapper runs as before and the failure summary names the missing jar and the fixes.

Fixes #3086
2026-11-13 17:10:15 +08:00
Mr0Bread b284c37588 feat(database): add default result tab navigation shortcuts (#3100)
Select Next / Previous Tab for database results get default shortcuts in every keymap preset, the same keys as editor-tab navigation (IntelliJ macOS ⇧⌘] / ⇧⌘[ and ⌃→ / ⌃←, IntelliJ Windows/Linux Alt+→ / Alt+←, VS Code and Eclipse their editor-tab keys). They apply while focus is in Results (tracked with focusin so it survives a replaced grid or tab) and move focus to the destination tab; elsewhere the keys keep switching editor tabs.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 17:07:09 +08:00
l0ng-ai 771575057e fix(database): keep unsubmitted result edits while another result tab is shown (#3114)
The result pane is keyed by tab and remounts when another result tab is selected, so its pending cell edits, added and deleted rows were dropped on every switch (mouse, menu or keyboard). Pending edits are now kept by the result rows they were made on; a re-run or another page still starts over.
2026-11-13 17:03:51 +08:00
acaoтаamphetamlne 563d2a75d4 perf(vcs): reuse the displayed graph for parent navigation (#3049)
Go to Parent rebuilt the displayed log graph for every candidate commit on every paging check (604 graph builds for a 300-commit log with the parent already loaded). Each check now builds one snapshot and uses it for both the parent adjacency and the candidates, recomputing after each loaded page so collapsed / linearized relationships and repository-qualified ids stay right. Paging stays uncapped, so deep parents are still found.

Co-authored-by: amphetamlne <[email protected]>
2026-11-13 17:01:29 +08:00
LunarFellerтаLunarFeller 3a64bba254 fix(database): preserve numeric precision in JSON exports (#3102)
The JSON extractor converted numeric cells through JavaScript Number before JSON.stringify, rounding BIGINTs and high-precision decimals in the preview, Copy as JSON, the dialog clipboard and file export. Numeric cells are now written as normalized decimal text (sign, leading zeros and an empty fraction normalized, digits and exponent kept); non-finite or non-decimal text stays a string as before.

Refs #3105

Co-authored-by: LunarFeller <[email protected]>
2026-11-13 16:58:16 +08:00
LunarFellerтаLunarFeller f4b4a0fa72 fix(database): preserve precision when sorting numeric cells (#3075)
Result sorting, filter value order and selection Min/Max compared numeric cells through JavaScript Number, so BIGINTs beyond 2^53 and high-precision decimals that round to the same double kept the wrong order. The comparator still uses the float difference when it is non-zero (rounding is monotonic) and compares the decimal digits and exponent order for rounded ties.

Refs #3105

Co-authored-by: LunarFeller <[email protected]>
2026-11-13 16:57:45 +08:00
Bake Yam 50bd134b65 feat(templates): built-in Jupyter Notebook file template (#3084)
New ▸ Jupyter Notebook creates a valid nbformat 4 notebook with a Python 3 kernelspec, either empty or starting with a Markdown heading named after the file. It uses the .ipynb file icon and is listed under Settings › Editor › File and Code Templates like the other built-ins.

Fixes #3082

Co-authored-by: Renjiankang <[email protected]>
2026-11-13 16:56:21 +08:00
Bake Yam 8840a1e58a fix(templates): write the file template body before opening the file (#3083)
Creating a file from a template whose extension opens in a non-text editor (an .ipynb notebook) left the file empty: the body was only ever inserted into a CodeMirror view, which never appears for such editors, so the wait timed out after ~4 s with "The template content could not be inserted."

Now the editor kind is probed before the file is created (its acquisitions released); a file a provided editor opens gets the rendered body written on create. Text templates keep inserting into the view (caret at $END$, reformat, saved with the editor's encoding).

Fixes #3081

Co-authored-by: Renjiankang <[email protected]>
2026-11-13 16:55:16 +08:00
l0ng-ai bf36cc0801 chore: v0.1.44 2026-11-13 15:46:55 +08:00
l0ng-ai 78c98d289f fix: Snowflake transactions and lost sessions, accept never writes over unreadable files (#3098)
Snowflake: COMMIT/ROLLBACK end a transaction without switching to the console's schema, and a failed one stays held to be ended again; after a lost session is signed in to again, the target's USE runs before the statement is repeated (a table-editor transaction is not repeated); the session key includes the project. Agent accept: a checkout file that exists but can't be read is skipped as the checkout's own instead of written over.
2026-11-13 15:46:25 +08:00
l0ng-ai 30b0fcb8fc fix(mcp): a late diagnostics publication for an older text version isn't fresh (#3097)
A publishDiagnostics tagged with a version older than the document's is still shown but no longer counts as a fresh answer for check_files.
2026-11-13 15:43:44 +08:00
l0ng-ai 586399deda fix(database): Snowflake runs nothing in another database and keeps sessions per data source (#3096)
A refused USE for a statement's database or schema no longer lets the statement run in the session's previous context (possibly another database of the data source) and is retried with the next statement. Sessions are keyed by data source as well as connection settings, so two data sources with the same credentials no longer share a Snowflake session and its transactions.
2026-11-13 15:33:09 +08:00
l0ng-ai e6e89860e6 fix(agent): don't overwrite a file at a case-only rename's new name on accept (#3095)
On a case-sensitive disk, accepting an a.txt -> A.txt rename removed a.txt and wrote A.txt unchecked, overwriting a file the user had created there. Both names are now checked before either changes (old: the checkout's file; new: absent or already the task's), and the new name is written before the old one is removed.
2026-11-13 15:32:53 +08:00
l0ng-ai d890c41352 fix: trusted interpreters only, plugin diffs off the disk, fresh check_files diagnostics, dotted Maven artifact ids (#3093)
- Python: All Interpreters ran the interpreter of every recent project, including ones last opened in Restricted Mode; only projects trusted in their own right are probed.
- Plugins: views.open-diff's file name became a path the diff viewer probed on disk; plugin diffs get a virtual plugin-diff: path and virtual paths aren't probed.
- MCP: check_files took a publication from before an already-applied outside edit as fresh; the LSP document records when its servers last got new text.
- Maven: version completion split dotted artifact ids into directories; only the group id is split.
2026-11-13 15:29:24 +08:00
l0ng-ai c942b87abb fix(editor): translate the code vision usage counts (#3088)
The usages hint above declarations was always English; it now goes through the text catalog, with Chinese entries.
2026-11-13 14:54:48 +08:00
l0ng-ai cdecb5916c fix(plugins): run the scripts of plugin html views in release builds (#3087)
A view's html body was loaded as a srcdoc frame, which inherits the workbench CSP; its script-src 'self' blocks inline scripts, so in every build that serves the CSP html views rendered but ran no script. In the app the body is now written into a page of the openidea-plugin scheme (.frame) whose response carries only the frame's own policy, sandboxed as before. Web views already load from the scheme; notebook HTML output never runs scripts.
2026-11-13 14:44:26 +08:00
Clarence 835c44615b fix(vcs): Branches popup search matches words, not scattered letters (#3076)
The Branches popup filtered with fuzzyMatch, whose last fallback accepts any subsequence, so `test` listed names like `sprint-1.1-device-reset`. It now uses wordMatch, as Search Everywhere and the Keymap page do: a substring, or fragments at word / camel-hump starts. Repository names in a multi-repository project keep the loose match. The search field takes the bar's free width, so the clear, Fetch and Options buttons sit at the right edge.

Co-authored-by: Clarence404 <[email protected]>
2026-11-13 14:19:19 +08:00
l0ng-ai df79c80aa8 feat(plugins): notes after file names in the project tree (API 1.7) (#3085)
Tree decorations could carry only a two-character badge and a tooltip. decorations.set-file-note (API 1.7) shows one line of muted text (at most 200 characters, optional tooltip) after a file or folder name in the project tree, replacing that tree set's item for the file and sharing the tree-set limits. The SDK wraps it as host::set_file_note with a mock; checked-in components rebuilt.
2026-11-13 14:18:04 +08:00
l0ng-ai 73df48db7c fix(vue): resolve the @ alias in Vue projects without Vue CLI (#3080)
Only projects depending on @vue/cli-service had their webpack aliases passed to tsserver, so @/ imports resolved to nothing in Vue CLI 2, custom webpack or Vite projects without jsconfig.json. Those now get their own webpack configuration's aliases (function / list / ES module exports handled), else the @ -> src alias Vue templates set up.
2026-11-13 14:01:38 +08:00
l0ng-ai 6143011880 fix(http-client): take the server port from the active Spring profiles' files (#3079)
Generated requests read server.port and the context path only from application.properties / application.yml, so a port in application-dev.yml under spring.profiles.active=dev was ignored. The active profiles' files are now read over the base ones, the last profile winning.
2026-11-13 13:55:41 +08:00
l0ng-ai d1872c07ce feat(http-client): request bodies from DTOs in other modules, header completion (#3078)
The @RequestBody type of a Spring mapping was looked up only beside the controller or in its source root, so a DTO in another Maven module produced {}. Types not found there are looked up through the project index in the imported (or same / wildcard-imported) package, and superclass fields are included. Header names and common header values complete in .http request blocks. Requests opened from an endpoint use the module's server address instead of a fixed localhost:8080.
2026-11-13 13:54:32 +08:00
l0ng-ai cd79293fd7 fix(libraries): keep External Libraries in a stable order and reload after a build sync (#3077)
External Libraries groups were listed in detection order, which differed between the streamed and final results; they are now sorted SDKs first, then libraries by name. A Maven sync that changed dependency versions without touching a manifest left old versions listed; every successful build sync now emits projectRootsChanged (Gradle already did) and External Libraries reloads on it.
2026-11-13 13:48:12 +08:00
rohitkumardandaтаrohitdanda 86f7b747a5 fix(security): stop other pages from embedding the app's own pages (#2998)
The app's CSP now includes `frame-ancestors 'none'`, so no page (for example a project's HTML in the built-in preview) can embed the IDE's own pages in a frame. Notebook HTML output, plugin views (srcdoc frames) and the built-in HTML preview still render in a debug bundle that serves the CSP.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 13:46:25 +08:00
l0ng-ai 2e5e9acd9f fix(ui): keep Select All in a reading pane out of the composer (#3073)
⌘A in an agent conversation selected the pane from its own start to its own end; WebKit paints a selection ending on the pane's boundary up to the next selectable text after it, so the composer looked selected too. Select All now runs from the pane's first to its last selectable text, like a clamped drag selection already does.
2026-11-13 13:43:14 +08:00
caojcm b5e9e27dad fix(updater): update portable Windows installs in place (#2846)
A portable Windows build (the raw exe from the portable ZIP, which has no bundle type) updated through the installer, which installed the app elsewhere instead of updating the folder it was unpacked in.

- Updater: an unbundled Windows build asks for the `windows-<arch>-portable` target. Its update is the signed portable ZIP (verified by the updater's download), unpacked into a staging folder next to the exe; only `openidea.exe` and `WebView2Loader.dll` are accepted. Each file is replaced with a backup, rolled back if any replacement fails, and the backups are removed on the next start.
- Release: the portable ZIP is signed with the updater key and `latest.json` lists `windows-x86_64-portable` and `windows-aarch64-portable`.

Co-authored-by: caojcm <[email protected]>
2026-11-13 13:34:56 +08:00
GuoYuтаguoyucode ff1bfeea9d feat(mcp): check_files for fresh diagnostics, exact read_file ranges, search results with the matched line and context (#3061)
- check_files syncs each file from disk, tells its language servers it was saved (so on-save checks such as rust-analyzer's cargo check run), waits until every server answered for the current text and went idle, and reports per file whether the problems are complete (checked / incomplete / serverUnavailable / noLanguageServer / untrusted / failed) with each server's status and reason.
- read_file with offset or limit returns exactly that range plus where to read on; the default read of a whole file keeps its head and tail.
- search_text / search_regex / search_symbol results carry the matched line (long lines cut around the match) and, with `context`, the lines before and after it.
- Merged main: check_files uses the run tools' uncapped timeout.

Co-authored-by: guoyucode <[email protected]>
2026-11-13 13:29:43 +08:00
GuoYuтаguoyucode 14ee1a97f2 feat(mcp): agent commands in Run tool window tabs, with streamed progress and no timeout cap (#2997)
The MCP server let agents read and edit code but not run a command and follow it, so they ran commands in their own shell, out of the user's sight. New tools run_command, wait_for_run, read_run_output, get_run_status, send_run_input, stop_run and list_runs run each command in a Run tool window tab _agent_<name>: the user sees it and can type into it, rerun, stop or close it, and results then carry interventions and advice so the agent stops and asks. The same name reuses its tab; a running one is refused. Every start asks the user unless tools run without confirmation. Commands run on pipes, with maxLinesCount / truncateMode, offsets to read on, and pattern waits. Finished agent tabs beyond a limit (default 1) move to an Archived (n) menu.

A call still running after a second is answered on an event stream with a progress notification every 15 s, so clients that drop a response not started within a minute keep waiting; the run tools (build_project and execute_run_configuration included) no longer cap timeout. A call is cancelled when its client goes away; the stdio bridge relays streamed answers.

Co-authored-by: guoyucode <[email protected]>
2026-11-13 13:23:56 +08:00
l0ng-ai 916c4243bf fix(agents): shorten the command permissions copy (#3072)
Settings › Agents › Command Execution Policy becomes Command Permissions, its description is split into short sentences, the columns drop the (Whitelist) / (Blacklist) suffixes, and the allowed-command placeholder no longer suggests a trailing `*` that allow entries ignore. Chinese: 命令权限, and Coding Agents reads 编程智能体 like its parent group.
2026-11-13 13:18:05 +08:00
l0ng-ai 9c63f174e4 fix(keymap): list the Git log's Show Diff in the VS Code preset (#3071)
#3065 bound vcs.log.changes.showDiff in the IntelliJ keymaps, and the VS Code
preset must name every IntelliJ action id (keymap.test.ts). VS Code has no Show
Diff key, so it gets none, like vcs.showDiff.
2026-11-13 13:13:44 +08:00
GuoYuтаguoyucode b463252bc5 fix(run): decode Windows programs' piped output in any system language (#3060)
Commands run on pipes (Run configurations without terminal emulation, and agent commands) showed mojibake for Windows' own programs: ping, dir, cmd's messages and PowerShell's native output write a pipe in the console (OEM) code page, wsl.exe and cmd /u write UTF-16LE, while a UTF-8 console decoded every byte as UTF-8.

On Windows, a UTF-8 console now decodes piped output line by line: a UTF-8 line stays UTF-8, any other line is decoded in the console code page (encoding_rs for GBK, Big5, Shift_JIS, EUC-KR, 866, 874 and the 125x pages; Windows for the single-byte OEM pages it lacks), and a read with NUL bytes is UTF-16LE. ASCII passes through at once; the rest of a line is held until it shows its encoding, and a held prompt is shown after a batch interval without more output. An explicit console encoding and terminal emulation are unchanged; other platforms keep the plain UTF-8 decoder.

Co-authored-by: guoyucode <[email protected]>
2026-11-13 13:06:11 +08:00
GuoYuтаguoyucode 2abdac4fdb fix(mcp): a run point in a Cargo workspace member runs that member's tests (#3059)
execute_run_configuration with a run point in a Cargo workspace member ran the first member's tests: every member's `cargo test -p <member>` configuration runs from the workspace root, so all were equally close to the file and the first won. A Cargo configuration now covers its member's directory (extras.module) when the closest configuration is picked; its working directory stays the workspace root.

Co-authored-by: guoyucode <[email protected]>
2026-11-13 13:04:35 +08:00
l0ng-ai 721f4144c1 fix(vcs): keep a commit's details shown while a refresh reloads them (#3070)
The Git log's commit details (containing branches, signature, refs) were keyed on
the repository's refresh version and dropped as soon as it changed, so each
refresh blanked the section until the reload finished. With something writing to
the repository's .git often, the section blinked continuously.

useCommitDetails now keeps the last details of the same commit until the reloaded
ones arrive; another commit still starts empty.
2026-11-13 13:01:09 +08:00
l0ng-ai 55057b8958 feat(vcs): open a file on its Git remote from Open In (#3069)
Open In in the project view and the editor (context menu and tab) now lists the
repository's hosting service, GitHub, GitLab, Bitbucket or Gitee, named after the
remote. It opens the file at the caret line, or the directory, on the remote's web
UI, at the current branch's upstream when it has one and at the HEAD commit
otherwise. The remote is the upstream's, else origin. Hosts other than the known
services are treated as self-managed GitLab and shown by host name.

The remote URL of each repository is read when the repositories or the upstream
change, so the menu is built without waiting for Git.
2026-11-13 12:57:02 +08:00
l0ng-ai 8abc8e7fbf fix(clipboard): read the clipboard natively for Paste from menus (#3068)
A Paste chosen from a menu (the terminal's context menu, Edit › Paste, Paste as
Plain Text and other paste buttons) has no paste event to read, so it used
navigator.clipboard.readText(), and the terminal tried execCommand("paste") first.
On Linux, WebKitGTK gives neither the text, so the terminal's context-menu Paste
did nothing while middle-click paste worked.

A new clipboard_text command reads the clipboard's text natively, as
clipboard_write_text already writes it: GTK on Linux, NSPasteboard on macOS, and
none on Windows, where the WebView's clipboard is still used.
readClipboardPlainText() wraps it with that fallback, and every menu paste reads
through it. On Linux the terminal pastes the native text before trying
execCommand; an image-only clipboard keeps the existing paths.
2026-11-13 12:52:26 +08:00
l0ng-ai 68dd949df3 fix(terminal): keep the first shifted character a macOS IME commits directly (#3067)
With WeChat or Pinyin on macOS, Shift+/ commits `?` with an insertText that
arrives before the character's keydown, while only Shift is down. xterm's input
gate was already armed by the Shift keydown and dropped the event, so the first
press produced nothing and the key had to be pressed twice.

The macOS input handler now also delivers an insertText that arrives while only
modifiers are down since the last keyup. Shifted keys xterm sends itself are
unchanged: their keydown comes first.
2026-11-13 12:41:17 +08:00
l0ng-ai b98bddbbab fix(vcs): fill the diff dialog with the editors (#3066)
The front diff dialog (the modal commit interface's Show Diff) left the area below a
short diff in the dialog's background: its body wrapper used flex: 1 inside the
dialog body, which is not a flex container, so it only took the content's height.
It now fills the body, and the editors' background reaches the footer.
2026-11-13 12:33:15 +08:00
l0ng-ai 21b05ab73a fix(vcs): show the commit's diff on ⌘D in the Git Log's changed files (#3065)
⌘D / Ctrl+D in the Git Log's changed files ran the global Show Diff, which diffs
the selected file's working copy against HEAD, and the context menu's Show Diff had
no shortcut. The changed files now set a vcsLogChangesFocus context; the log's Show
Diff is bound to the Show Diff key while it is set, so the menu shows it and the key
opens the commit's diff of the selected files, and the global Show Diff steps aside.
2026-11-13 12:31:56 +08:00
l0ng-ai 7d7de0b966 fix(python): exclude virtual environments from detected roots whatever their name (#3058)
Root detection only skipped environments named `venv` / `.venv`. A venv with another
name in the project (e.g. `code10`) was walked into, and any package in its
site-packages that ships `pyproject.toml`, `setup.py` or `requirements.txt` became a
source root. Source roots go on PYTHONPATH for runs, so a package directory with an
`io` package shadowed the standard library and Python 3.10 failed at startup with
`init_sys_streams: module 'io' has no attribute 'open'`.

A directory with `pyvenv.cfg` (venv, virtualenv, uv) or `conda-meta` (conda) is now
an excluded root and not searched for modules; creating one re-runs detection.
Project discovery in the workspace folder picker skips such directories too, so
packages inside an environment are not offered as projects.
2026-11-13 12:21:38 +08:00
Clarence 830f4aa802 perf(editor): don't rescan rendered doc comments on every keystroke (#3052)
With Render Documentation Comments on, every keystroke rescanned all comment nodes and rebuilt every rendered comment, so typing in a large file cost about 10 ms per keystroke against about 1 ms with rendering off.

A keystroke (input.type / delete) that touches no comment's lines now only maps the rendered comments and their decorations through the change and marks the view stale; the file is scanned again 300 ms after typing and parsing pause, by the scanner that already does this while rendering is off (renamed deferredScanner). Paste, drop, undo/redo, formatting, reloads, keystrokes on a comment's lines, toggles and relabels still rebuild at once, and parser progress rebuilds at once unless a scan is already waiting, so files still open with visible comments rendered.

Co-authored-by: Clarence404 <[email protected]>
2026-11-13 12:18:34 +08:00
l0ng-ai c905f1428e fix(maven): go from a dependency, plugin or parent to its own POM (#3057)
Go to Declaration in pom.xml found nothing on a dependency that names its own
version, on a <dependencyManagement> entry (a BOM import, say) or on <parent>.
They now open the artifact's POM: the project module with those coordinates, else
the local repository's POM of that version (properties resolved). A dependency
whose version a management entry provides still goes to that entry.
2026-11-13 12:10:46 +08:00
l0ng-ai a8036ce6fa feat(plugins): open two texts in the diff viewer (API 1.7) (#3055)
Plugins could show their own content in an editor tab (open-content-tab) but
could open the diff viewer only as the reply to one of that tab's actions. A
review plugin (a merge request's files, say) had no way to show a file's
before and after.

views.open-diff opens two texts in the invoking window's diff viewer, with
optional labels and title and a file name that picks the syntax highlighting;
web views send { op: "open-diff", ... }. The SDK wraps it as host::open_diff
with a mock. The plugin API moves to 1.7.0 (1.6.0 is released) and the checked-in
components are rebuilt.
2026-11-13 12:02:36 +08:00
l0ng-ai 28829ad756 style(vcs): format inline blame (#3051) 2026-11-13 11:22:44 +08:00
l0ng-ai d6c562049f test(e2e): load the app modules of the Branches resize spec through variables (#3050)
knip resolved the literal Vite paths in page.evaluate as imports and failed on them; the other specs pass the module path through a variable.
2026-11-13 11:21:04 +08:00
l0ng-ai 100075166c test(database): let FSEvents deliver the startup events before checking ignored files (#3048)
On macOS, FSEvents can report the creation of .openidea/ made just before the watch started after the watch is running, so ignores_other_files counted one change and failed on CI. The test now waits for those events and compares against the count after them.
2026-11-13 11:18:40 +08:00
l0ng-ai d5332a2197 feat(ui): show Go files with a gopher icon (#3047)
Go files get the gopher icon, as GoLand shows them, instead of a "GO" letter tile; go.mod and go.work keep theirs.
2026-11-13 11:16:32 +08:00
arookieofcтаarookieofc b0d20d7997 feat(vcs): Backend trait for the Changes-view surface (#2958)
Adds an ide-vcs Backend trait for what the Commit / Local Changes view, diff viewer, annotate and log need from a repository (status, file_at / file_mode_at, diffs, line hunks, blame, block history, log, commit files, revision diffs, stage / unstage / discard, commit, pending changes), with Repo implementing it by forwarding. Git-only concepts (branches, remotes, stash, rebase, worktrees) stay on Repo, reached through as_git.

The trait has default no-op hooks for IDE file operations (files_added, files_deleted, file_moved), which the host now calls after create, copy, delete and rename so a checkout-based backend can open files for add / delete / move. The Changes-surface call sites in ide-host go through &dyn Backend. No behaviour change for Git.

Co-authored-by: arookieofc <[email protected]>
2026-11-13 11:09:57 +08:00
rohitkumardandaтаrohitdanda 8df59cf5a7 fix(remote): decide routing, trust and accepted events on the desktop (#2999)
In a remote window the desktop took several decisions about itself from the server: it routed commands by the server's `commands` reply (a command the server left out ran locally), authorized its own commands from the server's `project_current().trusted`, and showed or ran whatever events the server forwarded, including any workbench action named by a plugin `executeCommand` event.

- Routing: commands go by `rpc::client_commands()`, the list a server of this build serves, compiled into the desktop; a test builds the server's dispatcher and compares the two.
- Trust: commands that run on the desktop follow the desktop's own decision, stored in `remote-project-trust.json` by target and root (a dev container also follows its local folder). The trust prompt appears unless the desktop's and the server's decisions agree; the answer and Revoke Trust update both. New commands `remote_trust_status` / `remote_trust_set`.
- Events: server `event`s pass an exhaustive allow-list (MCP approvals, update notices and window-group events are dropped); plugin `executeCommand` may only name `plugin.<id>.<command>`; `openProject` is dropped.
- Paths: a project root that the desktop would read as a share or device path is refused at connect.
- A remote plugin's `openUrl` and `copyToClipboard` ask first, showing the full address or the text.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 11:05:12 +08:00
Binтаffgenius 3c68d2ea7f fix(vcs): position commit history above message input (#3040)
Commit Message History opened below its button, over the message field, and the menu reserved an empty icon column. It now opens above the Amend bar, aligned with the message's left edge, and Menu takes iconColumn={false} to leave out the icon column for rows without icons.

Co-authored-by: ffgenius <+[email protected]>
2026-11-13 10:54:03 +08:00
l0ng-ai cfe1416dbd fix(index): keep the symbol index across releases that leave extraction unchanged (#3027)
The persistent symbol index was reset whenever Syntax::fingerprint() changed, and the fingerprint included CARGO_PKG_VERSION, so every release reparsed every file on first open.

The fingerprint is now a build-time hash of what decides extraction: ide-language's src/ and queries/, languages/*.toml, and the locked tree-sitter crates. The index stamp also carries a build-time hash of ide-index's own sources (ignored state, size limit, stored record), so a change there still resets the index without a manual SCHEMA bump.
2026-11-13 10:47:24 +08:00
l0ng-ai 44fc0a2ebc fix(java): build with Maven when JDT's annotation processing recreates a file (#3043)
With MapStruct-Plus, JDT runs MapStruct again over the mappers MapStruct-Plus
generated in the same build and fails with "FilerException: Source file already
created", so every run of the application stopped on that error. javac doesn't,
so a Maven module whose JDT build fails this way is now built with Maven, and a
launch already built with Maven ignores that JDT error instead of reporting the
build as failing.
2026-11-13 10:44:29 +08:00
l0ng-ai e52cb2e426 fix(agents): keep the conversation at its end after a send or an opened section (#2975)
After a send the composer shrank and the conversation view grew; the resulting scroll was taken for a user scroll, so following the end stopped and the reply arrived below the view. A change of the view height now counts as layout. Opening a folded section (reasoning, tool call, turn) keeps the view at the end when the end was in view, else scrolls the section into view.
2026-11-13 10:12:27 +08:00
l0ng-ai 519e90da9f fix(plugins): list the official marketplace in debug builds too (#2977)
Debug builds set no default marketplace index, so the Plugins page showed only the repository's local plugins and browsing, installs, update and revocation checks couldn't be exercised without entering the index URL. The official index is now the default in every build; local plugins keep their own group.
2026-11-13 10:12:18 +08:00
Deng Liuтаdengliu 9e73877f5e fix(plugins): raise the plugin HTTP response body limit to 16 MiB (#3033)
PluginApi::fetch capped response bodies at 8 MiB after reqwest's transparent decompression, so compact gzip API responses (a Kubernetes Table listing of a large cluster) failed once decompressed. The limit is now 16 MiB, the same as process output and file reads, and the error message follows the constant. Plugin network docs updated.

Co-authored-by: dengliu <[email protected]>
2026-11-13 10:12:10 +08:00
l0ng-ai 94a8761cc9 feat(editor): add Fix ESLint Problems, and let ESLint fix-all fix every rule (#3038)
Fix ESLint Problems runs vscode-eslint's source.fixAll.eslint on the whole file; the editor context menu offers it for files a running ESLint server lints, and it can be bound in the keymap.

The ESLint server settings sent codeActionOnSave.rules = [], which vscode-eslint reads as "fix no rule", so its fix-all (also from Code Cleanup and Actions on Save) always returned an empty edit. The key is left out now, so every configured rule is fixed.
2026-11-13 10:07:15 +08:00
l0ng-ai e66513668a fix(run): run JAR Applications on the project JDK by default (#3037)
A JAR Application without a chosen JRE ran java from PATH even with a Project JDK set. Its default JRE is now the project JDK, applied with the other toolchains (an explicit JRE stays), and the resolved path is left out when saving so it follows later changes. The JRE field's default reads "Default (the project's JDK)".
2026-11-13 10:06:31 +08:00
SunZhiQiang 565bdf170f feat(agents): pin tasks and arrange them in groups (#3016)
Tasks in the Agents tool window can be pinned above every group, and put in groups of your own from the context menu or by dropping them on a group header. A group's menu renames or removes it; removing it returns its tasks to their status groups. Pinned tasks keep their group, so unpinning returns them to it. Groups and pins are kept per project in localStorage.

Co-authored-by: panda-student <[email protected]>
2026-11-13 10:04:58 +08:00
SunZhiQiang 5712a5373d feat(agents): show a task's project, checkout and branch as chips (#3020)
The session status line packed provider, project, checkout, branch and state into one string, while the New Task composer showed them as separate controls. Both now use one Chip primitive (also a chip variant of Select), so project, checkout and branch read as separate facts, and the project is named for a plain project too. The changed-files and commits buttons move to the right of the status line, above the context meter, time and terminal.

Co-authored-by: panda-student <[email protected]>
2026-11-13 10:04:50 +08:00
SunZhiQiang 2a981dd642 fix(agents): centre Jump to Latest over the conversation (#3022)
Jump to Latest sat against the panel's right edge, so in a wide editor tab it floated far from the centred conversation. It is now centred over the end of the stream, like the content. The conversation keeps its readable width.

Co-authored-by: panda-student <[email protected]>
2026-11-13 10:04:43 +08:00
SunZhiQiang 3580cbacd7 feat(agents): show the context window's size in the usage meter (#3021)
The composer's context meter read only "12%". It now reads "24k/200k (12%)", so the window's size is visible at a glance. The bar fills by the percentage rather than using the label as a CSS width, and the tooltip keeps the exact counts.

Co-authored-by: panda-student <[email protected]>
2026-11-13 10:04:36 +08:00
l0ng-ai f91b87d790 feat: move configuration, user data and logs with environment variables (#3035)
OPENIDEA_CONFIG_DIR, OPENIDEA_DATA_DIR and OPENIDEA_LOG_DIR take an absolute path, like OPENIDEA_LOCAL_DATA_DIR, and the IDE uses <path>/<identifier> under it. Every place that resolved these directories now goes through ide_host::data_dirs (app state, invalidate caches, window zoom and window state, the MCP stdio bridge, logging, the openidea-plugin CLI), and the frontend takes the user data directory from app_info instead of Tauri's appDataDir. A relative path for the configuration or user data shows an error and exits.

Closes #2814
2026-11-13 09:53:08 +08:00
Mr0Bread 74abf32704 feat(database): add a context menu to query result tabs (#3012)
Database result tabs get Close, Close Others, Close All, Rename Tab, Select Next Tab and Select Previous Tab from the right-click menu or Shift+F10. They are actions, so they can also be bound in the keymap. Close, Close Others and Rename act on the clicked tab without selecting it, which keeps its hover look while the menu is open (showContextMenu takes an optional owner element). Navigation starts from the active tab and wraps through Output. Output, consoles and connections are never closed, and a renamed Query Plan keeps its title when the plan is replaced.

Blocking prompts (promptInput, chooseItem, pickItems, message dialogs) now suspend workbench shortcuts while open, so typing in Rename Tab can't edit the query console behind it. In plain text fields, Backspace/Delete with a modifier stays native text deletion.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 09:37:38 +08:00
Muuтаmuutot 4b1370742f feat(editor): show the date of the last change in the code vision author hint (#2918)
The code vision author hint showed only the name (`Thomas +2`). It now adds the leading author's last change date in the same compact form as the blame gutter (the Date Formats pattern when the override is on, the system's short date otherwise), and the tooltip gives the full date. Blame without a timestamp keeps the old label.

The line-end ("right") hint is drawn from a zero-width anchor with an absolutely positioned inner box, so a soft-wrapped line can't push it to the next row or grow the line height. It uses the Inline hints colour and the editor size minus 1. Inline blame steps past a hint, including one that appears after the blame. A lens is rebuilt when its position changes in an open editor.

Co-authored-by: muutot <[email protected]>
2026-11-13 09:31:42 +08:00
l0ng-ai 6700f57f79 fix(database): connect to MySQL servers that advertise DEPRECATE_EOF but send EOF packets (#3030)
Some servers and proxies (Aliyun RDS for MySQL 5.6) advertise CLIENT_DEPRECATE_EOF but still end result sets with legacy EOF packets; sqlx requested every advertised capability, so every query failed with "OK_Packet too short". sqlx now comes from l0ng-ai/sqlx at a rev adding MySqlConnectOptions::deprecate_eof (proposed upstream as transact-rs/sqlx#4524), and opening a MySQL pool retries once without the capability when the first connection fails with that error.

Fixes #1053
2026-11-13 09:27:41 +08:00
Mamaev Igor 578219de3b feat(editor): add relative and hybrid line numbers (#3015)
Settings › Editor › General › Appearance › Show line numbers gets a mode: Absolute (default), Relative (distance from the caret line, 0 on it) or Hybrid (the caret line keeps its own number). The relative modes use a gutter that keeps the cm-lineNumbers class, so styling, sticky lines and line-number clicks treat it like the absolute one; it re-renders when the caret moves to another line and when lines are inserted or removed above it.

Closes #3014

Co-authored-by: pxc1984 <[email protected]>
2026-11-13 09:23:58 +08:00
Mr0Bread afcdf57eb6 fix(database): inset result tabs from the Services pane edges (#3013)
The database Results view uses the unpadded compact tab strip, so the selected Output tab touched the Services pane edge. The Results strip now has a --space-2 inline inset of its own; other compact strips are unchanged.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 09:23:52 +08:00
Mr0Bread f145ea5d7b fix(database): select the neighbouring result when closing the active tab (#3017)
Closing the active database result selected the rightmost remaining result, jumping away from where the user was. It now selects the left neighbour, then the right one, and Output when none remain. Closing an inactive or missing result keeps the current selection.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-13 09:22:27 +08:00
Alexanderтаformiat 0c00f474e7 feat(vcs): resize Git Branches by its edges and corner (#3025)
Git Branches had a fixed size (380 px, 440 px with several repositories, at most 440 px of list), so long branch names were cut off and only about a dozen branches fit. Its right edge, bottom edge and bottom-right corner now resize it within a minimum size and the window. The popup keeps its anchored top-left corner, the branch list fills the new height, and the last size is kept in localStorage and used on the next open.

Popup gains resizable, minWidth, minHeight, initialSize and onResized, with the same meaning as on Dialog, so other anchored popups can opt in later. The e2e test measures the popup after its open animation.

Co-authored-by: formiat <[email protected]>
2026-11-13 09:20:02 +08:00
Gabriel Gondim 5cf01f682f fix(editor): keep server code lenses mapped across edits (#3011)
Language-server and plugin code lenses were cleared on every document change and only came back after the 400 ms refetch. Their block rows disappearing and reappearing changed the document height on each keystroke, so the viewport jumped while typing with servers that publish lenses (e.g. csharp-ls). Lens positions are now mapped through edits (MapMode.TrackDel), the same way usages lenses already are, until the next fetch replaces them.

Co-authored-by: GGVidal <[email protected]>
2026-11-13 09:14:36 +08:00
SunZhiQiang fec4eb34b9 fix(agents): keep worktree tasks refreshing on project VCS events (#2988)
#2979 matched vcsChanged events against the task's cwd. A worktree task's gitdir is <project>/.git/worktrees/<slug>, outside its checkout, so the watcher reports the project root when the agent commits there and the Changes pane stopped refreshing. Worktree tasks now match their project root; local tasks still match only their own checkout.

Co-authored-by: panda-student <[email protected]>
2026-11-13 09:13:58 +08:00
l0ng-ai 6454dbe169 fix(i18n): restore the commas between merged locale entries (#3029)
#3001 and #2987 each appended entries at the end of en.json and zh.json; merged together, the line between them lost its trailing comma, so neither file parsed.
2026-11-13 09:13:13 +08:00
Gabriel Gondim 8bdf623a28 fix(search): disable ligatures in Find results unless Enable ligatures is on (#3010)
Mono previews outside the editor (Find in Files results, Usages, ...) inherited the browser's default ligatures, so `!==` and `=>` rendered as ligatures while the editor, with `editor.ligatures` off by default, showed plain text. The root now disables ligatures and `data-ligatures`, set from Editor > Font > Enable ligatures, turns them back on for every mono surface, matching the editor.

Co-authored-by: GGVidal <[email protected]>
2026-11-13 09:13:03 +08:00
Muuтаarookieofc bff6a12e2f feat(python): manage every interpreter from a Python settings folder (#2987)
* feat(python): manage every interpreter from a Python settings folder

Settings › Languages & Frameworks now groups Python settings in one folder: All Interpreters (new), Dependencies (the project interpreter and its packages), Docstrings (split out of the interpreter page) and Ruff. All Interpreters lists interpreters detected on the machine, the ones other recently opened projects use and the ones added there, with a search field, PyCharm's "hide interpreters of other projects" and "show interpreter paths" toggles, and + / − to add an interpreter or remove it from the list (hidden, never deleted; a removed project interpreter falls back to the project virtualenv).

Each interpreter's source has its own icon (virtualenv, conda, uv, Poetry, pyenv/mise/asdf, system) in the list, the Add Interpreter dialog, the interpreter combo and the status bar menu. The IDE-wide list and its view options are global settings that a project cannot override; removed interpreters also disappear from the interpreter combo and the status bar popup.

* fix(python): compare interpreter paths by their own case rules and cap probes

A POSIX interpreter path kept only its own case (Linux allows `/opt/Python` and
`/opt/python` side by side); only Windows drive/UNC paths are folded now, and the
status bar uses the same key as the list. A configured `~/…` interpreter is matched
against the absolute path the backend resolves it to, so the "Project interpreter"
badge and removing it in use keep working without the frontend knowing the home
directory.

`sdk_python_project_interpreters` probes at most four recent projects at a time
instead of one thread per project, so opening All Interpreters does not start ~30
pythons at once. The hidden-but-in-use interpreter stays selectable in the combo and
status popup (custom entry / current fallback); that is now documented on `exclude`.

---------

Co-authored-by: arookieofc <[email protected]>
2026-11-13 08:56:17 +08:00
rohitkumardandaтаrohitdanda ee1f15eba7 feat(agents): choose whether new tasks start asking for approval (#3001)
Once an agent reports its modes, new tasks start in the mode that approves everything (Claude Code's bypassPermissions, Codex's full-access, which also turns off the sandbox), where the Command Execution Policy never applies. Settings › Agents › Chat › New tasks start (agents.newTaskApprovals: bypass or ask) chooses; the default keeps today's behaviour. With ask, new tasks start in a mode that asks: the agent's reported mode when it isn't a bypass mode, otherwise its default/ask mode, with allow-all permissions off and Cursor approvals set to ask. The setting is user-only (in the settings layer's USER_ONLY list): project files can't set it.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:55:46 +08:00
l0ng-ai 7ea4f36b6d fix(windows): don't show system error boxes for processes that fail to load (#2550)
Probing a broken Python install (for example one whose python3xx.dll is gone) blocked on a modal "System Error" dialog, because Windows shows critical-error boxes for any process that fails to load a DLL. The app now sets SEM_FAILCRITICALERRORS and SEM_NOOPENFILEERRORBOX at startup, keeping inherited bits, and children inherit the mode, so such tools exit with an error code instead. Covers interpreter detection, Run, Terminal and every other tool the desktop app starts.
2026-11-13 08:53:28 +08:00
Bruno Belloni ddfdbf1be6 fix(linux): keep the DMA-BUF renderer on Wayland with NVIDIA (#2179)
On Wayland with the NVIDIA driver WebKitGTK's DMA-BUF renderer fails with Error 71 unless something works around it. Turning the renderer off made WebKit paint on the CPU, so dragging a tool window stalled the UI for seconds. Setting __NV_DISABLE_EXPLICIT_SYNC=1 for the process avoids the protocol error and keeps GPU rendering. A user who already set either variable keeps their own setting; WEBKIT_DISABLE_DMABUF_RENDERER=1 still restores the old behaviour.

Co-authored-by: brunobelloni <[email protected]>
2026-11-13 08:49:21 +08:00
rohitkumardandaтаrohitdanda da412f14be fix(agents): keep agent permissions out of project settings (#3000)
A project's .openidea/settings.toml could loosen what coding agents do without asking: auto-approve commands, empty the blocked list, pick the mode new tasks start in, and add MCP servers every agent session starts. Project settings can no longer set agentSecurity.allowedCommands or agents.defaults (user settings only, like mcpServer), and a project's agentSecurity.blockedCommands adds to the user's list instead of replacing it. Workspace MCP servers start only once the user has turned that exact definition on in Settings › Agents › External MCP Servers; servers saved there count as turned on, servers imported or changed in the file stay off until turned on. The Command Execution Policy page drops its Global / This project switch.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:48:40 +08:00
l0ng-ai 104986ead4 fix(ci): build the first nightly when no nightly tag exists yet (#2747)
Without a nightly tag the compare call returns 404 and its error output was appended to the status instead of replacing it, so the scheduled nightly never built. The fallback now replaces the output.
2026-11-13 08:48:14 +08:00
rohitkumardandaтаrohitdanda fdeaa31b7d fix(agents): keep agents' environment values in the credential store (#3003)
Environment variables set for an agent (built-in, custom or imported from acp.json), typically API keys, were written as plain text into <config>/agents/<id>.toml. Their values now go to the OS credential store like the other saved passwords; the agent file keeps only the names. Values already in agent files move there the next time they are read. Without a credential store (a headless Linux host) a value is kept in an owner-only file in the user's data and the user is told.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:42:06 +08:00
rohitkumardandaтаrohitdanda 4d5c640451 fix(plugins): judge sandbox paths from their text before opening them (#3007)
The plugin sandbox resolved a requested path, which on Windows opens it, before checking whether it was allowed, so a network path (UNC or verbatim UNC) or a pipe or device path (\\.\pipe, GLOBALROOT) was opened and connected to before being refused. Every resolver now judges the path from its text first: devices, pipes, NTFS streams and reserved device names are always refused on Windows, a network path is resolved only under the allowed root or grant, and local paths go on to the existing real-path checks. workspace.project-of uses the same check.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:38:47 +08:00
rohitkumardandaтаrohitdanda 3e40295668 fix(agents): keep worktree Accept out of git's own files and links (#3004)
Accept in a worktree task copied every changed path into the local checkout without checking where it lands, so a path could reach the checkout's .git through a name that opens it (8.3 short name GIT~1, a case variant, trailing dots or spaces, an NTFS stream suffix) or go through a symlink or junction on either side. Accept now refuses, and lists as skipped, any path that names git metadata, leaves the tree once links are followed, lands in the repository's git dir or common dir, or is a link itself, on both the worktree and the checkout side. The .git-name and real-path helpers the plugin sandbox used move to ide_core::real_path so both share one implementation.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:36:29 +08:00
rohitkumardandaтаrohitdanda b43db63ff3 fix(agents): judge permission requests by their kind and command, not their title (#3006)
The Command Execution Policy and the permission card treated any request titled `Run …` as a shell command, but the title is free text the agent writes: a Codex request for wider sandbox access titled `Run git status` was approved by an allow entry `git status`. Only a tool call of kind `execute` now counts as a command, and only its structured `rawInput.command` is evaluated and shown; other requests ask with their own title. A command approval that also grants network access can still be blocked but is never auto-approved. Permission entries carry the command, so the card no longer parses the title; the native Codex driver puts the resolved command and any network context into rawInput.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:33:12 +08:00
rohitkumardandaтаrohitdanda 4dbb6920c8 fix(agents): auto-approve only commands every shell reads alike (#3005)
The Command Execution Policy split and checked commands with POSIX rules, but agents also run commands in PowerShell and cmd.exe, so `git status \; calc.exe` or `git status '&' calc.exe` passed as an allowed `git status` while the shell ran two commands. A sub-command is now auto-approved only when every shell reads it the same way (no `\\` outside a path, no separator or redirection outside double quotes, no `^` before a quote, no `%`, carriage return or typographic quote); otherwise the policy asks. Blocked patterns are also compared ignoring case, repeated spaces and the program's folder or .exe.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:31:35 +08:00
Vadim LazukтаInf1mum 10603b953b feat(maven): complete groupId, artifactId and version of pom.xml dependencies (#2995)
Inside a <dependency>'s <groupId>, <artifactId> or <version> in pom.xml the editor suggests the coordinates installed in the local repository (Settings › Maven's, else ~/.m2/repository) and, behind them, Maven Central's: the search endpoint's groups and artifacts and the artifact's published versions, cached for the session.

The local repository answers at once; Central gets one second and otherwise finishes in the background for the next request, so an unreachable Central never holds completion. Suggestions keep their order (installed first, newest versions first), and a version counts as installed only with its .pom or a .jar, not the .lastUpdated files a failed download leaves.

Fixes #2968

Co-authored-by: Inf1mum <[email protected]>
2026-11-13 08:27:24 +08:00
rohitkumardandaтаrohitdanda 59a0a86257 fix(ci): satisfy clippy 1.100's needless_bool in the Windows build (#3018)
Rust 1.100's clippy flags needless_bool when a #[cfg(unix)] block disappears on Windows and leaves an early `return false` followed by a literal. The three sites (ide-core tool_path, ide-extension ledger, ide-host mcp/edit) are rewritten without early returns; behaviour is unchanged.

Co-authored-by: rohitdanda <[email protected]>
2026-11-13 08:14:53 +08:00
l0ng-ai f05066b18f fix(java): open the Kotlin source for a Kotlin class used from Java (#3026)
Java code sees the project's Kotlin classes through class folders on JDT's classpath, so Go to Declaration on a Kotlin class from a Java file returned the folder's class and opened the decompiled view.

A goto result whose element handle names a class in a class folder (never a dependency jar or JDK module) now opens the project's source of that class: the .kt / .java file named after it (or after a FileKt facade, or the outer class) that declares its package, preferring the directory matching the package and skipping build output. The name under the caret selects the declaration in it.

Fixes #2969
2026-11-13 08:11:38 +08:00
arookieofc 4c15fe5a23 test(e2e): take the first 'Plain PHP documentation' token (#2994)
The locator resolves to two leaf spans — the comment tokenizes as
multiple spans — so the strict-mode expect flakes. Every other
multi-match token in the spec already uses .first().
2026-11-13 03:33:18 +08:00
arookieofc 8f6a4b5b39 style: format the plugin settings test (#2980 merged with a format slip) (#2992)
oxfmt --check fails on main, failing the Frontend job for every PR.
2026-11-13 02:30:22 +08:00
l0ng-ai 4f9dc0a56c fix(vue): go to the ref and its component's members from this.$refs (#2989)
Vue 2.7 types `$refs` as an index signature, so the TypeScript service sends `this.$refs.child` to that signature in vue.d.ts and finds nothing for `this.$refs.child.focusIt`.

Go to Declaration in a Vue SFC's script now reads the template: `this.$refs.<name>` goes to the `ref="<name>"` attribute, and a member read from a component's ref goes to that member in the component's options (methods, computed, props, data() / setup() return). The component comes from the default import its tag names; relative paths resolve directly, aliases through the language service, then `@/` as `src/`.

Fixes #2940
2026-11-12 23:37:26 +08:00
l0ng-ai 67afcc9805 perf(remote): list the remote tree with one find before a run (#2986)
Comparing the project with the server (sync before a run, Sync with
Deployed) listed the remote tree folder by folder over SFTP: thousands of
round trips on a big project, even when nothing had changed. Over SSH it
now runs one GNU `find -printf` for the whole tree, pruning folders the
deployment excludes by a plain name pattern, and keeps the same entries the
walk would (excluded names anywhere in the path, excluded remote paths).
Hosts without GNU find (BusyBox), an unreadable folder, or FTP list folder
by folder as before.

Refs #2873
2026-11-12 23:19:58 +08:00
l0ng-ai 632eb9cbf7 fix(database): list the PostgreSQL schemas a user may use, not only owned ones (#2985)
The schema list came from information_schema.schemata, which before
PostgreSQL 10 (and on servers built on such a core) lists only schemas owned
by the user's roles, so a read-only account saw an empty database. Schemas
now come from pg_namespace, filtered by has_schema_privilege(USAGE), with
the same system / pg_toast / pg_temp exclusions.

Fixes #2964
2026-11-12 23:12:35 +08:00
Bin 88af3cb1eb fix(terminal): clear PowerShell scrollback on cls (#2882)
PowerShell's cls left old output in the terminal's scrollback. With shell integration on (plain Windows PowerShell / PowerShell 7 starts only), Clear-Host now also writes an explicit OpenIDEA marker, which the terminal turns into a full clear (CSI 2J + 3J) in output order. Ordinary ED 2 clears, ConPTY resize repaints and other shells keep their history.

Co-authored-by: ffgenius <[email protected]>
2026-11-12 23:07:44 +08:00
pHтаvvnadolskyy 59eb6446d4 feat(database): add Snowflake data sources (#2972)
Snowflake is a built-in data source kind: connection dialog, key-pair, browser single sign-on, password or token sign-in, the database tree, SQL consoles, and Snowflake-aware SQL assistance and syntax colours. The driver speaks Snowflake's session protocol over the existing reqwest + rustls stack, with no JDBC/ODBC or native library and no new package. Metadata is read with SHOW / DESCRIBE, which need no running warehouse; introspected names are kept on disk for completion.

Co-authored-by: vvnadolskyy <[email protected]>
2026-11-12 23:06:10 +08:00
lifeir 51c14d3b03 fix(plugins): list a plugin's secrets on its settings page (#2980)
A plugin's [[contributes.secrets]] were listed only on its details page under Plugins; its own settings page, where users look for an API key, now lists them too, with the same fields.

Co-authored-by: lifeir <[email protected]>
2026-11-12 23:01:06 +08:00
arookieofcтаa-rookie-of-C-language c727c5b842 fix(agent): apply a case-only rename on a case-sensitive disk (#2970)
Accept Changes applied a case-only rename only where both spellings resolve to one file. On a case-sensitive disk the checkout kept the old name and never got the new one. Now the old file's unchanged content guards the rename there: the task's version is written under the new name and the old one removed; where both spellings are one file it is still renamed in place. The worktree test fixtures also pin core.autocrlf=false.

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-12 22:55:10 +08:00
SunZhiQiang b296ac1ab7 fix(agents): refresh a task's changes only for its own checkout (#2979)
vcsChanged fires for every repository under the project, and the Changes pane re-read the task's changes (walking base..HEAD when grouped by commit) for each one. It now refreshes only when the event's root is the task's cwd or contains it.

Co-authored-by: panda-student <[email protected]>
2026-11-12 22:54:00 +08:00
Alexanderтаformiat 8cce7ce390 fix(libraries): write no Cargo.lock into crates outside the project's workspace (#2976)
External Libraries ran `cargo metadata --offline` in every Cargo workspace it found. A crate excluded from the project's workspace is its own workspace, so cargo resolved it and wrote a Cargo.lock there. Such a crate without its own Cargo.lock is now queried with `--locked`, which fails without writing anything; the project's workspace and crates with their own Cargo.lock resolve as before.

Co-authored-by: formiat <[email protected]>
2026-11-12 22:53:14 +08:00
l0ng-ai 1e41fa86ff test(sdk): drop a stray mut in the interpreter lookup test (#2984) 2026-11-12 22:52:23 +08:00
l0ng-ai 65e0feb98f fix(console): keep the scroll width of long output fixed (#2983)
Console rows are virtualized and absolutely placed, and such rows widen the
scroll area only while they are mounted. Scrolling a long line out of the
rendered range removed the horizontal scrollbar, and where scrollbars take
layout space (Windows) the taller viewport moved the range back, so the end
of a finished run flickered and its scrollbar jumped.

The widest line, measured incrementally in monospace columns (tabs to their
stop, CJK and emoji as two), now sets the content's minimum width. The same
applies to the Build and Debug consoles, which place their rows the same way.

Fixes #2933
2026-11-12 22:50:59 +08:00
l0ng-ai 76faa39bc5 fix(ssh): open SSH sessions with the built-in client and saved credentials (#2982)
Tools › Start SSH Session ran the system `ssh -t` in a terminal, which never
used the password saved in the SSH configuration and, on Windows, could not
take a typed password either.

The session now opens a login shell over the IDE's SSH connection: the
terminal manager can attach a channel (here an SSH shell) next to local PTYs,
with the same output batching, UTF-8 decoding and exit events. A password or
passphrase that isn't saved is asked for in a dialog, and an unknown host key
asks to be trusted, before reconnecting. Closing the tab hangs the shell up.

Also: a remote process closed by this side now ends without waiting for a
close that russh never forwards, and each connection attempt gets its own IPC
channel, since a failed attempt invalidates the one it was given.

Fixes #2874
2026-11-12 22:42:27 +08:00
l0ng-ai 7f9bf3727f fix(theme): switch themes without flicker, preview them at once, drop the built-in One themes (#2981)
Switching a theme with several windows open flickered for seconds: the plugin theme is shared by every window, and each window put back the theme its own settings named whenever it changed, before those settings caught up. Windows now act on the plugin theme only when their own selection changes, and applying the theme already shown keeps its stylesheet instead of replacing it.

Theme previews in the gear menu, Quick Switch and Settings waited 500 ms, and on themes written with hsl() or color-mix() the terminal took up to a second to read its colours: each token was resolved through a new element inserted under the root, restyling the whole page every time. Previews now show at once, and token colours resolve through one element kept for it.

The built-in One Light and One Dark themes are removed in favour of the One Dark Pro plugin; settings naming them read as Islands Light and Islands Dark.
2026-11-12 22:42:10 +08:00
l0ng-ai 405343b7b7 fix(code-insight): show a language server busy only once it stays busy (#2953)
jdtls reports a $/progress task on almost every edit, save, search and diagnostics publish, most lasting 0-20 ms, so the status bar's language-server dot kept switching to busy and back. A running server now shows busy only after it has been busy for 500 ms, the delay background tasks already use. Other statuses and the switch back to running still show at once; only the status bar display is delayed.
2026-11-12 21:47:39 +08:00
Muuтаmuutot cc29842e6b fix(python): workspace members inherit the project interpreter, and a deleted one falls back (#2920)
A workspace member attached to a project resolved its Python interpreter only from its own settings and folder, so a member with no Python of its own ran on the PATH python while the status widget named the primary project's interpreter. Members now inherit the primary project's interpreter when they have none (explicit setting, then virtualenv or pin, as a module without its own SDK inherits the project SDK), sdk_python_effective answers with the same chain, and the status widget asks about the open file's member.

A configured interpreter whose executable was deleted is skipped during resolution, so runs fall back to the project virtualenv or the inherited interpreter. Settings mark it Invalid (decided from what the backend resolves, so working custom paths are not flagged), the status widget warns when nothing resolves, and starting a Python configuration on a missing path names the cause.

Co-authored-by: muutot <[email protected]>
2026-11-12 21:45:37 +08:00
l0ng-ai de0f738bb2 fix(editor): keep Enter for new lines in a completion's snippet (#2967)
Completing an XML tag inserts the server's snippet <tag>$1</tag>$0. Enter moved to the next snippet field in every snippet, so it jumped to the end of the line instead of splitting the tag. Enter now moves between fields only in live templates and Emmet; in a completion's snippet Tab does and Enter inserts a line break.

Closes #2951
2026-11-12 21:25:54 +08:00
l0ng-ai d781df542a fix(lsp): answer Java usages without waiting for jdtls' background work (#2945)
A usages search that found nothing waited up to 10 s for every $/progress task to end before asking again, and jdtls reports background work (builds, validation) for long stretches, so Show Usages on a method nothing calls could take 10 s. jdtls' searches wait for its index themselves, so after ServiceReady an empty answer is final: it is neither waited on nor flagged as incomplete. Before ServiceReady, and for other servers, running progress still counts as indexing.

Refs #2770
2026-11-12 21:12:54 +08:00
SunZhiQiang abfc4cbade feat(agents): review a task's commits in the Changes pane (#2960)
A worktree task's Changes pane only listed the cumulative diff since the task started, so work the agent committed was mixed into the current changes. It now groups the task's work: uncommitted files compared with HEAD, and each commit made since the task started with the files it changed against its first parent. Double-clicking a commit or one of its files opens the parent-to-commit diff; commit files are history and are not rolled back. Rollback in this view restores files to HEAD. Accept still brings over the task's whole work, committed or not. View Options > Group By > Commit switches back to the flat since-start list; tasks in the project's own checkout keep that list.

Co-authored-by: panda-student <[email protected]>
2026-11-12 21:10:33 +08:00
arookieofcтаa-rookie-of-C-language 646e5f8364 fix(vcs): explain a mode-only change in the diff viewer (#2864)
A chmod-only change is a real modification: git status lists it and a commit records the new mode, so status keeps reporting it (core.fileMode=false still hides it). The diff viewer used to show "Contents are identical" with no explanation; when both texts are equal but the git modes differ it now says "File mode changed: 100644 → 100755". The modes come from a new vcs_file_mode command (index/tree entry mode for revisions, worktree permissions for the working copy).

Closes #2853

Co-authored-by: a-rookie-of-C-language <[email protected]>
2026-11-12 21:01:13 +08:00
Mr0Bread a3b48b78fd fix(editor): keep narrow editor groups usable with vertical tabs (#2962)
With tabs placed left or right, the tab column was sized to its longest label (140–260 px) and never shrank, so in a narrow split group a long file name left the editor almost no room. The column is now capped at 40% of its group; long labels truncate and keep the full path in the tooltip. Top and bottom placement are unchanged.

Co-authored-by: Mr0Bread <[email protected]>
2026-11-12 20:55:15 +08:00
Ernstjan FreriksтаEjaYF 5e4fa861bb fix(agents): move the caret with Home and End in the agent composer on macOS (#2943)
WebKit maps Home / End in a text box to scrolling and leaves the caret where it is, so they did nothing in the agent composer on macOS. The composer now moves the caret to the start or end of the visual line (Shift extends the selection); with ⌘, Ctrl or Alt the keys keep their native behaviour, and other platforms are unchanged.

Co-authored-by: EjaYF <[email protected]>
2026-11-12 20:55:06 +08:00
hwlLL0703 068ce8e721 feat(maven): import external Maven modules from Project Structure (#2922)
A Maven project outside the opened root can be linked explicitly: Add Maven Projects accepts an external pom.xml, and Project Structure › Modules › Import Module from Existing Sources (also on the project root's context menu) imports a directory with a pom.xml. The linked project becomes an extra project-tree root and an independent Maven project with its own working directory and wrapper; links are stored as canonical paths, survive reloads and reopening, and unlinking removes only the association. The opened project's jdtls gets the external directory as an extra workspace folder, keeping automatic discovery under the opened root. Neither POM is edited.

Merged with main's aggregation fix: a project an aggregator lists later becomes its module only when the aggregator lies inside the opened root and is not on the same chain, so a linked sibling does not take over the opened project.

Closes #2884

Co-authored-by: hwlLL0703 <[email protected]>
2026-11-12 20:52:00 +08:00
l0ng-ai e5cc1ed712 fix(http-client): show the request in the tool window while it is sent (#2966)
The HTTP Client tool window opened only once a response arrived, so slow requests looked unanswered. It now opens as the request is sent, and the response pane shows the request line, a progress indicator and Stop Request until the response replaces it.

Fixes #2937
2026-11-12 20:40:59 +08:00
l0ng-ai abe6285629 feat(app): resizable New Project dialog (#2965)
The New Project dialog gets a resize grip and remembers its size; the template list and settings pane grow with it. It opens at the same default size as before.

Part of #2903
2026-11-12 20:36:00 +08:00
l0ng-ai 66a829c24c fix(maven): an aggregator in a subfolder holds the root POM it lists as ../ (#2963)
The root POM was found first as a top-level project and skipped when parent/pom.xml listed it again as <module>../</module>, and module paths kept their '..', so the Maven tool window showed the root module beside the aggregator. Module paths are normalised and a project an aggregator lists later becomes its module.

Fixes #2877
2026-11-12 20:32:00 +08:00
l0ng-ai 04f3f9434f fix(remote): serve Open in Browser and the built-in preview from the remote host (#2961)
In a remote window the built-in web server ran on the client and read the local file system, so URLs for remote files were 404s. The server now lives in the host: openidea-server answers web_url with its own server on the remote loopback, and the client forwards that port over the SSH or dev container connection and returns the forwarded URL. The Host check accepts any port with 127.0.0.1 / localhost, since the forwarded port can differ.

Fixes #2915
2026-11-12 20:18:11 +08:00
l0ng-ai 4330034403 perf(remote): list the deployment tree concurrently and report sync phases (#2959)
Sync before run listed the remote project one folder after another and checked each external classpath entry in turn, a round trip each, silently. Folders are now listed 16 at a time and classpath entries checked the same way; the run console reports the comparing / checking phases and the elapsed time. The walk still visits every folder, so results are unchanged.

Part of #2873
2026-11-12 20:05:43 +08:00
l0ng-ai de8afe31d2 fix(remote): finish SSH runs whose channel outlives the process (#2957)
A remote run finished only when its SSH channel closed, so a channel that outlived the process (a child holding the inherited stdout, or a server that never closes it) left the run, and Stop / Rerun, hanging. The exit status now ends the process once the channel has been quiet for a second, and Stop finishes the run locally when the channel still reports nothing 3 s after KILL.

Fixes #2872
2026-11-12 20:02:25 +08:00
l0ng-ai 579e5eee9b feat(languages): add Prisma (#2955)
Prisma schema files get highlighting, structure view and folding from the bundled tree-sitter-prisma-io grammar, a file icon, and code insight from @prisma/language-server (installable from the IDE). The language definition provides the `prisma` settings section the server reads before each validation; without it the server crashed.

Closes #2926
2026-11-12 20:00:13 +08:00
l0ng-ai 099ede3478 fix(vcs): Commit… from Local Changes includes the selected files and focuses the message (#2952)
With the modal commit interface, Commit… (⌘K) and Commit and Push… invoked from Local Changes include just the selected files, and the commit message field takes focus. Invoked elsewhere, the dialog starts with the default inclusion instead of the previous dialog's leftovers.

Fixes #2923
2026-11-12 19:33:15 +08:00
l0ng-ai fb269afdfa fix(lsp): run every server's fix-all on save, and give ESLint its codeActionOnSave setting (#2950)
Run code cleanup applied one source.fixAll action, so in TypeScript files vtsls's won and ESLint's was dropped; fix-all now runs once per server that offers it, each asked again after the previous edit. ESLint rejected every fix-all request because the built-in settings lacked codeActionOnSave; all language files attaching eslint now send { mode = "all", rules = [] }. Failed save actions are logged.

Fixes #2927
Fixes #2928
2026-11-12 19:23:59 +08:00
Alexanderтаformiat 435b00d893 feat(search): colour Find in Files preview matches with the editor's search result colour (#2949)
Matches in the Find in Files and Search Everywhere preview use the editor scheme's search result background (IntelliJ's SEARCH_RESULT_ATTRIBUTES, new token --bg-search-result with values for every built-in theme) instead of the lists' SearchMatch yellow, as a background only so syntax colours stay readable. SEARCH_RESULT_ATTRIBUTES now imports into the new token; --bg-search-match keeps coming from SearchMatch.

Closes #2858

Co-authored-by: formiat <[email protected]>
2026-11-12 19:20:34 +08:00
jiejie1133 14fe542617 fix(editor): leave the caret after a completion snippet without fields (#2939)
CodeMirror leaves the caret before an inserted snippet that has no fields, so accepting a field like `val` at `root.` put the caret before it. LSP snippets without tab stops, choices or unresolved variables now get a final ${0} stop; SQL functions without parameters had the same problem and get one too. Snippets with fields keep their selection.

Closes #2936

Co-authored-by: jiejie1133 <[email protected]>
2026-11-12 19:19:37 +08:00
persicaTaoтаpersicaTao 513762147a fix(db): do not require TLS when SQL Server SSL is disabled (#2938)
Tiberius defaults to EncryptionLevel::Required with rustls, and trust_cert only skips certificate checks, so a SQL Server data source with SSL = Disable still negotiated TLS and timed out against a server without encryption (or one rustls can't speak TLS with). Disable now selects NotSupported; Prefer still trusts the certificate, Require and Verify keep validated TLS.

Closes #2917

Co-authored-by: persicaTao <[email protected]>
2026-11-12 19:18:49 +08:00
l0ng-ai 489f55f52c fix(vcs): don't commit a file that was added to the index and then deleted (#2947)
A file added to the index and then deleted from disk is listed nowhere in the Commit view, but it stayed in the index copy the commit is made from, so committing other files committed it too. Such staged entries are now excluded like unchecked staged files.

Fixes #2908
2026-11-12 19:15:23 +08:00
835 змінених файлів з 40232 додано та 2922 видалено

No files matched your search

+1 -1
Переглянути файл
@@ -47,7 +47,7 @@ jobs:
exit 0
fi
# No `nightly` tag yet (the first run): compare fails and the build goes on.
status="$(gh api "repos/$GH_REPO/compare/nightly...$sha" --jq .status 2>/dev/null || echo ahead)"
status="$(gh api "repos/$GH_REPO/compare/nightly...$sha" --jq .status 2>/dev/null)" || status=ahead
if [ "$status" != ahead ] && [ "$FORCE" != true ]; then
echo "::notice::$sha is not newer than the last nightly"
exit 0
+9
Переглянути файл
@@ -151,6 +151,13 @@ jobs:
$hash = (Get-FileHash $asset -Algorithm SHA256).Hash.ToLower()
"$hash $asset" | Out-File -Encoding ascii "$asset.sha256"
gh release upload $env:TAG $asset "$asset.sha256" --clobber
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
if ($env:TAURI_SIGNING_PRIVATE_KEY) {
pnpm exec tauri signer sign --app-version $version $asset
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
gh release upload $env:TAG "$asset.sig" --clobber
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
}
# The remote development server, per host platform it can run on. The desktop
# app downloads the one for the host (ide-tools `server`) and installs it over SSH.
@@ -233,6 +240,8 @@ jobs:
[/_x64\.app\.tar\.gz$/, ["darwin-x86_64", "darwin-x86_64-app"]],
...windows("aarch64", /_arm64_en-US\.msi$/, /_arm64-setup\.exe$/),
...windows("x86_64", /_x64_en-US\.msi$/, /_x64-setup\.exe$/),
[/_windows_arm64_portable\.zip$/, ["windows-aarch64-portable"]],
[/_windows_x64_portable\.zip$/, ["windows-x86_64-portable"]],
];
const platforms = {};
for (const sig of sigs) {
Створено
+69 -51
Переглянути файл
@@ -4144,7 +4144,7 @@ checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "ide-agent"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dunce",
"encoding_rs",
@@ -4168,7 +4168,7 @@ dependencies = [
[[package]]
name = "ide-analysis"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ast-grep-core",
"ide-core",
@@ -4182,7 +4182,7 @@ dependencies = [
[[package]]
name = "ide-android"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"serde",
@@ -4193,7 +4193,7 @@ dependencies = [
[[package]]
name = "ide-build"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"blake3",
"chardetng",
@@ -4211,7 +4211,7 @@ dependencies = [
[[package]]
name = "ide-cli"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dirs 6.0.0",
"serde",
@@ -4225,7 +4225,7 @@ dependencies = [
[[package]]
name = "ide-clipboard"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"gtk",
"objc2 0.6.5",
@@ -4235,7 +4235,7 @@ dependencies = [
[[package]]
name = "ide-core"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dunce",
"encoding_rs",
@@ -4252,7 +4252,7 @@ dependencies = [
[[package]]
name = "ide-coverage"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"ide-run",
@@ -4266,7 +4266,7 @@ dependencies = [
[[package]]
name = "ide-dap"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"futures",
"ide-core",
@@ -4292,20 +4292,24 @@ dependencies = [
[[package]]
name = "ide-db"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"base64 0.22.1",
"bb8",
"blake3",
"dirs 6.0.0",
"dunce",
"flate2",
"futures",
"ide-core",
"ide-http",
"ide-ssh",
"notify",
"open",
"oracledb",
"parking_lot",
"reqwest",
"rsa",
"rustls",
"serde",
"serde_json",
@@ -4319,11 +4323,12 @@ dependencies = [
"toml_edit 0.25.17+spec-1.1.0",
"tracing",
"uuid",
"zeroize",
]
[[package]]
name = "ide-docker"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"serde",
@@ -4336,7 +4341,7 @@ dependencies = [
[[package]]
name = "ide-extension"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4357,7 +4362,7 @@ dependencies = [
[[package]]
name = "ide-fonts"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"objc2-core-foundation",
"objc2-core-text",
@@ -4369,7 +4374,7 @@ dependencies = [
[[package]]
name = "ide-github"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"ide-vcs",
@@ -4382,7 +4387,7 @@ dependencies = [
[[package]]
name = "ide-heap"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"criterion",
"ide-core",
@@ -4400,7 +4405,7 @@ dependencies = [
[[package]]
name = "ide-host"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -4443,6 +4448,7 @@ dependencies = [
"ide-tools",
"ide-vcs",
"ide-vfs",
"ide-web",
"ide-workspace",
"ignore",
"jsonschema",
@@ -4470,7 +4476,7 @@ dependencies = [
[[package]]
name = "ide-http"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"ide-vfs",
@@ -4489,7 +4495,7 @@ dependencies = [
[[package]]
name = "ide-index"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"blake3",
"criterion",
@@ -4511,7 +4517,7 @@ dependencies = [
[[package]]
name = "ide-jdwp"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-jvm-inspect",
"proptest",
@@ -4523,7 +4529,7 @@ dependencies = [
[[package]]
name = "ide-jsonrpc"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"encoding_rs",
"futures",
@@ -4540,7 +4546,7 @@ dependencies = [
[[package]]
name = "ide-jvm-attach"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"libc",
"tempfile",
@@ -4549,7 +4555,7 @@ dependencies = [
[[package]]
name = "ide-jvm-inspect"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"futures",
"serde",
@@ -4560,7 +4566,7 @@ dependencies = [
[[package]]
name = "ide-language"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"blake3",
"criterion",
@@ -4601,6 +4607,7 @@ dependencies = [
"tree-sitter-nix",
"tree-sitter-ocaml",
"tree-sitter-php",
"tree-sitter-prisma-io",
"tree-sitter-proto",
"tree-sitter-python",
"tree-sitter-ruby",
@@ -4616,7 +4623,7 @@ dependencies = [
[[package]]
name = "ide-lsp"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dunce",
"encoding_rs",
@@ -4644,7 +4651,7 @@ dependencies = [
[[package]]
name = "ide-mcp"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"bytes",
"dunce",
@@ -4663,7 +4670,7 @@ dependencies = [
[[package]]
name = "ide-notebook"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"futures",
"ide-core",
@@ -4686,7 +4693,7 @@ dependencies = [
[[package]]
name = "ide-plugin"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"blake3",
"dunce",
@@ -4709,7 +4716,7 @@ dependencies = [
[[package]]
name = "ide-profiler"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"ide-jvm-attach",
@@ -4723,9 +4730,10 @@ dependencies = [
[[package]]
name = "ide-remote"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dunce",
"futures",
"globset",
"ide-core",
"ide-run",
@@ -4742,7 +4750,7 @@ dependencies = [
[[package]]
name = "ide-run"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dunce",
"encoding_rs",
@@ -4766,7 +4774,7 @@ dependencies = [
[[package]]
name = "ide-scaffold"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"ide-tools",
@@ -4782,7 +4790,7 @@ dependencies = [
[[package]]
name = "ide-sdk"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dunce",
"ide-core",
@@ -4803,7 +4811,7 @@ dependencies = [
[[package]]
name = "ide-search"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"criterion",
"encoding_rs",
@@ -4824,7 +4832,7 @@ dependencies = [
[[package]]
name = "ide-spell"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"parking_lot",
@@ -4837,7 +4845,7 @@ dependencies = [
[[package]]
name = "ide-ssh"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dirs 6.0.0",
"flate2",
@@ -4862,7 +4870,7 @@ dependencies = [
[[package]]
name = "ide-syntax"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"ide-language",
@@ -4877,7 +4885,7 @@ dependencies = [
[[package]]
name = "ide-terminal"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"ide-core",
"libc",
@@ -4890,7 +4898,7 @@ dependencies = [
[[package]]
name = "ide-text"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"criterion",
"ide-core",
@@ -4903,7 +4911,7 @@ dependencies = [
[[package]]
name = "ide-tools"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"flate2",
"hex",
@@ -4924,7 +4932,7 @@ dependencies = [
[[package]]
name = "ide-vcs"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"blake3",
"dunce",
@@ -4945,7 +4953,7 @@ dependencies = [
[[package]]
name = "ide-vfs"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"chardetng",
"dunce",
@@ -4969,7 +4977,7 @@ dependencies = [
[[package]]
name = "ide-web"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"dirs 6.0.0",
"ide-core",
@@ -4985,7 +4993,7 @@ dependencies = [
[[package]]
name = "ide-workspace"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"blake3",
"dunce",
@@ -6586,7 +6594,7 @@ dependencies = [
[[package]]
name = "openidea"
version = "0.1.43"
version = "0.1.44"
dependencies = [
"base64 0.22.1",
"blake3",
@@ -8719,7 +8727,7 @@ checksum = "3a0219bd7d979d58245a4f41f695e1ac9f8befdffadd7f61f1bae9e39abc6620"
[[package]]
name = "sqlx"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"sqlx-core",
"sqlx-macros",
@@ -8731,7 +8739,7 @@ dependencies = [
[[package]]
name = "sqlx-core"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"base64 0.22.1",
"bytes",
@@ -8766,7 +8774,7 @@ dependencies = [
[[package]]
name = "sqlx-macros"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"proc-macro2",
"quote",
@@ -8778,7 +8786,7 @@ dependencies = [
[[package]]
name = "sqlx-macros-core"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"cfg-if",
"dotenvy",
@@ -8802,7 +8810,7 @@ dependencies = [
[[package]]
name = "sqlx-mysql"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"bitflags 2.13.2",
"byteorder",
@@ -8827,7 +8835,7 @@ dependencies = [
[[package]]
name = "sqlx-postgres"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"atoi",
"base64 0.22.1",
@@ -8861,7 +8869,7 @@ dependencies = [
[[package]]
name = "sqlx-sqlite"
version = "0.9.0"
source = "git+https://github.com/transact-rs/sqlx?rev=10462fa4414c6c7f1ed1d39bb31678e3b726878c#10462fa4414c6c7f1ed1d39bb31678e3b726878c"
source = "git+https://github.com/l0ng-ai/sqlx?rev=9894985a1babc7e6bc0437da4fd43e3143184073#9894985a1babc7e6bc0437da4fd43e3143184073"
dependencies = [
"atoi",
"flume",
@@ -10437,6 +10445,16 @@ dependencies = [
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-prisma-io"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c35912d65a6adb07f781db46d843bf0464935e267da6ed192705516adafc7c24"
dependencies = [
"cc",
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-proto"
version = "0.6.0"
+2 -2
Переглянути файл
@@ -5,7 +5,7 @@ members = ["crates/*", "src-tauri"]
exclude = ["sdk", "plugins"]
[workspace.package]
version = "0.1.43"
version = "0.1.44"
edition = "2024"
rust-version = "1.96"
license = "Apache-2.0"
@@ -111,7 +111,7 @@ reqwest = { version = "0.13", default-features = false, features = ["rustls-no-p
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
rustls-platform-verifier = "0.7"
# Until SQLx releases its libsqlite3-sys 0.39 update (RUSTSEC-2026-0459).
sqlx = { version = "0.9", git = "https://github.com/transact-rs/sqlx", rev = "10462fa4414c6c7f1ed1d39bb31678e3b726878c", default-features = false, features = ["runtime-tokio", "tls-rustls-ring-webpki", "sqlite", "postgres", "mysql"] }
sqlx = { version = "0.9", git = "https://github.com/l0ng-ai/sqlx", rev = "9894985a1babc7e6bc0437da4fd43e3143184073", default-features = false, features = ["runtime-tokio", "tls-rustls-ring-webpki", "sqlite", "postgres", "mysql"] }
tiberius = { version = "0.13", default-features = false, features = ["tds73", "rustls", "sql-browser-tokio"] }
# Pure-Rust thin Oracle driver; pinned to the beta until it stabilizes.
oracledb = "=26.0.0-beta.5"
+1
Переглянути файл
@@ -107,6 +107,7 @@ fn manager(dir: &Path, agent: Arc<FakeV2Agent>) -> AgentManager {
models_file: dir.join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
+1
Переглянути файл
@@ -154,6 +154,7 @@ async fn discovery_reads_effort_and_fast_from_session_new_then_stops_the_agent()
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
+62 -3
Переглянути файл
@@ -42,9 +42,38 @@ pub(crate) fn bypass_approvals(config: &[AgentConfigOption]) -> bool {
let selected = |category: &str| {
config.iter().find(|o| o.category.as_deref() == Some(category)).and_then(|o| o.current.as_deref())
};
selected("mode").is_some_and(is_bypass_mode)
|| selected("permissions").is_some_and(|v| matches!(v.to_ascii_lowercase().as_str(), "on" | "true"))
|| selected("approval").is_some_and(|v| v.eq_ignore_ascii_case("bypass"))
["mode", "permissions", "approval"]
.into_iter()
.any(|category| selected(category).zip(bypass_value(category)).is_some_and(|(value, bypass)| bypass(value)))
}
/// For an option of `category` that can approve everything, what tells its approving values.
fn bypass_value(category: &str) -> Option<fn(&str) -> bool> {
match category {
"mode" => Some(is_bypass_mode),
"permissions" => Some(|v| matches!(v.to_ascii_lowercase().as_str(), "on" | "true")),
"approval" => Some(|v| v.eq_ignore_ascii_case("bypass")),
_ => None,
}
}
/// Moves options that approve everything without asking to a value that asks: the agent's
/// default or ask mode, else its first one that asks.
pub(crate) fn ask_for_approvals(config: &mut [AgentConfigOption]) {
for option in config {
let Some(bypass) = option.category.as_deref().and_then(bypass_value) else { continue };
if option.current.as_deref().is_some_and(|c| !bypass(c)) {
continue;
}
let asks = |v: &&AgentConfigValue| !bypass(&v.value);
let value = ["default", "ask"]
.iter()
.find_map(|preferred| option.values.iter().filter(asks).find(|v| v.value == *preferred))
.or_else(|| option.values.iter().find(asks));
if let Some(value) = value {
option.current = Some(value.value.clone());
}
}
}
/// The mode that approves everything without asking (`bypassPermissions`, `yolo`, `full-access`…), if the agent has one.
@@ -341,6 +370,36 @@ mod tests {
assert!(bypass_approvals(&url_mode));
}
#[test]
fn asking_for_approvals_keeps_a_mode_that_asks_and_leaves_one_that_does_not() {
let mode = |current: &str, values: &[&str]| {
options_of(&json!({"modes": {"currentModeId": current,
"availableModes": values.iter().map(|v| json!({"id": v})).collect::<Vec<_>>()}}))
};
let codex = ["read-only", "ask", "auto", "full-access"];
for (current, expected) in [("auto", "auto"), ("read-only", "read-only"), ("full-access", "ask")] {
let mut c = mode(current, &codex);
ask_for_approvals(&mut c);
assert_eq!(c[0].current.as_deref(), Some(expected), "{current}");
assert!(!bypass_approvals(&c));
}
let mut claude = mode("bypassPermissions", &["acceptEdits", "default", "plan", "bypassPermissions"]);
ask_for_approvals(&mut claude);
assert_eq!(claude[0].current.as_deref(), Some("default"));
let mut other = mode("yolo", &["yolo", "code", "architect"]);
ask_for_approvals(&mut other);
assert_eq!(other[0].current.as_deref(), Some("code"));
let mut c = options_of(&json!({"configOptions": [
{"id": "allow_all", "category": "permissions", "type": "select", "currentValue": "on",
"options": [{"value": "on"}, {"value": "off"}]},
{"id": "approval", "category": "approval", "type": "select", "currentValue": "bypass",
"options": [{"value": "bypass"}, {"value": "ask"}]}
]}));
ask_for_approvals(&mut c);
assert_eq!(ids(&c), [("allow_all", Some("off")), ("approval", Some("ask"))]);
}
#[test]
fn effort_and_fast_categories_are_inferred_from_ids() {
let c = options_of(&json!({
+82 -19
Переглянути файл
@@ -43,6 +43,9 @@ pub type AgentSink = Arc<dyn Fn(AgentEvent) + Send + Sync>;
/// Returns the active command execution security policy for the project.
pub type SecurityHook = Arc<dyn Fn() -> ide_core::settings::AgentSecuritySettings + Send + Sync>;
/// Returns the user's choice of the approvals new tasks start with.
pub type NewTaskApprovalsHook = Arc<dyn Fn() -> ide_core::settings::NewTaskApprovals + Send + Sync>;
/// Entry changes are batched for this long before they are sent.
const FLUSH_DELAY: Duration = Duration::from_millis(40);
/// Updates that arrive this late after `session/load` returns are still replayed history.
@@ -89,6 +92,18 @@ pub struct AgentPaths {
pub custom_agents_dir: Option<PathBuf>,
/// The user's home: its `.claude` holds their own skills and commands.
pub home_dir: Option<PathBuf>,
/// Where the values of the agent files' environment variables are kept; without it they stay in the files.
pub env_secrets: Option<Arc<dyn EnvSecrets>>,
}
/// Keeps the values of agent files' environment variables (`agent/<id>/env/<name>`) out of
/// the files, which keep only the names; the app keeps them in the OS credential store.
pub trait EnvSecrets: Send + Sync {
fn get(&self, key: &str) -> Option<String>;
fn set(&self, key: &str, value: &str) -> IdeResult<()>;
fn delete(&self, key: &str);
/// The keys that start with `prefix`.
fn keys(&self, prefix: &str) -> Vec<String>;
}
/// Starts the ACP server of a provider in a directory, speaking to `client`.
@@ -107,6 +122,7 @@ struct Shared {
connector: Connector,
sink: RwLock<Option<AgentSink>>,
security: RwLock<Option<SecurityHook>>,
new_task_approvals: RwLock<Option<NewTaskApprovalsHook>>,
known: Mutex<HashMap<String, Known>>,
providers: RwLock<Vec<provider::ProviderSpec>>,
/// Last probe of each provider.
@@ -136,6 +152,10 @@ impl Shared {
self.providers.read().iter().find(|p| p.id == id).cloned()
}
fn env_secrets(&self) -> Option<&dyn EnvSecrets> {
self.paths.env_secrets.as_deref()
}
fn emit(&self, event: AgentEvent) {
let sink = self.sink.read().clone();
if let Some(sink) = sink {
@@ -155,6 +175,11 @@ impl Shared {
// Model-specific options are offered separately, never as provider-wide defaults.
config.retain(|o| o.category.as_deref() != Some("thought_level"));
config::add_client_options(provider, &mut config);
let hook = self.new_task_approvals.read().clone();
if hook.is_some_and(|ask| ask() == ide_core::settings::NewTaskApprovals::Ask) {
config::ask_for_approvals(&mut config);
return config;
}
if let Some((o, v)) = config::bypass_mode(&config).map(|(o, v)| (o.id.clone(), v.value.clone()))
&& let Some(o) = config.iter_mut().find(|c| c.id == o)
{
@@ -446,13 +471,14 @@ impl AgentManager {
fn with_connector(paths: AgentPaths, runtime: tokio::runtime::Handle, connector: Connector) -> Self {
let known =
std::fs::read(&paths.models_file).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default();
let providers = RwLock::new(provider::load(paths.custom_agents_dir.as_deref()));
let providers = RwLock::new(provider::load(paths.custom_agents_dir.as_deref(), paths.env_secrets.as_deref()));
let shared = Arc::new(Shared {
paths,
runtime,
connector,
sink: RwLock::new(None),
security: RwLock::new(None),
new_task_approvals: RwLock::new(None),
known: Mutex::new(known),
providers,
probes: Mutex::new(HashMap::new()),
@@ -501,9 +527,14 @@ impl AgentManager {
*self.shared.security.write() = Some(hook);
}
/// Sets the callback that reads which approvals new tasks start with.
pub fn set_new_task_approvals_hook(&self, hook: NewTaskApprovalsHook) {
*self.shared.new_task_approvals.write() = Some(hook);
}
/// The agents (custom ones re-read), whether each is installed, and what they offered last time.
pub fn providers(&self) -> Vec<AgentProvider> {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref(), self.shared.env_secrets());
let in_use: std::collections::HashSet<String> =
self.tasks.read().values().map(|t| t.state.lock().task.provider.clone()).collect();
let list = specs
@@ -537,7 +568,7 @@ impl AgentManager {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::set_builtin_command(&dir, id, command.as_deref())
provider::set_builtin_command(&dir, self.shared.env_secrets(), id, command.as_deref())
}
/// Sets the environment variables built-in `id` starts with; empty clears them.
@@ -545,7 +576,7 @@ impl AgentManager {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::set_builtin_env(&dir, id, env)
provider::set_builtin_env(&dir, self.shared.env_secrets(), id, env)
}
/// Saves a new custom ACP agent through its agent file.
@@ -553,7 +584,7 @@ impl AgentManager {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::save_custom(&dir, &input)
provider::save_custom(&dir, self.shared.env_secrets(), &input)
}
/// Removes a custom agent's file (registry installs go through `registry_remove`).
@@ -561,7 +592,7 @@ impl AgentManager {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::remove_custom(&dir, id)
provider::remove_custom(&dir, self.shared.env_secrets(), id)
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom agents.
@@ -569,12 +600,12 @@ impl AgentManager {
let Some(dir) = self.shared.paths.custom_agents_dir.clone() else {
return Err(IdeError::unavailable("there is no custom agents directory"));
};
provider::import_acp(&dir, text)
provider::import_acp(&dir, self.shared.env_secrets(), text)
}
/// Re-reads configured/installed agents, drops cached probes, and notifies subscribers.
pub fn reload_providers(&self) {
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref());
let specs = provider::load(self.shared.paths.custom_agents_dir.as_deref(), self.shared.env_secrets());
*self.shared.providers.write() = specs;
self.shared.probes.lock().clear();
self.shared.emit(AgentEvent::Providers);
@@ -1343,6 +1374,15 @@ impl AgentManager {
blocking(move || at.changes(&before)).await
}
/// The task's uncommitted changes and, for a worktree task, the commits made since it
/// started (newest first). Both are read against the same `HEAD`.
pub async fn grouped_changes(&self, id: &str) -> IdeResult<AgentChanges> {
let task = self.task(id)?;
let at = task.base();
let before = task.state.lock().before.clone();
blocking(move || at.grouped_changes(&before)).await
}
/// Content of `path` (in the task's directory) when the task started.
pub async fn base_text(&self, id: &str, path: PathBuf) -> IdeResult<Option<String>> {
let at = self.task(id)?.base();
@@ -1353,9 +1393,14 @@ impl AgentManager {
.await
}
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as they
/// were when it started (files the user had changed then, as the user left them).
pub async fn rollback_changes(&self, id: &str, rel_paths: Option<Vec<String>>) -> IdeResult<()> {
/// Rollback: puts the task's changed files among `rel_paths` (all with `None`) back as
/// `view` compares them: as they were when it started, or as they are in `HEAD`.
pub async fn rollback_changes(
&self,
id: &str,
rel_paths: Option<Vec<String>>,
view: AgentChangesView,
) -> IdeResult<()> {
let task = self.task(id)?;
if task.state.lock().busy {
return Err(IdeError::invalid("the agent is working: stop it before rolling back its changes"));
@@ -1366,8 +1411,10 @@ impl AgentManager {
(s.before.clone(), s.saved.as_ref().map(|name| self.shared.paths.store_dir.join(name)))
};
let restamped = blocking(move || {
let changes = at.changes(&before)?;
let rev = at.rev()?;
let (changes, rev) = match view {
AgentChangesView::Group => (at.grouped_changes(&before)?.uncommitted, worktree::head_commit(at.cwd())?),
AgentChangesView::SinceStart => (at.changes(&before)?, at.rev()?),
};
let sources = match rev.as_deref() {
Some(rev) => worktree::rename_sources(at.cwd(), rev)?,
None => Default::default(),
@@ -2250,15 +2297,19 @@ impl Task {
.iter()
.map(|question| question.answer_key.clone().unwrap_or_else(|| question.question.clone()))
.collect();
let command = security::extract_command(params);
// What the card names as the command: not for a request that grants more than running it.
let shown = command.as_deref().filter(|_| !security::widens_access(params)).map(shown_command);
if let Some(hook) = self.shared.security.read().clone()
&& let Some(command) = security::extract_command(params)
&& let Some(command) = command.as_deref()
{
match security::evaluate_command(&command, &hook()) {
match security::evaluate_command(command, &hook()) {
security::PolicyDecision::Blocked(pattern) => {
let reject = options.iter().find(|o| o.kind.starts_with("reject")).cloned();
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
command: shown.clone(),
options,
questions: Vec::new(),
chosen: Some(format!("Blocked by policy ({pattern})")),
@@ -2268,7 +2319,7 @@ impl Task {
None => json!({ "outcome": { "outcome": "cancelled" } }),
};
}
security::PolicyDecision::Allowed => {
security::PolicyDecision::Allowed if !security::widens_access(params) => {
if let Some(allow) = options
.iter()
.find(|o| o.kind == "allow_once")
@@ -2278,6 +2329,7 @@ impl Task {
let _ = self.push_entry(AgentEntry::Permission {
id,
title,
command: shown.clone(),
options,
questions: Vec::new(),
chosen: Some(format!("{} (whitelisted)", allow.name)),
@@ -2285,12 +2337,18 @@ impl Task {
return json!({ "outcome": { "outcome": "selected", "optionId": allow.id } });
}
}
security::PolicyDecision::AskUser => {}
security::PolicyDecision::Allowed | security::PolicyDecision::AskUser => {}
}
}
let (tx, rx) = std::sync::mpsc::channel();
let entry =
self.push_entry(AgentEntry::Permission { id, title, options: options.clone(), questions, chosen: None });
let entry = self.push_entry(AgentEntry::Permission {
id,
title,
command: shown,
options: options.clone(),
questions,
chosen: None,
});
self.permissions.lock().insert(id, tx);
self.set_status(AgentStatus::NeedsInput, None);
self.approve_pending();
@@ -2452,6 +2510,11 @@ fn permission_title(params: &Value) -> String {
title.unwrap_or("The agent asks for permission").to_owned()
}
/// A command as a permission card shows it: the script of a `<shell> -c <script>` line.
fn shown_command(command: &str) -> String {
shell_script(command).unwrap_or_else(|| command.to_owned())
}
/// The script of a `<shell> -c <quoted script>` command line, when the whole line is exactly that.
pub(crate) fn shell_script(command: &str) -> Option<String> {
let (shell, rest) = command.split_once(' ')?;
+44 -1
Переглянути файл
@@ -384,6 +384,10 @@ pub enum AgentEntry {
Permission {
id: u32,
title: String,
/// The shell command the request asks to run, from its structured input; `None` for
/// any other request, whatever its title says.
#[serde(default)]
command: Option<String>,
options: Vec<AgentPermissionOption>,
/// Set when the request asks `AskUserQuestion`'s questions; answered with a JSON object
/// of question to answer instead of an option.
@@ -484,7 +488,8 @@ pub struct NewAgentTask {
pub base: Option<String>,
}
/// A file the task changed, relative to [`AgentTask::base_commit`].
/// A file the task changed: against [`AgentTask::base_commit`], or against `HEAD` in the
/// grouped view.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentChange {
@@ -493,6 +498,44 @@ pub struct AgentChange {
pub kind: ide_vcs::ChangeKind,
pub added: u32,
pub removed: u32,
/// Path before a rename; the left side of its diff reads this path.
pub old_path: Option<String>,
}
/// A commit made while the task ran, with the files it changed against its first parent.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentCommit {
pub id: String,
pub short_id: String,
pub summary: String,
pub message: String,
pub time_ms: f64,
/// First parent, or `None` for a root commit.
pub parent: Option<String>,
pub files: Vec<AgentChange>,
}
/// Which comparison a rollback of the task's changes uses.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum AgentChangesView {
/// Everything since the commit the task started from.
#[default]
SinceStart,
/// The uncommitted files against `HEAD`; commits made since the start stay.
Group,
}
/// A task's uncommitted files and the commits made since it started.
#[derive(Clone, Debug, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct AgentChanges {
/// Files changed against `HEAD`, less those changed before the task started and untouched since.
pub uncommitted: Vec<AgentChange>,
/// Commits from the task's start (exclusive) to `HEAD` (inclusive), newest first. Empty
/// for tasks working in the project's own checkout.
pub commits: Vec<AgentCommit>,
}
/// Streamed to the window that subscribed (`AgentManager::subscribe`).
+8 -7
Переглянути файл
@@ -869,13 +869,14 @@ impl CodexShim {
"item/commandExecution/requestApproval" => {
let choices = translate::decisions(params, true);
let title = translate::command_approval_title(params, item.as_ref());
let call = tool_call_of(
&item_id,
&title,
"execute",
reason,
Some(json!({ "command": params.get("command"), "cwd": params.get("cwd") })),
);
let mut input = json!({
"command": translate::approval_command(params, item.as_ref()),
"cwd": params.get("cwd"),
});
if let Some(network) = params.get("networkApprovalContext").filter(|c| !c.is_null()) {
input["networkApprovalContext"] = network.clone();
}
let call = tool_call_of(&item_id, &title, "execute", reason, Some(input));
Ok(self.ask(&thread, call, &choices, decline))
}
"item/fileChange/requestApproval" => {
+12 -8
Переглянути файл
@@ -757,19 +757,23 @@ pub(super) fn answer_of(choices: &[Choice], outcome: &Value, decline: Value) ->
choices.iter().find(|c| selected && c.option["optionId"].as_str() == chosen).map_or(decline, |c| c.answer.clone())
}
/// The command a command approval is about: the request's own, else its item's.
pub(super) fn approval_command<'a>(params: &'a Value, item: Option<&'a Value>) -> Option<&'a str> {
// Approval payloads can describe a different command from the item's earlier summary.
let item = item.filter(|i| i.get("id") == params.get("itemId"));
params
.get("command")
.and_then(Value::as_str)
.filter(|c| !c.is_empty())
.or_else(|| item.and_then(|i| i.get("command")).and_then(Value::as_str).filter(|c| !c.is_empty()))
}
/// The ACP title of a command approval.
pub(super) fn command_approval_title(params: &Value, item: Option<&Value>) -> String {
if let Some(host) = params.pointer("/networkApprovalContext/host").and_then(Value::as_str) {
return format!("Allow network access to {host}");
}
// Approval payloads can describe a different command from the item's earlier summary.
let item = item.filter(|i| i.get("id") == params.get("itemId"));
let command = params
.get("command")
.and_then(Value::as_str)
.filter(|c| !c.is_empty())
.or_else(|| item.and_then(|i| i.get("command")).and_then(Value::as_str).filter(|c| !c.is_empty()));
match command {
match approval_command(params, item) {
Some(command) => format!("Run {}", crate::shell_script(command).unwrap_or_else(|| command.to_owned())),
None => "Run a command".to_owned(),
}
+219 -68
Переглянути файл
@@ -2,7 +2,7 @@
//! `<config>/agents/<id>.toml` (written by hand, or by installing one from the ACP
//! Registry, see `registry`).
use crate::{AcpImportOutcome, AgentCommandReadOnlyReason, AgentProvider, AgentSource, CustomAgentInput};
use crate::{AcpImportOutcome, AgentCommandReadOnlyReason, AgentProvider, AgentSource, CustomAgentInput, EnvSecrets};
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -231,23 +231,40 @@ pub(crate) fn describe(spec: &ProviderSpec) -> AgentProvider {
/// Overrides built-in `id`'s command through its agent file, preserving other
/// customizations. `None` restores the built-in command and removes a redundant file.
pub(crate) fn set_builtin_command(custom_dir: &Path, id: &str, command: Option<&str>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, builtin| agent.command = command.unwrap_or(&builtin.command).to_owned())
pub(crate) fn set_builtin_command(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
id: &str,
command: Option<&str>,
) -> IdeResult<()> {
update_builtin(custom_dir, secrets, id, |agent, builtin| {
agent.command = command.unwrap_or(&builtin.command).to_owned()
})
}
/// The environment built-in `id` starts with, through its agent file (empty clears it).
pub(crate) fn set_builtin_env(custom_dir: &Path, id: &str, env: BTreeMap<String, String>) -> IdeResult<()> {
update_builtin(custom_dir, id, |agent, _| agent.env = env)
pub(crate) fn set_builtin_env(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
id: &str,
env: BTreeMap<String, String>,
) -> IdeResult<()> {
update_builtin(custom_dir, secrets, id, |agent, _| agent.env = env)
}
/// Changes built-in `id`'s agent file; a file left with only the built-in's values is removed.
fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAgent, &ProviderSpec)) -> IdeResult<()> {
fn update_builtin(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
id: &str,
change: impl FnOnce(&mut CustomAgent, &ProviderSpec),
) -> IdeResult<()> {
let Some(builtin) = builtin_spec(id) else {
return Err(IdeError::invalid(format!("{id} is not a built-in agent")));
};
let file = custom_dir.join(format!("{id}.toml"));
let mut agent = if file.try_exists()? {
CustomAgent::read(&file).map_err(IdeError::invalid)?
CustomAgent::read_with(&file, id, secrets).map_err(IdeError::invalid)?
} else {
CustomAgent {
name: Some(builtin.name.clone()),
@@ -272,16 +289,20 @@ fn update_builtin(custom_dir: &Path, id: &str, change: impl FnOnce(&mut CustomAg
Err(e) if e.kind() != std::io::ErrorKind::NotFound => return Err(e.into()),
_ => {}
}
forget_env(secrets, id, &[]);
return Ok(());
}
agent.write(&file)?;
Ok(())
agent.write_with(&file, id, secrets)
}
/// Saves a new custom agent; refuses built-in and ACP Registry ids, taken ids
/// and an empty command. The id is also the agent file's name, so it must be
/// a safe file stem.
pub(crate) fn save_custom(custom_dir: &Path, input: &CustomAgentInput) -> IdeResult<()> {
pub(crate) fn save_custom(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
input: &CustomAgentInput,
) -> IdeResult<()> {
let id = validated_custom_id(&input.id)?;
if builtin_spec(&id).is_some() {
return Err(IdeError::invalid(format!("{id} is a built-in agent; change its command instead")));
@@ -306,12 +327,11 @@ pub(crate) fn save_custom(custom_dir: &Path, input: &CustomAgentInput) -> IdeRes
env: input.env.clone(),
registry: None,
}
.write(&file)?;
Ok(())
.write_with(&file, &id, secrets)
}
/// Removes a custom agent's file; refuses built-in, ACP Registry and missing ids.
pub(crate) fn remove_custom(custom_dir: &Path, id: &str) -> IdeResult<()> {
pub(crate) fn remove_custom(custom_dir: &Path, secrets: Option<&dyn EnvSecrets>, id: &str) -> IdeResult<()> {
let id = validated_custom_id(id)?;
let id = id.as_str();
if builtin_spec(id).is_some() {
@@ -325,14 +345,21 @@ pub(crate) fn remove_custom(custom_dir: &Path, id: &str) -> IdeResult<()> {
}
match std::fs::remove_file(&file) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e.into()),
_ => Ok(()),
_ => {
forget_env(secrets, id, &[]);
Ok(())
}
}
}
/// Imports a JetBrains `acp.json` file's `agent_servers` entries as custom
/// agents, one outcome per entry; entries that collide, lack a command or
/// have no usable name are skipped with a reason.
pub(crate) fn import_acp(custom_dir: &Path, text: &str) -> IdeResult<Vec<AcpImportOutcome>> {
pub(crate) fn import_acp(
custom_dir: &Path,
secrets: Option<&dyn EnvSecrets>,
text: &str,
) -> IdeResult<Vec<AcpImportOutcome>> {
#[derive(Deserialize)]
struct AcpJson {
#[serde(default)]
@@ -370,7 +397,7 @@ pub(crate) fn import_acp(custom_dir: &Path, text: &str) -> IdeResult<Vec<AcpImpo
env: server.env,
registry: None,
}
.write(&custom_dir.join(format!("{id}.toml")))?;
.write_with(&custom_dir.join(format!("{id}.toml")), &id, secrets)?;
outcome.saved = true;
}
outcomes.push(outcome);
@@ -400,7 +427,7 @@ fn slugify_id(name: &str) -> String {
}
/// An agent file, `<config>/agents/<id>.toml`.
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub(crate) struct CustomAgent {
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -447,11 +474,67 @@ impl CustomAgent {
std::fs::write(&tmp, text)?;
std::fs::rename(&tmp, path)
}
/// Reads agent `id`'s file with its environment values filled in from `secrets`; values
/// still written in the file (by hand, by an older version) move there first. A registry
/// install keeps the environment its registry entry gives it in the file.
pub(crate) fn read_with(path: &Path, id: &str, secrets: Option<&dyn EnvSecrets>) -> Result<Self, String> {
let mut agent = Self::read(path)?;
let Some(secrets) = secrets.filter(|_| agent.registry.is_none()) else {
return Ok(agent);
};
let written = agent.env.values().any(|v| !v.is_empty());
for (name, value) in agent.env.iter_mut() {
if value.is_empty()
&& let Some(saved) = secrets.get(&env_key(id, name))
{
*value = saved;
}
}
if written && let Err(e) = agent.write_with(path, id, Some(secrets)) {
tracing::warn!("agent {id}: could not move environment values out of {}: {}", path.display(), e.message);
}
Ok(agent)
}
/// Writes agent `id`'s file with the environment's names only, its values saved in `secrets`.
pub(crate) fn write_with(&self, path: &Path, id: &str, secrets: Option<&dyn EnvSecrets>) -> IdeResult<()> {
let Some(secrets) = secrets.filter(|_| self.registry.is_none()) else {
return Ok(self.write(path)?);
};
let mut keys = Vec::with_capacity(self.env.len());
for (name, value) in &self.env {
let key = env_key(id, name);
if value.is_empty() {
secrets.delete(&key);
} else if secrets.get(&key).as_deref() != Some(value.as_str()) {
secrets.set(&key, value)?;
}
keys.push(key);
}
forget_env(Some(secrets), id, &keys);
let names = self.env.keys().map(|name| (name.clone(), String::new())).collect();
Ok(Self { env: names, ..self.clone() }.write(path)?)
}
}
fn env_key(id: &str, name: &str) -> String {
format!("agent/{id}/env/{}", name.replace('%', "%25").replace('/', "%2F"))
}
/// Forgets agent `id`'s saved environment values, except `keep`.
fn forget_env(secrets: Option<&dyn EnvSecrets>, id: &str, keep: &[String]) {
let Some(secrets) = secrets else { return };
for key in secrets.keys(&format!("agent/{id}/env/")) {
if !keep.contains(&key) {
secrets.delete(&key);
}
}
}
/// Built-in agents, then the `*.toml` agents of `custom_dir`; a custom agent with a
/// built-in id replaces it.
pub(crate) fn load(custom_dir: Option<&Path>) -> Vec<ProviderSpec> {
pub(crate) fn load(custom_dir: Option<&Path>, secrets: Option<&dyn EnvSecrets>) -> Vec<ProviderSpec> {
let mut specs = builtins();
let mut files: Vec<PathBuf> = custom_dir
.and_then(|d| std::fs::read_dir(d).ok())
@@ -464,7 +547,7 @@ pub(crate) fn load(custom_dir: Option<&Path>) -> Vec<ProviderSpec> {
files.sort();
for path in files {
let Some(id) = path.file_stem().and_then(|s| s.to_str()).map(str::to_owned) else { continue };
let agent = match CustomAgent::read(&path) {
let agent = match CustomAgent::read_with(&path, &id, secrets) {
Ok(a) => a,
Err(e) => {
tracing::warn!("skipping agent {}: {e}", path.display());
@@ -534,7 +617,7 @@ mod tests {
#[test]
fn qoder_global_and_cn_editions_are_both_built_in() {
let info = |id: &str| describe(&load(None).into_iter().find(|s| s.id == id).unwrap());
let info = |id: &str| describe(&load(None, None).into_iter().find(|s| s.id == id).unwrap());
let (global, cn) = (info("qoder"), info("qodercn"));
assert_eq!((global.name.as_str(), global.command.as_str()), ("Qoder", "qodercli --acp"));
assert_eq!((cn.name.as_str(), cn.command.as_str()), ("Qoder CN", "qoderclicn --acp"));
@@ -544,7 +627,7 @@ mod tests {
#[test]
fn qodercn_is_available_without_a_file_and_preserves_existing_customizations() {
let dir = tempfile::tempdir().unwrap();
let qodercn = load(None).into_iter().find(|s| s.id == "qodercn").unwrap();
let qodercn = load(None, None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(qodercn.driver, Driver::Acp);
assert!(!is_unlisted(&qodercn));
let info = describe(&qodercn);
@@ -559,13 +642,13 @@ mod tests {
"command = \"/custom/qoderclicn\"\nargs = [\"--acp\"]\n[env]\nK = \"v\"\n",
)
.unwrap();
let specs = load(Some(dir.path()));
let specs = load(Some(dir.path()), None);
assert_eq!(specs.iter().filter(|s| s.id == "qodercn").count(), 1);
let custom = specs.iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(custom.command, "/custom/qoderclicn");
assert_eq!(custom.env, [("K".into(), "v".into())]);
set_builtin_command(dir.path(), "qodercn", None).unwrap();
let reset = load(Some(dir.path())).into_iter().find(|s| s.id == "qodercn").unwrap();
set_builtin_command(dir.path(), None, "qodercn", None).unwrap();
let reset = load(Some(dir.path()), None).into_iter().find(|s| s.id == "qodercn").unwrap();
assert_eq!(reset.command, "qoderclicn");
assert_eq!(reset.args, ["--acp"]);
assert_eq!(reset.env, custom.env);
@@ -680,7 +763,7 @@ mod tests {
#[test]
fn launch_metadata_distinguishes_defaults_overrides_and_read_only_agents() {
let dir = tempfile::tempdir().unwrap();
let list = || load(Some(dir.path())).iter().map(describe).collect::<Vec<_>>();
let list = || load(Some(dir.path()), None).iter().map(describe).collect::<Vec<_>>();
let defaults = list();
let codex = defaults.iter().find(|p| p.id == "codex").unwrap();
assert_eq!(codex.program, "codex");
@@ -700,14 +783,14 @@ mod tests {
p.id.ends_with("-acp").then_some(AgentCommandReadOnlyReason::PackageRunner)
);
}
set_builtin_command(dir.path(), "codex", Some("/path with spaces/codex")).unwrap();
set_builtin_command(dir.path(), None, "codex", Some("/path with spaces/codex")).unwrap();
let overridden = list().into_iter().find(|p| p.id == "codex").unwrap();
assert_eq!(overridden.program, "/path with spaces/codex");
assert_eq!(overridden.args, codex.args);
assert_eq!(overridden.default_program, codex.default_program);
assert!(overridden.command_overridden && overridden.command_editable);
assert_eq!(overridden.source, AgentSource::Builtin);
set_builtin_command(dir.path(), "codex", Some("codex")).unwrap();
set_builtin_command(dir.path(), None, "codex", Some("codex")).unwrap();
assert!(!list().iter().find(|p| p.id == "codex").unwrap().command_overridden);
std::fs::write(
dir.path().join("gemini.toml"),
@@ -732,7 +815,7 @@ mod tests {
write("mine.toml", "name = \"Mine\"\ncommand = \"/opt/mine\"\nargs = [\"acp\"]\n[env]\nK = \"v\"\n");
write("gemini.toml", "command = \"gemini\"\nargs = [\"--acp\"]\n");
write("broken.toml", "nope = 1\n");
let specs = load(Some(dir.path()));
let specs = load(Some(dir.path()), None);
let mine = specs.iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
@@ -748,34 +831,35 @@ mod tests {
fn set_builtin_env_writes_and_clears_the_environment() {
let dir = tempfile::tempdir().unwrap();
let env = BTreeMap::from([("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
set_builtin_env(dir.path(), "claude", env.clone()).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
set_builtin_env(dir.path(), None, "claude", env.clone()).unwrap();
let claude = load(Some(dir.path()), None).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!(claude.env, vec![("CLAUDE_CODE_ENTRYPOINT".to_owned(), "openidea".to_owned())]);
assert_eq!(describe(&claude).env, env);
// The command override and the environment are kept independently.
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
set_builtin_command(dir.path(), None, "claude", Some("/opt/claude")).unwrap();
set_builtin_command(dir.path(), None, "claude", None).unwrap();
assert!(dir.path().join("claude.toml").is_file());
set_builtin_env(dir.path(), "claude", BTreeMap::new()).unwrap();
set_builtin_env(dir.path(), None, "claude", BTreeMap::new()).unwrap();
assert!(!dir.path().join("claude.toml").exists());
set_builtin_env(dir.path(), "not-a-builtin", env).unwrap_err();
set_builtin_env(dir.path(), None, "not-a-builtin", env).unwrap_err();
}
#[test]
fn acp_adapters_of_native_agents_are_unlisted_until_customized() {
let dir = tempfile::tempdir().unwrap();
let unlisted = |dir: &Path| load(Some(dir)).into_iter().filter(is_unlisted).map(|s| s.id).collect::<Vec<_>>();
let unlisted =
|dir: &Path| load(Some(dir), None).into_iter().filter(is_unlisted).map(|s| s.id).collect::<Vec<_>>();
assert_eq!(unlisted(dir.path()), ["claude-acp", "codex-acp"]);
set_builtin_env(dir.path(), "codex-acp", BTreeMap::from([("A".to_owned(), "1".to_owned())])).unwrap();
set_builtin_env(dir.path(), None, "codex-acp", BTreeMap::from([("A".to_owned(), "1".to_owned())])).unwrap();
assert_eq!(unlisted(dir.path()), ["claude-acp"]);
}
#[test]
fn set_builtin_command_overrides_and_clears_a_builtin() {
let dir = tempfile::tempdir().unwrap();
set_builtin_command(dir.path(), "not-a-builtin", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), "claude", Some("/opt/claude")).unwrap();
let claude = load(Some(dir.path())).into_iter().find(|s| s.id == "claude").unwrap();
set_builtin_command(dir.path(), None, "not-a-builtin", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), None, "claude", Some("/opt/claude")).unwrap();
let claude = load(Some(dir.path()), None).into_iter().find(|s| s.id == "claude").unwrap();
assert_eq!((claude.command.as_str(), claude.driver), ("/opt/claude", Driver::Claude));
assert!(claude.args.is_empty());
// A registry install is neither overwritten nor removed by a command override.
@@ -790,14 +874,14 @@ mod tests {
}
.write(&path)
.unwrap();
set_builtin_command(dir.path(), "codex", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), "codex", None).unwrap_err();
set_builtin_command(dir.path(), None, "codex", Some("/x")).unwrap_err();
set_builtin_command(dir.path(), None, "codex", None).unwrap_err();
assert!(path.exists());
// Clearing (twice) returns the built-in to auto-detection.
set_builtin_command(dir.path(), "claude", None).unwrap();
set_builtin_command(dir.path(), "claude", None).unwrap();
set_builtin_command(dir.path(), None, "claude", None).unwrap();
set_builtin_command(dir.path(), None, "claude", None).unwrap();
assert!(!dir.path().join("claude.toml").exists());
assert_eq!(load(Some(dir.path())).iter().find(|s| s.id == "claude").unwrap().command, "claude");
assert_eq!(load(Some(dir.path()), None).iter().find(|s| s.id == "claude").unwrap().command, "claude");
}
#[test]
@@ -825,11 +909,11 @@ mod tests {
}
agent.write(&path).unwrap();
set_builtin_command(dir.path(), "claude", Some("/new/claude")).unwrap();
set_builtin_command(dir.path(), None, "claude", Some("/new/claude")).unwrap();
agent.command = "/new/claude".into();
assert_eq!(CustomAgent::read(&path).unwrap(), agent, "{field}");
set_builtin_command(dir.path(), "claude", None).unwrap();
set_builtin_command(dir.path(), None, "claude", None).unwrap();
agent.command = "claude".into();
assert_eq!(CustomAgent::read(&path).unwrap(), agent, "{field}");
}
@@ -845,11 +929,11 @@ mod tests {
)
.unwrap();
set_builtin_command(dir.path(), "codex", None).unwrap();
set_builtin_command(dir.path(), None, "codex", None).unwrap();
assert!(!path.exists());
assert_eq!(
load(Some(dir.path())).into_iter().find(|s| s.id == "codex").unwrap(),
load(Some(dir.path()), None).into_iter().find(|s| s.id == "codex").unwrap(),
builtin_spec("codex").unwrap()
);
}
@@ -870,7 +954,7 @@ mod tests {
let text = std::fs::read_to_string(&path).unwrap();
assert!(text.contains("[registry]") && !text.contains("driver"), "{text}");
assert_eq!(CustomAgent::read(&path).unwrap(), agent);
let spec = load(Some(path.parent().unwrap())).into_iter().find(|s| s.id == "amp-acp").unwrap();
let spec = load(Some(path.parent().unwrap()), None).into_iter().find(|s| s.id == "amp-acp").unwrap();
assert_eq!(spec.source, AgentSource::Registry { version: "0.10.0".into(), update: None });
assert!(toml::from_str::<CustomAgent>("command = \"x\"\n[registry]\nid = \"x\"\n").is_err());
}
@@ -888,50 +972,117 @@ mod tests {
#[test]
fn custom_agents_save_list_and_remove() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap();
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "mine").unwrap();
save_custom(dir.path(), None, &custom_input("mine")).unwrap();
let mine = load(Some(dir.path()), None).into_iter().find(|s| s.id == "mine").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("Mine", "/opt/mine"));
assert_eq!(mine.env, vec![("K".to_owned(), "v".to_owned())]);
assert_eq!(mine.source, AgentSource::Custom);
remove_custom(dir.path(), "mine").unwrap();
assert!(!load(Some(dir.path())).iter().any(|s| s.id == "mine"));
remove_custom(dir.path(), "mine").unwrap_err();
remove_custom(dir.path(), None, "mine").unwrap();
assert!(!load(Some(dir.path()), None).iter().any(|s| s.id == "mine"));
remove_custom(dir.path(), None, "mine").unwrap_err();
}
#[test]
fn custom_agents_refuse_builtin_registry_and_bad_ids() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("claude")).unwrap_err();
save_custom(dir.path(), &custom_input("../evil")).unwrap_err();
save_custom(dir.path(), &custom_input("")).unwrap_err();
save_custom(dir.path(), None, &custom_input("claude")).unwrap_err();
save_custom(dir.path(), None, &custom_input("../evil")).unwrap_err();
save_custom(dir.path(), None, &custom_input("")).unwrap_err();
let mut empty_command = custom_input("mine");
empty_command.command = " ".into();
save_custom(dir.path(), &empty_command).unwrap_err();
save_custom(dir.path(), &custom_input("mine")).unwrap();
save_custom(dir.path(), &custom_input("mine")).unwrap_err();
remove_custom(dir.path(), "claude").unwrap_err();
save_custom(dir.path(), None, &empty_command).unwrap_err();
save_custom(dir.path(), None, &custom_input("mine")).unwrap();
save_custom(dir.path(), None, &custom_input("mine")).unwrap_err();
remove_custom(dir.path(), None, "claude").unwrap_err();
// An id that isn't a plain file stem never reaches outside the agents directory.
let outside = dir.path().join("outside.toml");
std::fs::write(&outside, "command = \"/bin/sh\"\n").unwrap();
let agents = dir.path().join("agents");
std::fs::create_dir(&agents).unwrap();
remove_custom(&agents, "../outside").unwrap_err();
remove_custom(&agents, None, "../outside").unwrap_err();
assert!(outside.exists());
std::fs::write(
dir.path().join("amp-acp.toml"),
"command = \"/registry/amp\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n",
)
.unwrap();
save_custom(dir.path(), &custom_input("amp-acp")).unwrap_err();
remove_custom(dir.path(), "amp-acp").unwrap_err();
save_custom(dir.path(), None, &custom_input("amp-acp")).unwrap_err();
remove_custom(dir.path(), None, "amp-acp").unwrap_err();
}
#[derive(Default)]
struct MemorySecrets(parking_lot::Mutex<BTreeMap<String, String>>);
impl EnvSecrets for MemorySecrets {
fn get(&self, key: &str) -> Option<String> {
self.0.lock().get(key).cloned()
}
fn set(&self, key: &str, value: &str) -> IdeResult<()> {
self.0.lock().insert(key.to_owned(), value.to_owned());
Ok(())
}
fn delete(&self, key: &str) {
self.0.lock().remove(key);
}
fn keys(&self, prefix: &str) -> Vec<String> {
self.0.lock().keys().filter(|k| k.starts_with(prefix)).cloned().collect()
}
}
#[test]
fn environment_values_go_to_the_secrets_and_the_files_keep_the_names() {
let dir = tempfile::tempdir().unwrap();
let secrets = MemorySecrets::default();
let s = Some(&secrets as &dyn EnvSecrets);
let env = BTreeMap::from([("API_KEY".to_owned(), "v-91f".to_owned()), ("A/B".to_owned(), String::new())]);
set_builtin_env(dir.path(), s, "claude", env.clone()).unwrap();
save_custom(dir.path(), s, &CustomAgentInput { env: env.clone(), ..custom_input("mine") }).unwrap();
for id in ["claude", "mine"] {
let text = std::fs::read_to_string(dir.path().join(format!("{id}.toml"))).unwrap();
assert!(text.contains("API_KEY") && !text.contains("v-91f"), "{text}");
let spec = load(Some(dir.path()), s).into_iter().find(|p| p.id == id).unwrap();
assert_eq!(describe(&spec).env, env);
}
assert_eq!(secrets.keys("agent/"), ["agent/claude/env/API_KEY", "agent/mine/env/API_KEY"]);
set_builtin_command(dir.path(), s, "claude", Some("/opt/claude")).unwrap();
assert_eq!(secrets.get("agent/claude/env/API_KEY").as_deref(), Some("v-91f"));
set_builtin_env(dir.path(), s, "claude", BTreeMap::from([("OTHER".to_owned(), "x".to_owned())])).unwrap();
assert_eq!(secrets.keys("agent/claude/"), ["agent/claude/env/OTHER"]);
set_builtin_command(dir.path(), s, "claude", None).unwrap();
set_builtin_env(dir.path(), s, "claude", BTreeMap::new()).unwrap();
remove_custom(dir.path(), s, "mine").unwrap();
assert_eq!(secrets.keys("agent/"), Vec::<String>::new());
}
#[test]
fn values_written_in_agent_files_move_to_the_secrets_unless_installed_from_the_registry() {
let dir = tempfile::tempdir().unwrap();
let secrets = MemorySecrets::default();
let s = Some(&secrets as &dyn EnvSecrets);
std::fs::write(dir.path().join("mine.toml"), "command = \"/opt/mine\"\n[env]\nK = \"v-91f\"\nS = \"\"\n")
.unwrap();
secrets.set("agent/mine/env/S", "saved").unwrap();
let registry =
"command = \"/registry/amp\"\n[env]\nR = \"1\"\n[registry]\nid = \"amp-acp\"\nversion = \"1.0\"\n";
std::fs::write(dir.path().join("amp-acp.toml"), registry).unwrap();
let specs = load(Some(dir.path()), s);
assert_eq!(
specs.iter().find(|p| p.id == "mine").unwrap().env,
[("K".to_owned(), "v-91f".to_owned()), ("S".to_owned(), "saved".to_owned())]
);
assert!(!std::fs::read_to_string(dir.path().join("mine.toml")).unwrap().contains("v-91f"));
assert_eq!(secrets.keys("agent/"), ["agent/mine/env/K", "agent/mine/env/S"]);
assert_eq!(std::fs::read_to_string(dir.path().join("amp-acp.toml")).unwrap(), registry);
}
#[test]
fn acp_json_imports_servers_and_skips_collisions() {
let dir = tempfile::tempdir().unwrap();
save_custom(dir.path(), &custom_input("taken")).unwrap();
save_custom(dir.path(), None, &custom_input("taken")).unwrap();
let outcomes = import_acp(
dir.path(),
None,
r#"{"agent_servers": {
"My Agent!": {"command": "/opt/agent", "args": ["--acp"], "env": {"K": "v"}},
"taken": {"command": "/opt/taken"},
@@ -944,10 +1095,10 @@ mod tests {
let saved: Vec<&str> = outcomes.iter().filter(|o| o.saved).map(|o| o.id.as_str()).collect();
assert_eq!(saved, ["my-agent"]);
assert!(outcomes.iter().filter(|o| !o.saved).all(|o| !o.reason.is_empty()));
let mine = load(Some(dir.path())).into_iter().find(|s| s.id == "my-agent").unwrap();
let mine = load(Some(dir.path()), None).into_iter().find(|s| s.id == "my-agent").unwrap();
assert_eq!((mine.name.as_str(), mine.command.as_str()), ("My Agent!", "/opt/agent"));
assert_eq!(mine.source, AgentSource::Custom);
import_acp(dir.path(), "not json").unwrap_err();
assert!(import_acp(dir.path(), "{}").unwrap().is_empty());
import_acp(dir.path(), None, "not json").unwrap_err();
assert!(import_acp(dir.path(), None, "{}").unwrap().is_empty());
}
}
+4 -4
Переглянути файл
@@ -826,7 +826,7 @@ mod tests {
assert!(!shown.iter().any(|a| a.id == "cline"));
write_atomic(&p.cache_file, INDEX.as_bytes()).unwrap();
let mut providers: Vec<AgentProvider> = provider::load(Some(&p.agents_dir))
let mut providers: Vec<AgentProvider> = provider::load(Some(&p.agents_dir), None)
.into_iter()
.filter(|s| s.id == "amp-acp")
.map(|s| provider::describe(&s))
@@ -918,7 +918,7 @@ mod tests {
assert_eq!(agent.command, v1.to_string_lossy());
assert_eq!((agent.args.as_slice(), agent.env["A"].as_str()), (&["acp".to_owned()][..], "1"));
assert_eq!(agent.registry, Some(RegistryRef { id: "demo".into(), version: "1.0.0".into() }));
let spec = provider::load(Some(&p.agents_dir)).into_iter().find(|s| s.id == "demo").unwrap();
let spec = provider::load(Some(&p.agents_dir), None).into_iter().find(|s| s.id == "demo").unwrap();
assert_eq!(spec.source, AgentSource::Registry { version: "1.0.0".into(), update: None });
assert!(provider::installed(&spec.command));
@@ -991,7 +991,7 @@ mod tests {
assert_eq!((shown[0].installed.as_deref(), shown[0].update), (Some("1.0.104"), true));
write_atomic(&p.cache_file, index.as_bytes()).unwrap();
let spec = provider::load(Some(&p.agents_dir)).into_iter().find(|s| s.id == "copilot").unwrap();
let spec = provider::load(Some(&p.agents_dir), None).into_iter().find(|s| s.id == "copilot").unwrap();
assert_eq!(spec.name, "GitHub Copilot");
assert_eq!(spec.login_hint, "Run `copilot` in a terminal and sign in.");
assert_eq!(spec.source, AgentSource::Registry { version: "1.0.104".into(), update: None });
@@ -1004,7 +1004,7 @@ mod tests {
);
remove(&p, "github-copilot-cli").unwrap();
let fallback = provider::load(Some(&p.agents_dir)).into_iter().find(|s| s.id == "copilot").unwrap();
let fallback = provider::load(Some(&p.agents_dir), None).into_iter().find(|s| s.id == "copilot").unwrap();
assert_eq!(fallback.source, AgentSource::Builtin);
}
}
+1
Переглянути файл
@@ -24,6 +24,7 @@ fn paths(dir: &Path) -> AgentPaths {
models_file: dir.join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
}
}
+135 -22
Переглянути файл
@@ -13,29 +13,23 @@ pub enum PolicyDecision {
AskUser,
}
/// Extracts the shell command string from an ACP `session/request_permission` payload
/// when the request is for executing a command.
/// The shell command an ACP `session/request_permission` asks to run: only a tool call of kind
/// `execute`, and only its structured `rawInput` command. A title is free text the agent
/// writes, so it never names the command.
pub fn extract_command(params: &Value) -> Option<String> {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"))?;
if tool.get("kind").and_then(Value::as_str) != Some("execute") {
return None;
}
let raw_cmd = tool.pointer("/rawInput/command").or_else(|| tool.pointer("/rawInput/cmd"));
if let Some(cmd) = raw_cmd.and_then(command_from_value) {
return Some(cmd);
}
let kind = tool.get("kind").and_then(Value::as_str).unwrap_or_default();
let name = tool.get("name").and_then(Value::as_str).unwrap_or_default();
let title =
tool.get("title").and_then(Value::as_str).or_else(|| params.get("title").and_then(Value::as_str))?.trim();
if let Some(cmd) = title.strip_prefix("Run ").map(str::trim).filter(|s| !s.is_empty()) {
return Some(cmd.to_owned());
}
if (kind == "execute" || matches!(name, "Bash" | "PowerShell" | "Execute" | "Run" | "terminal"))
&& !title.is_empty()
&& title != "Terminal"
&& title != "Run a command"
{
return Some(title.to_owned());
}
None
raw_cmd.and_then(command_from_value)
}
/// Whether allowing the request grants more than running its command, such as network access
/// for it: the command policy may block such a request but never approves it.
pub fn widens_access(params: &Value) -> bool {
let tool = params.get("toolCall").or_else(|| params.pointer("/subject/toolCall"));
tool.and_then(|t| t.pointer("/rawInput/networkApprovalContext")).is_some_and(|c| !c.is_null())
}
fn command_from_value(val: &Value) -> Option<String> {
@@ -298,6 +292,41 @@ fn split_subcommands(command: &str) -> Vec<&str> {
out
}
/// Whether every shell an agent may run `command` in (a POSIX shell, PowerShell, cmd.exe) reads
/// it as the one command the policy sees. Outside double quotes no separator or redirection may
/// remain after splitting (cmd.exe takes single quotes literally); `\` may only stand in a
/// path (it escapes only in POSIX shells); no `^` may escape a quote (cmd.exe); no `%` (cmd.exe
/// expands it before parsing); no carriage return; no typographic quotes (PowerShell ends
/// strings at them).
fn reads_alike_in_every_shell(command: &str) -> bool {
let chars: Vec<char> = command.chars().collect();
let mut in_double = false;
for (i, &c) in chars.iter().enumerate() {
let next = chars.get(i + 1).copied();
match c {
'"' => in_double = !in_double,
'\\' if !next.is_none_or(|n| n.is_ascii_alphanumeric() || matches!(n, '\\' | '.' | '_' | '-')) => {
return false;
}
'^' if matches!(next, Some('"' | '\'' | '\n' | '\r')) => return false,
'%' | '\r' | '\u{2018}'..='\u{201f}' => return false,
';' | '&' | '|' | '<' | '>' | '\n' if !in_double => return false,
_ => {}
}
}
true
}
/// A command as blocked patterns are compared with it: lower case, single spaces, and the
/// program without its folder or `.exe` (`/bin/rm` and `RM.EXE` are `rm`).
fn normalized_for_blocking(command: &str) -> String {
let mut words = command.split_whitespace().map(str::to_lowercase);
let Some(program) = words.next() else { return String::new() };
let program = program.rsplit(['/', '\\']).next().unwrap_or(&program);
let program = program.strip_suffix(".exe").unwrap_or(program);
std::iter::once(program.to_owned()).chain(words).collect::<Vec<_>>().join(" ")
}
/// Evaluates `command` against `settings`:
/// 1. Blacklist takes precedence: if the full command or any sub-command matches a
/// `blocked_commands` entry, returns [`PolicyDecision::Blocked`].
@@ -312,12 +341,18 @@ pub fn evaluate_command(command: &str, settings: &AgentSecuritySettings) -> Poli
}
let subcommands = split_subcommands(cmd);
let normalized: Vec<String> =
std::iter::once(cmd).chain(subcommands.iter().copied()).map(normalized_for_blocking).collect();
for pat in &settings.blocked_commands {
let pat = pat.trim();
if pat.is_empty() {
continue;
}
if matches_command(pat, cmd) || subcommands.iter().any(|sub| matches_command(pat, sub)) {
let normalized_pat = normalized_for_blocking(pat);
if matches_command(pat, cmd)
|| subcommands.iter().any(|sub| matches_command(pat, sub))
|| normalized.iter().any(|c| matches_command(&normalized_pat, c))
{
return PolicyDecision::Blocked(pat.to_owned());
}
}
@@ -328,7 +363,7 @@ pub fn evaluate_command(command: &str, settings: &AgentSecuritySettings) -> Poli
let all_allowed = subcommands.iter().all(|sub| {
let s = sub.trim();
if s.is_empty() || has_unsafe_shell_syntax(s) {
if s.is_empty() || has_unsafe_shell_syntax(s) || !reads_alike_in_every_shell(s) {
return false;
}
settings.allowed_commands.iter().any(|entry| matches_allow_entry(entry, s))
@@ -455,6 +490,58 @@ mod tests {
assert_eq!(evaluate_command("git status && rm -rf /", &settings), PolicyDecision::Blocked("rm -rf *".into()));
}
/// The command may run in PowerShell or cmd.exe as well as a POSIX shell: what any of them
/// would split or expand differently from the policy's reading is never auto-approved.
#[test]
fn what_another_shell_reads_as_two_commands_is_not_auto_approved() {
let settings = AgentSecuritySettings {
allowed_commands: vec!["git status".into(), "git commit".into()],
blocked_commands: vec![],
};
for cmd in [
// `\` escapes only in POSIX shells.
"git status \\; calc.exe",
"git status \\& calc.exe",
"git status \\| calc.exe",
"git status \\\\\\; calc.exe",
"git status \"\\\" ; calc.exe ; \\\"\"",
// cmd.exe: single quotes don't quote, `^` escapes a quote, `%` expands first.
"git status '&' calc.exe",
"git status ';' calc.exe",
"git status '>' out.txt",
"git status ^\" & calc.exe ^\"",
"git status %COMSPEC%",
// PowerShell also closes a string at a typographic quote.
"git status \"a \u{201d} ; calc.exe ; \u{201c} b\"",
"git status \u{2018}x\u{2019}",
// A carriage return ends the command line too.
"git status\r calc.exe",
] {
assert_eq!(evaluate_command(cmd, &settings), PolicyDecision::AskUser, "{cmd:?}");
}
// Text every shell reads alike stays approvable.
for cmd in [
"git status -s",
"git status && git status",
"git commit -m \"fix <bug> & (issue); done\"",
"git status src\\main.rs C:\\work\\repo",
"git status HEAD^",
] {
assert_eq!(evaluate_command(cmd, &settings), PolicyDecision::Allowed, "{cmd:?}");
}
}
#[test]
fn blocked_patterns_ignore_case_spacing_and_the_programs_folder() {
let settings = AgentSecuritySettings::default();
for cmd in
["rm -rf /", "RM -RF /", "/bin/rm -rf /", "Rm -Rf /tmp/x", "C:\\Windows\\System32\\shutdown.exe /s"]
{
assert!(matches!(evaluate_command(cmd, &settings), PolicyDecision::Blocked(_)), "{cmd:?}");
}
assert_eq!(evaluate_command("git status", &settings), PolicyDecision::AskUser);
}
#[test]
fn extracts_command_from_v1_v2_and_codex_payloads() {
let claude = json!({
@@ -484,4 +571,30 @@ mod tests {
});
assert_eq!(extract_command(&write), None);
}
/// Only a tool call of kind `execute` runs a command, and its command is in `rawInput`:
/// a title says what the agent wants it to say.
#[test]
fn a_request_is_a_command_only_by_its_kind_and_input() {
let escalation = json!({ "sessionId": "t", "toolCall": {
"toolCallId": "i1", "title": "Run git status", "kind": "other", "status": "pending",
"rawInput": { "fileSystem": { "write": ["C:\\"] }, "network": { "enabled": true } } },
"options": [ { "optionId": "turn", "name": "Allow for this turn", "kind": "allow_once" },
{ "optionId": "decline", "name": "Decline", "kind": "reject_once" } ] });
assert_eq!(extract_command(&escalation), None);
let titled_only = json!({ "toolCall": { "title": "Run git status", "kind": "execute" } });
assert_eq!(extract_command(&titled_only), None);
let named_only = json!({ "toolCall": { "title": "git status", "name": "Bash" } });
assert_eq!(extract_command(&named_only), None);
let mislabelled = json!({ "toolCall": { "title": "Run git status", "kind": "edit",
"rawInput": { "command": "git status" } } });
assert_eq!(extract_command(&mislabelled), None);
let network = json!({ "toolCall": { "title": "Allow network access to example.com", "kind": "execute",
"rawInput": { "command": "curl https://example.com", "networkApprovalContext": { "host": "example.com" } } } });
assert_eq!(extract_command(&network).as_deref(), Some("curl https://example.com"));
assert!(widens_access(&network));
let plain = json!({ "toolCall": { "title": "Run git status", "kind": "execute",
"rawInput": { "command": "git status" } } });
assert!(!widens_access(&plain));
}
}
+94 -10
Переглянути файл
@@ -264,6 +264,23 @@ impl Handler for FakeAgent {
self.say(&session, &format!("late chose {chosen}"));
return Ok(json!({"stopReason": "end_turn"}));
}
if text == "escalate" {
// A request for wider access, titled with text the model wrote.
let peer = self.latest();
let answer: Value = tokio::runtime::Handle::current()
.block_on(peer.request(
"session/request_permission",
json!({"sessionId": session, "toolCall": {"toolCallId": "e1", "title": "Run cargo test",
"kind": "other", "rawInput": {"network": {"enabled": true}}},
"options": [{"optionId": "yes", "name": "Allow for this turn", "kind": "allow_once"},
{"optionId": "no", "name": "Decline", "kind": "reject_once"}]}),
None,
))
.unwrap();
let chosen = answer.pointer("/outcome/optionId").and_then(Value::as_str).unwrap_or("none");
self.say(&session, &format!("escalation chose {chosen}"));
return Ok(json!({"stopReason": "end_turn"}));
}
if text == "ask" {
let peer = self.latest();
let answer: Value = tokio::runtime::Handle::current()
@@ -344,7 +361,8 @@ impl Handler for FakeAgent {
let answer: Value = tokio::runtime::Handle::current()
.block_on(peer.request(
"session/request_permission",
json!({"sessionId": session, "toolCall": {"toolCallId": "t1", "title": "Run cargo test"},
json!({"sessionId": session, "toolCall": {"toolCallId": "t1", "title": "Run cargo test",
"kind": "execute", "rawInput": {"command": "cargo test"}},
"options": [{"optionId": "yes", "name": "Allow", "kind": "allow_once"},
{"optionId": "no", "name": "Reject", "kind": "reject_once"}]}),
None,
@@ -366,7 +384,8 @@ fn manager(dir: &Path, agent: Arc<FakeAgent>) -> AgentManager {
fn manager_at(dir: &Path, models_file: &Path, agent: Arc<FakeAgent>) -> AgentManager {
// These tests speak to FakeAgent, never an installed CLI, including after a provider reload.
let agents = dir.join("agents");
provider::set_builtin_command(&agents, "claude", Some(&dir.join("missing-claude").to_string_lossy())).unwrap();
provider::set_builtin_command(&agents, None, "claude", Some(&dir.join("missing-claude").to_string_lossy()))
.unwrap();
let connector: Connector = Arc::new(move |_spec, _cwd, client, _stderr| {
let (a, b) = duplex(1 << 20);
let (ar, aw) = tokio::io::split(a);
@@ -385,6 +404,7 @@ fn manager_at(dir: &Path, models_file: &Path, agent: Arc<FakeAgent>) -> AgentMan
// The project is the boundary of the skill walk: nothing above it (such as the
// user's profile, when temp lives under it) leaks into the test.
home_dir: Some(dir.to_path_buf()),
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
@@ -1410,6 +1430,7 @@ async fn real_claude_task_in_a_worktree() {
models_file: tmp.path().join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
);
@@ -2243,6 +2264,7 @@ async fn reload_providers_updates_registry_install_clears_stale_probes_and_notif
models_file: tmp.path().join("models.json"),
custom_agents_dir: Some(agents_dir.clone()),
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
@@ -2553,6 +2575,7 @@ async fn delete_running_agent(no_load: bool) {
models_file: root.join("models.json"),
custom_agents_dir: None,
home_dir: None,
env_secrets: None,
},
tokio::runtime::Handle::current(),
connector,
@@ -2853,14 +2876,14 @@ async fn rolling_back_restores_the_start_of_the_task_and_keeps_the_users_own_edi
}
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt", "other.txt"]);
mgr.rollback_changes(&task.id, Some(vec!["other.txt".into()])).await.unwrap();
mgr.rollback_changes(&task.id, Some(vec!["other.txt".into()]), AgentChangesView::SinceStart).await.unwrap();
assert_eq!(read("other.txt"), "base\n");
assert_eq!(changed(&mgr, &task.id).await, ["mine.txt", "new.txt"]);
// After a restart, from the saved copy.
mgr.shutdown();
let mgr = manager(&dir, Arc::new(FakeAgent::default()));
mgr.rollback_changes(&task.id, None).await.unwrap();
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(read("mine.txt"), "my edit\n", "the user's edit stays");
assert!(!dir.join("new.txt").exists());
assert!(changed(&mgr, &task.id).await.is_empty());
@@ -2973,7 +2996,7 @@ async fn work_uncommitted_by_a_reset_during_a_local_task_is_not_rolled_back() {
let base = mgr.base_text(&task.id, repo.dir.join("a.txt")).await.unwrap();
assert_eq!(base.map(|b| b.replace("\r\n", "\n")).as_deref(), Some("base\n"), "{eol:?}");
mgr.rollback_changes(&task.id, None).await.unwrap();
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("mine.txt"), "my work\n", "{eol:?}: the user's work survives the rollback");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
@@ -2996,7 +3019,7 @@ async fn files_a_mixed_reset_during_a_local_task_leaves_behind_are_not_rolled_ba
repo.git(&["reset", "-q", "HEAD~1"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None).await.unwrap();
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("mine.txt"), "my work\n", "{eol:?}");
assert_eq!(repo.read("mine-new.txt"), "my new file\n", "{eol:?}");
@@ -3023,7 +3046,7 @@ async fn a_branch_switched_to_during_a_local_task_is_left_alone() {
repo.git(&["checkout", "-q", "feature"]);
assert_eq!(changed(&mgr, &task.id).await, ["a.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None).await.unwrap();
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "base\n", "{eol:?}");
assert_eq!(repo.read("other.txt"), "feature\n", "{eol:?}");
assert_eq!(repo.read("only-feature.txt"), "feature\n", "{eol:?}");
@@ -3048,7 +3071,7 @@ async fn commits_made_during_a_local_task_are_kept_by_its_rollback() {
repo.write("a.txt", "agent\nagain\n");
assert_eq!(changed(&mgr, &task.id).await, ["a.txt", "b.txt"], "{eol:?}");
mgr.rollback_changes(&task.id, None).await.unwrap();
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "{eol:?}: back to the commit, not to before it");
assert!(!repo.dir.join("b.txt").exists(), "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
@@ -3069,7 +3092,7 @@ async fn rolling_back_a_rename_brings_the_old_file_back() {
let listed = changed(&mgr, &task.id).await;
assert_eq!(listed, ["CASE.java", "New.java"], "{eol:?}");
mgr.rollback_changes(&task.id, Some(listed)).await.unwrap();
mgr.rollback_changes(&task.id, Some(listed), AgentChangesView::SinceStart).await.unwrap();
assert_eq!(repo.read("Old.java"), "class Old {}\n", "{eol:?}");
assert!(!repo.dir.join("New.java").exists(), "{eol:?}");
assert_eq!(repo.read("Case.java"), "class Case {}\n", "{eol:?}");
@@ -3082,6 +3105,27 @@ async fn rolling_back_a_rename_brings_the_old_file_back() {
}
}
#[tokio::test(flavor = "multi_thread")]
async fn the_group_view_rolls_back_only_uncommitted_files() {
for eol in EOLS {
let repo = Checkout::new(eol);
repo.write("a.txt", "base\n");
repo.commit("init");
let (mgr, task) = repo.task().await;
repo.write("a.txt", "agent\n");
repo.commit("agent work");
repo.write("a.txt", "agent\nagain\n");
let grouped = mgr.grouped_changes(&task.id).await.unwrap();
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"], "{eol:?}");
assert!(grouped.commits.is_empty(), "{eol:?}: local tasks have no commits to group");
mgr.rollback_changes(&task.id, None, AgentChangesView::Group).await.unwrap();
assert_eq!(repo.read("a.txt"), "agent\n", "{eol:?}: back to the commit, not to the start");
assert!(mgr.grouped_changes(&task.id).await.unwrap().uncommitted.is_empty(), "{eol:?}");
}
}
#[tokio::test(flavor = "multi_thread")]
async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_changes() {
for eol in EOLS {
@@ -3092,7 +3136,7 @@ async fn a_local_task_in_a_repository_without_commits_lists_and_rolls_back_its_c
assert_eq!(changed(&mgr, &task.id).await, ["new.txt"], "{eol:?}");
assert_eq!(mgr.base_text(&task.id, repo.dir.join("new.txt")).await.unwrap(), None, "{eol:?}");
mgr.rollback_changes(&task.id, None).await.unwrap();
mgr.rollback_changes(&task.id, None, AgentChangesView::SinceStart).await.unwrap();
assert!(!repo.dir.join("new.txt").exists(), "{eol:?}");
assert_eq!(repo.read("mine.txt"), "mine\n", "{eol:?}");
assert!(changed(&mgr, &task.id).await.is_empty(), "{eol:?}");
@@ -3166,3 +3210,43 @@ async fn security_policy_auto_approves_whitelist_and_rejects_blacklist() {
"{blocked_entries:?}"
);
}
#[tokio::test(flavor = "multi_thread")]
async fn security_policy_never_auto_approves_a_request_that_only_reads_like_a_command() {
use ide_core::settings::AgentSecuritySettings;
let tmp = tempfile::tempdir().unwrap();
let agent = Arc::new(FakeAgent::default());
let mgr = manager(tmp.path(), agent);
mgr.set_security_hook(Arc::new(|| AgentSecuritySettings {
allowed_commands: vec!["cargo test".into()],
blocked_commands: vec![],
}));
let task = mgr
.create(NewAgentTask {
project_root: None,
prompt: text("escalate"),
provider: "claude".into(),
config: vec![],
isolation: Isolation::Local,
base: None,
})
.await
.unwrap();
wait_for(&mgr, &task.id, AgentStatus::NeedsInput).await;
let entries = mgr.entries(&task.id, None).unwrap();
let pending = entries
.iter()
.find_map(|e| match e {
AgentEntry::Permission { id, chosen: None, .. } => Some(*id),
_ => None,
})
.unwrap_or_else(|| panic!("the request was answered without the user: {entries:?}"));
mgr.respond(&task.id, pending, Some("no".into())).unwrap();
wait_for(&mgr, &task.id, AgentStatus::Ready).await;
let entries = mgr.entries(&task.id, None).unwrap();
assert!(
entries.iter().any(|e| matches!(e, AgentEntry::Agent { text } if text == "escalation chose no")),
"{entries:?}"
);
}
+1
Переглянути файл
@@ -438,6 +438,7 @@ mod tests {
t.push(AgentEntry::Permission {
id: 1,
title: "Run?".into(),
command: None,
options: vec![],
questions: vec![],
chosen: None,
+442 -27
Переглянути файл
@@ -1,8 +1,8 @@
//! Task worktrees and the changes made in them.
use crate::model::AgentChange;
use crate::model::{AgentChange, AgentChanges, AgentCommit};
use ide_core::{IdeError, IdeResult, IpcPath};
use ide_vcs::{ChangeKind, Repo};
use ide_vcs::{ChangeKind, LogQuery, Repo};
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::{Duration, UNIX_EPOCH};
@@ -35,6 +35,14 @@ pub(crate) fn head_of(path: &Path) -> IdeResult<(String, String)> {
Ok((info.head.unwrap_or_else(|| commit.clone()), commit))
}
/// `HEAD`'s commit in `cwd`, `None` outside a repository or before its first commit.
pub(crate) fn head_commit(cwd: &Path) -> IdeResult<Option<String>> {
match discover(cwd)? {
Some(repo) => Ok(repo.head()?.1),
None => Ok(None),
}
}
/// `git worktree add -b agent/<slug> <dir>/<slug> <base>`.
pub(crate) fn create(project_root: &Path, worktrees_dir: &Path, title: &str, base: Option<&str>) -> IdeResult<Created> {
let repo = repo_at(project_root)?;
@@ -111,7 +119,14 @@ pub(crate) fn changes(cwd: &Path, base: &str) -> IdeResult<Vec<AgentChange>> {
.into_iter()
.map(|f| {
let (added, removed) = stat(&f.path);
AgentChange { path: IpcPath(root.join(&f.path)), rel_path: f.path, kind: f.kind, added, removed }
AgentChange {
path: IpcPath(root.join(&f.path)),
rel_path: f.path,
kind: f.kind,
added,
removed,
old_path: f.old_path,
}
})
.collect();
let tracked: std::collections::HashSet<String> = out.iter().map(|c| c.rel_path.clone()).collect();
@@ -140,6 +155,7 @@ pub(crate) fn rename_sources(cwd: &Path, rev: &str) -> IdeResult<BTreeMap<String
kind: ChangeKind::Deleted,
added: 0,
removed: 0,
old_path: None,
};
Some((f.path, source))
})
@@ -148,7 +164,57 @@ pub(crate) fn rename_sources(cwd: &Path, rev: &str) -> IdeResult<BTreeMap<String
fn added(path: IpcPath, rel_path: String) -> AgentChange {
let added = std::fs::read_to_string(&*path).map_or(0, |t| t.lines().count() as u32);
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0 }
AgentChange { path, rel_path, kind: ChangeKind::Added, added, removed: 0, old_path: None }
}
/// The commits from `base` (exclusive) to `head` (inclusive) in `cwd`, newest first, each
/// with the files it changed against its first parent and their line counts.
pub(crate) fn commits(cwd: &Path, base: &str, head: &str) -> IdeResult<Vec<AgentCommit>> {
if base == head {
return Ok(Vec::new());
}
let Some(repo) = discover(cwd)? else { return Ok(Vec::new()) };
let root = repo.root().to_path_buf();
let query = LogQuery { range: Some(format!("{base}..{head}")), ..LogQuery::default() };
let mut out = Vec::new();
for commit in repo.log(&query)? {
let parent = commit.parents.first().cloned();
// A root commit has no parent to count lines against; its files still show.
let stats: std::collections::HashMap<String, (u32, u32)> = match &parent {
Some(parent) => repo
.diff_numstat(parent, &commit.id, &[])?
.into_iter()
.map(|s| (s.path, (s.added, s.removed)))
.collect(),
None => std::collections::HashMap::new(),
};
let mut files: Vec<AgentChange> = repo
.commit_files(&commit.id)?
.into_iter()
.map(|f| {
let (added, removed) = stats.get(&f.path).copied().unwrap_or((0, 0));
AgentChange {
path: IpcPath(root.join(&f.path)),
rel_path: f.path,
kind: f.kind,
added,
removed,
old_path: f.old_path,
}
})
.collect();
files.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
out.push(AgentCommit {
id: commit.id,
short_id: commit.short_id,
summary: commit.summary,
message: commit.message,
time_ms: commit.time_ms,
parent,
files,
});
}
Ok(out)
}
/// Whether two paths name one file (a case-insensitive disk resolving both spellings).
@@ -159,6 +225,15 @@ fn same_file(a: &Path, b: &Path) -> bool {
}
}
/// The bytes of `path` (`Some(None)`: no file there), or `None` when it is there but can't be read.
fn existing(path: &Path) -> Option<Option<Vec<u8>>> {
match std::fs::read(path) {
Ok(bytes) => Some(Some(bytes)),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Some(None),
Err(_) => None,
}
}
fn crlf_to_lf(bytes: &[u8]) -> Vec<u8> {
let mut out = Vec::with_capacity(bytes.len());
for (i, &b) in bytes.iter().enumerate() {
@@ -243,6 +318,34 @@ impl TaskBase {
.collect())
}
/// The task's uncommitted changes and, for a worktree task, the commits made since it
/// started (newest first). Both read the same `HEAD`, so a commit landing between the two
/// reads cannot put a file in both lists.
pub(crate) fn grouped_changes(&self, before: &BTreeMap<String, Stamp>) -> IdeResult<AgentChanges> {
let cwd = self.cwd();
let Some(repo) = discover(cwd)? else { return Ok(AgentChanges::default()) };
let head = repo.head()?.1;
let uncommitted = match &head {
Some(head) => without_untouched(changes(cwd, head)?, before),
None => {
// No commits yet: every file is new, as in `changes` for a local task.
let mut all: Vec<AgentChange> = repo
.status()?
.into_iter()
.filter(|s| s.path.is_file())
.map(|s| added(s.path, s.rel_path))
.collect();
all.sort_by(|a, b| a.rel_path.cmp(&b.rel_path));
without_untouched(all, before)
}
};
let commits = match (self, &head) {
(Self::Worktree { cwd, base }, Some(head)) => commits(cwd, base, head)?,
_ => Vec::new(),
};
Ok(AgentChanges { uncommitted, commits })
}
/// The revision the task's changed files go back to on rollback, and are compared with:
/// the commit the task started from, or the current `HEAD` once the user committed on top
/// of it, so those commits stay. `None` while the repository has no commits.
@@ -264,6 +367,38 @@ impl TaskBase {
}
}
/// One side of an Accept (the task's worktree or the checkout): where its files really are, and
/// git's own folders for it.
struct Confined {
root: PathBuf,
metadata: Vec<PathBuf>,
}
impl Confined {
fn new(root: &Path, repo: Option<&Repo>) -> IdeResult<Self> {
let metadata = match repo {
Some(repo) => repo.metadata_dirs()?.iter().map(|d| ide_core::real_path::resolve_real(d)).collect(),
None => Vec::new(),
};
Ok(Self { root: dunce::canonicalize(root)?, metadata })
}
/// Whether `rel` may be read or written here: it doesn't name git's metadata under any
/// spelling (case, trailing dots or spaces, stream suffixes, short names), stays in the tree
/// once links are followed, and isn't a link itself.
fn allows(&self, rel: &str) -> bool {
use ide_core::real_path::{names_git_dir, reaches_git_dir, resolve_real};
let path = self.root.join(rel);
if names_git_dir(Path::new(rel)) || path.symlink_metadata().is_ok_and(|m| m.file_type().is_symlink()) {
return false;
}
let real = resolve_real(&path);
real.starts_with(&self.root)
&& !reaches_git_dir(&self.root, &real)
&& !self.metadata.iter().any(|dir| real.starts_with(dir))
}
}
/// Copies the task's `changes` (against `base`) from the worktree `cwd` into the
/// local checkout `root`, uncommitted: the user reviews and commits them with the
/// VCS tools. Files the checkout changed itself are left alone and reported, so
@@ -276,28 +411,40 @@ pub(crate) fn accept(
) -> IdeResult<crate::AgentAcceptOutcome> {
let repo = repo_at(cwd)?;
// Change paths are repository-relative, and the task's project may be a folder inside it.
let local_root = discover(root)?.map_or_else(|| root.to_path_buf(), |r| r.root().to_path_buf());
let local = discover(root)?;
let local_root = local.as_ref().map_or_else(|| root.to_path_buf(), |r| r.root().to_path_buf());
// Paths come from what the agent left in its worktree: neither side may lead into git's own
// files or out through a link.
let task_side = Confined::new(repo.root(), Some(&repo))?;
let checkout_side = Confined::new(&local_root, local.as_ref())?;
let copyable = |rel: &str| task_side.allows(rel) && checkout_side.allows(rel);
let renamed_from: BTreeMap<String, String> =
repo.diff_with_worktree(base)?.into_iter().filter_map(|f| Some((f.path, f.old_path?))).collect();
// Brings one path of the checkout to the task's version, unless the checkout changed it itself.
let apply = |rel: &str| -> IdeResult<bool> {
let local = local_root.join(rel);
// The task's version of one path of the checkout (`Some(None)`: absent there), or `None` when
// the checkout changed it itself.
let target = |rel: &str| -> IdeResult<Option<Option<Vec<u8>>>> {
let in_task = repo.root().join(rel);
let base_bytes = repo.file_bytes_at(&in_task, Some(base))?;
let current = std::fs::read(&in_task).ok();
let local_bytes = std::fs::read(&local).ok();
// A file there that can't be read is the checkout's own: never written over.
let Some(local_bytes) = existing(&local_root.join(rel)) else { return Ok(None) };
if local_bytes == current {
return Ok(true);
return Ok(Some(current));
}
let base_bytes = repo.file_bytes_at(&in_task, Some(base))?;
// The checkout may hold the committed LF text with CRLF line ends (`core.autocrlf`).
let unchanged = match (&local_bytes, &base_bytes) {
(Some(now), Some(at_base)) => now == at_base || crlf_to_lf(now) == *at_base,
(now, at_base) => now == at_base,
};
if !unchanged {
return Ok(false);
Ok(unchanged.then_some(current))
};
// Puts `bytes` (`None`: nothing) at `rel` of the checkout.
let write = |rel: &str, bytes: Option<Vec<u8>>| -> IdeResult<()> {
let local = local_root.join(rel);
if std::fs::read(&local).ok() == bytes {
return Ok(());
}
match current {
match bytes {
Some(bytes) => {
if let Some(parent) = local.parent() {
std::fs::create_dir_all(parent)?;
@@ -309,6 +456,12 @@ pub(crate) fn accept(
_ => {}
},
}
Ok(())
};
// Brings one path of the checkout to the task's version, unless the checkout changed it itself.
let apply = |rel: &str| -> IdeResult<bool> {
let Some(bytes) = target(rel)? else { return Ok(false) };
write(rel, bytes)?;
Ok(true)
};
let mut accepted = Vec::with_capacity(changes.len());
@@ -317,25 +470,59 @@ pub(crate) fn accept(
path: path.to_owned(),
reason: "the checkout has changes of its own to it; accept it by hand".to_owned(),
};
let refuse = |path: &str| crate::AgentAcceptSkipped {
path: path.to_owned(),
reason: "it leads into git's own files or through a link, so it is not copied".to_owned(),
};
for change in changes {
// A case-only rename on a case-insensitive disk: both names reach the same file, so
// rename it in place before bringing it to the task's version.
if let Some(old) = renamed_from.get(&change.rel_path)
&& old != &change.rel_path
&& old.to_lowercase() == change.rel_path.to_lowercase()
&& local_root.join(old).exists()
&& same_file(&local_root.join(old), &local_root.join(&change.rel_path))
{
std::fs::rename(local_root.join(old), local_root.join(&change.rel_path))?;
let old = renamed_from.get(&change.rel_path);
if !copyable(&change.rel_path) {
skipped.push(refuse(&change.rel_path));
continue;
}
if let Some(old) = old.filter(|old| !copyable(old)) {
skipped.push(refuse(old));
continue;
}
// A case-only rename (`a.txt` → `A.txt`). Where the disk resolves both
// spellings to one file it is renamed in place and `apply` then refreshes
// the content. On a case-sensitive disk the two names are different files:
// both are checked before either changes — the old one must be the file the
// checkout had, the new one absent or already the task's — then the task's
// version lands at the new name before the old one goes.
let case_only =
old.is_some_and(|o| o != &change.rel_path && o.to_lowercase() == change.rel_path.to_lowercase());
if case_only {
let old = old.expect("case_only implies a rename source");
let (old_path, new_path) = (local_root.join(old), local_root.join(&change.rel_path));
if same_file(&old_path, &new_path) {
std::fs::rename(&old_path, &new_path)?;
} else {
let Some(old_target) = target(old)? else {
skipped.push(skip(old));
continue;
};
let task_bytes = std::fs::read(repo.root().join(&change.rel_path)).ok();
let at_new = existing(&new_path);
if at_new.as_ref().is_none_or(|at_new| at_new.is_some() && *at_new != task_bytes) {
skipped.push(skip(&change.rel_path));
continue;
}
write(&change.rel_path, task_bytes)?;
write(old, old_target)?;
accepted.push(change.rel_path.clone());
continue;
}
}
if !apply(&change.rel_path)? {
skipped.push(skip(&change.rel_path));
continue;
}
accepted.push(change.rel_path.clone());
// A rename also takes the file away from its old path (unless only its case changed).
if let Some(old) = renamed_from.get(&change.rel_path)
&& old.to_lowercase() != change.rel_path.to_lowercase()
// A rename also takes the file away from its old path (a case-only rename
// is handled above).
if !case_only
&& let Some(old) = old
&& !apply(old)?
{
skipped.push(skip(old));
@@ -550,7 +737,18 @@ mod tests {
fn git(dir: &Path, args: &[&str]) {
let ok = std::process::Command::new("git")
.args(["-c", "user.name=t", "-c", "user.email=t@t", "-c", "commit.gpgsign=false"])
.args([
"-c",
"user.name=t",
"-c",
"user.email=t@t",
"-c",
"commit.gpgsign=false",
// Hermetic line endings: a developer or CI runner with
// `core.autocrlf=true` would otherwise check the fixtures out as CRLF.
"-c",
"core.autocrlf=false",
])
.args(args)
.current_dir(dir)
.status()
@@ -598,6 +796,88 @@ mod tests {
assert!(!created.path.exists());
}
#[test]
fn commits_since_the_base_are_listed_with_their_files() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let created = create(&repo, &tmp.path().join("wt"), "Edit A", None).unwrap();
assert!(commits(&created.path, &created.base_commit, &created.base_commit).unwrap().is_empty());
std::fs::write(created.path.join("a.txt"), "one\ntwo\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["commit", "-q", "-m", "change a"]);
std::fs::write(created.path.join("b.txt"), "b\n").unwrap();
git(&created.path, &["add", "."]);
git(&created.path, &["commit", "-q", "-m", "add b"]);
let head = head_of(&created.path).unwrap().1;
let log = commits(&created.path, &created.base_commit, &head).unwrap();
assert_eq!(log.iter().map(|c| c.summary.as_str()).collect::<Vec<_>>(), ["add b", "change a"]);
assert_eq!(log[0].files.iter().map(|f| f.rel_path.as_str()).collect::<Vec<_>>(), ["b.txt"]);
assert_eq!((log[0].files[0].added, log[0].files[0].removed), (1, 0));
assert_eq!(log[1].files.iter().map(|f| f.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
assert_eq!((log[1].files[0].added, log[1].files[0].removed), (1, 0));
assert!(log.iter().all(|c| c.parent.is_some() && !c.message.is_empty()));
// Nothing is uncommitted after the commits; editing a committed file brings it back.
let at = TaskBase::Worktree { cwd: created.path.clone(), base: created.base_commit.clone() };
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert!(grouped.uncommitted.is_empty());
assert_eq!(grouped.commits.len(), 2);
std::fs::write(created.path.join("a.txt"), "one\ntwo\nthree\n").unwrap();
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
assert_eq!(grouped.commits.len(), 2);
}
#[test]
fn a_rename_in_a_commit_keeps_its_old_path() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\nthree\nfour\nfive\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
let base = head_of(&repo).unwrap().1;
std::fs::rename(repo.join("a.txt"), repo.join("b.txt")).unwrap();
std::fs::write(repo.join("b.txt"), "one\ntwo\nthree\nfour\nFIVE\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "rename a"]);
let head = head_of(&repo).unwrap().1;
let log = commits(&repo, &base, &head).unwrap();
assert_eq!(log.len(), 1);
assert_eq!(log[0].files.len(), 1);
assert_eq!(log[0].files[0].rel_path, "b.txt");
assert_eq!(log[0].files[0].old_path.as_deref(), Some("a.txt"));
assert_eq!(log[0].files[0].kind, ChangeKind::Renamed);
}
#[test]
fn a_local_task_has_no_commits() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
std::fs::write(repo.join("a.txt"), "one\n").unwrap();
git(&repo, &["add", "."]);
git(&repo, &["commit", "-q", "-m", "init"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
let at = TaskBase::Local { cwd: repo.clone(), start: None };
let grouped = at.grouped_changes(&BTreeMap::new()).unwrap();
assert!(grouped.commits.is_empty());
assert_eq!(grouped.uncommitted.iter().map(|c| c.rel_path.as_str()).collect::<Vec<_>>(), ["a.txt"]);
}
#[test]
fn a_broken_worktree_registration_still_leaves_a_deletable_folder() {
let tmp = tempfile::tempdir().unwrap();
@@ -634,6 +914,9 @@ mod tests {
let repo = tmp.path().join("repo");
std::fs::create_dir(&repo).unwrap();
git(&repo, &["init", "-q", "-b", "main"]);
// Hermetic line endings: a machine with `core.autocrlf=true` would check
// the fixtures and the worktree out as CRLF.
git(&repo, &["config", "core.autocrlf", "false"]);
std::fs::write(repo.join("a.txt"), "one\ntwo\n").unwrap();
std::fs::write(repo.join("gone.txt"), "bye\n").unwrap();
git(&repo, &["add", "."]);
@@ -697,6 +980,138 @@ mod tests {
assert_eq!(names, ["A.txt"]);
}
#[test]
fn a_case_only_rename_keeps_a_file_the_checkout_has_at_the_new_name() {
let (_tmp, repo, created) = accept_repo();
std::fs::write(repo.join("A.txt"), "mine\n").unwrap();
if std::fs::read_to_string(repo.join("a.txt")).unwrap() == "mine\n" {
eprintln!("this disk doesn't tell a.txt from A.txt; nothing to check");
return;
}
git(&created.path, &["mv", "a.txt", "A.txt"]);
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert_eq!(outcome.skipped.iter().map(|s| s.path.as_str()).collect::<Vec<_>>(), ["A.txt"]);
assert_eq!(std::fs::read_to_string(repo.join("A.txt")).unwrap(), "mine\n");
assert_eq!(std::fs::read_to_string(repo.join("a.txt")).unwrap(), "one\ntwo\n");
}
#[cfg(unix)]
#[test]
fn accept_leaves_a_checkout_file_it_cannot_read() {
use std::os::unix::fs::PermissionsExt;
let (_tmp, repo, created) = accept_repo();
std::fs::write(created.path.join("new.txt"), "task\n").unwrap();
std::fs::write(repo.join("new.txt"), "mine\n").unwrap();
std::fs::set_permissions(repo.join("new.txt"), std::fs::Permissions::from_mode(0o200)).unwrap();
if std::fs::read(repo.join("new.txt")).is_ok() {
eprintln!("running as a user that reads any file; nothing to check");
return;
}
let changes = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
std::fs::set_permissions(repo.join("new.txt"), std::fs::Permissions::from_mode(0o644)).unwrap();
assert_eq!(outcome.skipped.iter().map(|s| s.path.as_str()).collect::<Vec<_>>(), ["new.txt"]);
assert_eq!(std::fs::read_to_string(repo.join("new.txt")).unwrap(), "mine\n");
}
#[cfg(windows)]
#[test]
fn accept_does_not_write_into_the_checkouts_git_folder_through_its_short_name() {
let (_tmp, repo, created) = accept_repo();
if !repo.join("GIT~1").join("HEAD").is_file() {
eprintln!("8.3 short names are disabled on this volume; nothing to check");
return;
}
// Written in the worktree under the name the checkout's `.git` folder also has.
let wt = &created.path;
std::fs::rename(wt.join(".git"), wt.join("gitfile")).unwrap();
std::fs::create_dir_all(wt.join("GIT~1").join("hooks")).unwrap();
std::fs::write(wt.join("GIT~1").join("hooks").join("pre-commit"), "#!/bin/sh\nexit 0\n").unwrap();
std::fs::rename(wt.join("gitfile"), wt.join(".git")).unwrap();
std::fs::write(wt.join("a.txt"), "one\nTWO\n").unwrap();
let changes = listed(wt, &created.base_commit);
assert!(changes.iter().any(|c| c.rel_path == "GIT~1/hooks/pre-commit"), "control: git lists it");
let outcome = accept(&repo, wt, &created.base_commit, &changes).unwrap();
assert!(!repo.join(".git").join("hooks").join("pre-commit").exists(), "{outcome:?}");
assert_eq!(outcome.accepted, ["a.txt"]);
assert_eq!(outcome.skipped.iter().map(|s| s.path.as_str()).collect::<Vec<_>>(), ["GIT~1/hooks/pre-commit"]);
}
#[test]
fn accept_refuses_paths_that_lead_into_git_metadata_under_any_spelling() {
let (tmp, repo, _created) = accept_repo();
let checkout = Confined::new(&repo, Repo::discover(&repo).unwrap().as_ref()).unwrap();
for rel in [
".git/hooks/pre-commit",
".GIT/hooks/pre-commit",
".git./hooks/pre-commit",
".git /hooks/pre-commit",
".git::$INDEX_ALLOCATION/hooks/pre-commit",
"sub/.git/config",
"../outside.txt",
] {
assert!(!checkout.allows(rel), "{rel}");
}
assert!(checkout.allows("src/new.txt"));
assert!(checkout.allows(".github/workflows/ci.yml"));
// git's folder under another name inside the tree (`--separate-git-dir`).
let separate = tmp.path().join("separate");
std::fs::create_dir(&separate).unwrap();
git(&separate, &["init", "-q", "--separate-git-dir", "meta", "-b", "main"]);
let side = Confined::new(&separate, Repo::discover(&separate).unwrap().as_ref()).unwrap();
assert!(!side.allows("meta/hooks/pre-commit"));
assert!(side.allows("metadata.txt"));
}
/// A link the task left in its worktree, or a linked folder in the checkout: neither is followed.
#[test]
fn accept_does_not_follow_links_on_either_side() {
let (tmp, repo, created) = accept_repo();
let outside = tmp.path().join("outside");
std::fs::create_dir(&outside).unwrap();
std::fs::write(outside.join("secret.txt"), "secret\n").unwrap();
// A folder link in the checkout (a junction on Windows, which needs no privilege).
#[cfg(unix)]
std::os::unix::fs::symlink(&outside, repo.join("linked")).unwrap();
#[cfg(windows)]
assert!(
std::process::Command::new("cmd")
.args(["/c", "mklink", "/J"])
.arg(repo.join("linked"))
.arg(&outside)
.output()
.unwrap()
.status
.success()
);
std::fs::create_dir(created.path.join("linked")).unwrap();
std::fs::write(created.path.join("linked/new.txt"), "x\n").unwrap();
std::fs::write(created.path.join("a.txt"), "one\nTWO\n").unwrap();
// A file link in the worktree, where the platform lets this user create one.
#[cfg(unix)]
let file_link = std::os::unix::fs::symlink(outside.join("secret.txt"), created.path.join("copied.txt"));
#[cfg(windows)]
let file_link = std::os::windows::fs::symlink_file(outside.join("secret.txt"), created.path.join("copied.txt"));
if let Err(e) = &file_link {
eprintln!("no file link ({e}); checking the folder link only");
}
let changes: Vec<_> = listed(&created.path, &created.base_commit);
let outcome = accept(&repo, &created.path, &created.base_commit, &changes).unwrap();
assert!(!outside.join("new.txt").exists(), "{outcome:?}");
assert!(!repo.join("copied.txt").exists(), "{outcome:?}");
assert!(outcome.accepted.contains(&"a.txt".to_owned()));
let refused: Vec<_> = outcome.skipped.iter().map(|s| s.path.as_str()).collect();
if file_link.is_ok() {
assert_eq!(refused, ["copied.txt", "linked/new.txt"]);
} else {
assert_eq!(refused, ["linked/new.txt"]);
}
}
#[test]
fn accept_copies_new_modified_and_deleted_files_into_the_checkout() {
let (_tmp, repo, created) = accept_repo();
+2 -1
Переглянути файл
@@ -12,7 +12,8 @@ mod up_to_date;
pub use fingerprint::{LaunchFingerprint, launch_fingerprint, menu_fingerprint};
pub use launch_resources::process_launch_resources;
pub use single::{
SingleFileTarget, compile_file_plan, find_javac, javac_step, single_file_classpath, single_file_target,
SingleFileTarget, compile_file_plan, find_javac, javac_step, module_classpath, single_file_classpath,
single_file_target,
};
pub use up_to_date::{menu_up_to_date, record_menu_build};
+210 -3
Переглянути файл
@@ -4,12 +4,21 @@
//! walks the whole reactor; compiling one file only needs that file's module
//! outputs plus its dependency classpath, which is much faster to assemble.
use super::*;
use serde::{Deserialize, Serialize};
use std::fs;
use std::process::Command as ProcCommand;
/// Output file of the `dependency:build-classpath` probe, relative to each module.
const CP_FILE: &str = "target/ide-compile-cp.txt";
/// The resolved dependency classpath for one module and scope, tied to the same
/// inputs as its launch build so POM, JDK, settings and profile changes invalidate it.
#[derive(Debug, Deserialize, Serialize)]
struct CachedClasspath {
fingerprint: LaunchFingerprint,
classpath: String,
}
/// A `.java` source and the module build that compiles it.
pub struct SingleFileTarget {
/// Absolute module directory.
@@ -64,6 +73,28 @@ pub fn single_file_target(root: &Path, model: &BuildModel, file: &Path) -> IdeRe
Ok(SingleFileTarget { module, test, out_dir, top: top.clone(), pl, java_version: project.java_version })
}
/// The Maven reactor target for a launch module whose source file is not known yet.
fn module_target(root: &Path, model: &BuildModel, module: &Path, test: bool) -> IdeResult<SingleFileTarget> {
let Some(m) = &model.maven else {
return Err(IdeError::not_found("no linked Maven project found for module"));
};
let module = if module.is_absolute() { module.to_path_buf() } else { root.join(module) };
let project = m
.projects
.iter()
.find(|p| root.join(&p.dir) == module)
.ok_or_else(|| IdeError::not_found("no linked Maven project found for module"))?;
if project.packaging == "pom" {
return Err(IdeError::not_found("the launch module is a Maven aggregator without sources"));
}
let top = maven_reactor(m, &project.dir)
.ok_or_else(|| IdeError::not_found("no linked Maven reactor found for module"))?;
let top_dir = subdir(root, &top.dir);
let pl = module.strip_prefix(&top_dir).map(|r| r.to_string_lossy().replace('\\', "/")).unwrap_or_default();
let out_dir = module.join("target").join(if test { "test-classes" } else { "classes" });
Ok(SingleFileTarget { module, test, out_dir, top: top.clone(), pl, java_version: project.java_version })
}
/// `javac` for the module: its `JAVA_HOME`, else `PATH`.
pub fn find_javac(env: &BTreeMap<String, String>) -> String {
let home = env.get("JAVA_HOME").map(Path::new);
@@ -78,6 +109,88 @@ pub fn find_javac(env: &BTreeMap<String, String>) -> String {
/// `mvn dependency:build-classpath` (which resolves without compiling anything,
/// so `-am` stays cheap). Entries are platform-separated (`;` on Windows).
pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFileTarget) -> IdeResult<String> {
classpath(root, model, target, "test", false)
}
/// Runtime (or test) dependency classpath for a launch module.
///
/// The Java language server runs on its own JDK, so its Maven model can select a
/// different JDK profile and dependency classifier than the project launch. Use the
/// same Maven process that builds the module instead.
pub fn module_classpath(
root: &Path,
model: &BuildModel,
module: &Path,
test: bool,
local_repo: Option<PathBuf>,
cache: &Path,
cancel: &ide_core::CancellationToken,
) -> IdeResult<String> {
let target = module_target(root, model, module, test)?;
let fingerprint = launch_fingerprint(root, model, &target.module, test, local_repo, cancel)?;
let cache_path = classpath_cache_path(root, cache, &target.module, test);
if let (Some(fingerprint), Some(path)) = (&fingerprint, &cache_path)
&& let Ok(bytes) = fs::read(path)
&& let Ok(cached) = serde_json::from_slice::<CachedClasspath>(&bytes)
&& cached.fingerprint == *fingerprint
&& classpath_entries_exist(root, &cached.classpath)
{
return Ok(cached.classpath);
}
let classpath = classpath(root, model, &target, if test { "test" } else { "runtime" }, true)?;
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
if let (Some(fingerprint), Some(path)) = (fingerprint, cache_path) {
let cached = CachedClasspath { fingerprint, classpath: classpath.clone() };
if let Err(error) = cache_classpath(&path, &cached) {
tracing::debug!(path = %path.display(), %error, "Cannot cache the Java launch classpath");
}
}
Ok(classpath)
}
/// One cache file per module and scope. Mirroring the module path keeps nested
/// modules and their different dependency scopes from overwriting each other.
fn classpath_cache_path(root: &Path, cache: &Path, module: &Path, test: bool) -> Option<PathBuf> {
let relative = module.strip_prefix(root).ok()?;
let mut path = cache.join("java-launch/classpaths");
path.push(if relative.as_os_str().is_empty() { Path::new("_root") } else { relative });
path.push(if test { "test.json" } else { "runtime.json" });
Some(path)
}
/// Best-effort cache persistence. A failed write only costs one later Maven
/// probe, so the launch still succeeds even when the cache directory is unusable.
fn cache_classpath(path: &Path, cached: &CachedClasspath) -> IdeResult<()> {
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
fs::write(path, serde_json::to_vec(cached).map_err(|e| IdeError::internal(e.to_string()))?)?;
Ok(())
}
/// A cache entry stays valid while every file exists. Maven also emits reactor
/// output directories for modules with no classes, which may legitimately never be
/// created; the launch snapshot already supplies the module's classes in that case.
fn classpath_entries_exist(root: &Path, classpath: &str) -> bool {
// Temp directories on macOS are commonly reached through `/var`, while Maven
// reports the canonical `/private/var` path. Compare against both spellings.
let canonical_root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
std::env::split_paths(classpath).all(|entry| {
entry.exists()
|| ((entry.starts_with(root) || entry.starts_with(&canonical_root))
&& (entry.ends_with(Path::new("target/classes")) || entry.ends_with(Path::new("target/test-classes"))))
})
}
fn classpath(
root: &Path,
model: &BuildModel,
target: &SingleFileTarget,
scope: &str,
compile_first: bool,
) -> IdeResult<String> {
let mut entries: Vec<String> = Vec::new();
// The module's own classes first: the file almost always uses classes next to it.
let own: &[&str] = if target.test { &["target/test-classes", "target/classes"] } else { &["target/classes"] };
@@ -110,7 +223,14 @@ pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFil
args.extend(["-pl", target.pl.as_str(), "-am"]);
}
let output_arg = format!("-Dmdep.outputFile={probe}");
args.extend(["dependency:build-classpath", output_arg.as_str(), "-Dmdep.includeScope=test", "-q"]);
let scope_arg = format!("-Dmdep.includeScope={scope}");
// A launch build may only have reached `compile`. Running `compile` in this
// same Maven session lets the dependency plugin resolve Reactor siblings from
// their target/classes without paying for package-phase plugins.
if compile_first {
args.push("compile");
}
args.extend(["dependency:build-classpath", output_arg.as_str(), scope_arg.as_str(), "-q"]);
let top_dir = subdir(root, &target.top.dir);
let status = ProcCommand::new(&target.top.command)
.args(&args)
@@ -119,8 +239,10 @@ pub fn single_file_classpath(root: &Path, model: &BuildModel, target: &SingleFil
.output()
.map_err(|e| IdeError::internal(format!("cannot resolve the compile classpath: {e}")))?;
if !status.status.success() {
let stdout = String::from_utf8_lossy(&status.stdout);
let stderr = String::from_utf8_lossy(&status.stderr);
let tail: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
// Maven reports reactor/dependency failures on stdout, not only stderr.
let tail: Vec<&str> = stdout.lines().chain(stderr.lines()).filter(|l| !l.trim().is_empty()).collect();
let tail = tail.iter().rev().take(5).rev().cloned().collect::<Vec<_>>().join("\n");
return Err(IdeError::internal(format!("cannot resolve the compile classpath:\n{tail}")));
}
@@ -305,7 +427,7 @@ mod tests {
let script = root.join("mvnw");
fs::write(
&script,
"#!/bin/sh\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/a.jar:/repo/b.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
"#!/bin/sh\nprintf '%s\\n' \"$@\" > args.txt\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/a.jar:/repo/b.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
@@ -327,5 +449,90 @@ mod tests {
assert!(cp.contains("/repo/a.jar"), "{cp}");
assert!(cp.contains("/repo/b.jar"), "{cp}");
assert!(cp.split(separator).count() >= 3, "{cp}");
let args = fs::read_to_string(root.join("args.txt")).unwrap();
assert!(!args.lines().any(|a| a == "compile" || a == "package"), "{args}");
}
/// A launch uses runtime dependencies, while tests additionally see test-scope ones.
#[cfg(unix)]
#[test]
fn launch_module_classpath_uses_the_matching_dependency_scope() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
fixture(root);
let script = root.join("mvnw");
fs::write(
&script,
"#!/bin/sh\nprintf '%s\\n' \"$@\" > args.txt\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target\"\nprintf '/repo/runtime.jar:/repo/test.jar' > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
let model = crate::load(root, &root.join("cache"), None);
let cache = root.join("cache");
let runtime =
module_classpath(root, &model, &root.join("app"), false, None, &cache, &CancellationToken::new()).unwrap();
assert!(runtime.contains("/repo/runtime.jar"), "{runtime}");
let args = fs::read_to_string(root.join("args.txt")).unwrap();
assert!(args.lines().any(|a| a == "compile"), "{args}");
assert!(!args.lines().any(|a| a == "package"), "{args}");
assert!(args.lines().any(|a| a == "-Dmdep.includeScope=runtime"), "{args}");
let test =
module_classpath(root, &model, &root.join("app"), true, None, &cache, &CancellationToken::new()).unwrap();
assert!(test.contains("/repo/test.jar"), "{test}");
let args = fs::read_to_string(root.join("args.txt")).unwrap();
assert!(args.lines().any(|a| a == "-Dmdep.includeScope=test"), "{args}");
}
/// The launch classpath follows the Maven runner JDK and is reused until that
/// build input changes; the probe does not run packaging plugins.
#[cfg(unix)]
#[test]
fn launch_classpath_cache_follows_the_runner_jdk() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
fixture(root);
let script = root.join("mvnw");
// `dependency:build-classpath` reports reactor outputs as absolute paths. Keep
// the fake probe faithful so the cache's existence check sees the output.
fs::write(
&script,
"#!/bin/sh\nprintf x >> calls.txt\npl='.'\nwhile [ $# -gt 0 ]; do\nif [ \"$1\" = \"-pl\" ]; then pl=\"$2\"; shift 2; else shift; fi\ndone\nmkdir -p \"$pl/target/classes\" \"$JAVA_HOME/classpath\"\n: > \"$JAVA_HOME/classpath/jdk.jar\"\nprintf '%s/classpath/jdk.jar:%s/%s/target/classes:%s/empty/target/classes' \"$JAVA_HOME\" \"$(pwd)\" \"$pl\" \"$(pwd)\" > \"$pl/target/ide-compile-cp.txt\"\n",
)
.unwrap();
fs::set_permissions(&script, fs::Permissions::from_mode(0o755)).unwrap();
let mut model = crate::load(root, &root.join("cache"), None);
for (name, version) in [("jdk21", "21"), ("jdk8", "8")] {
let home = root.join(name);
fs::create_dir_all(home.join("bin")).unwrap();
fs::write(home.join("release"), format!("JAVA_VERSION=\"{version}\"\n")).unwrap();
fs::write(home.join("bin/java"), "").unwrap();
}
let cache = root.join("classpath-cache");
let cancel = CancellationToken::new();
let jdk21 = root.join("jdk21");
for project in &mut model.maven.as_mut().unwrap().projects {
project.env.insert("JAVA_HOME".into(), jdk21.to_string_lossy().into_owned());
}
let first = module_classpath(root, &model, &root.join("app"), false, None, &cache, &cancel).unwrap();
assert!(first.contains("jdk21/classpath/jdk.jar"), "{first}");
assert!(!first.contains("jdk8/classpath/jdk.jar"), "{first}");
let second = module_classpath(root, &model, &root.join("app"), false, None, &cache, &cancel).unwrap();
assert_eq!(second, first);
assert_eq!(fs::read_to_string(root.join("calls.txt")).unwrap(), "x");
let jdk8 = root.join("jdk8");
for project in &mut model.maven.as_mut().unwrap().projects {
project.env.insert("JAVA_HOME".into(), jdk8.to_string_lossy().into_owned());
}
let switched = module_classpath(root, &model, &root.join("app"), false, None, &cache, &cancel).unwrap();
assert!(switched.contains("jdk8/classpath/jdk.jar"), "{switched}");
assert!(!switched.contains("jdk21/classpath/jdk.jar"), "{switched}");
assert_eq!(fs::read_to_string(root.join("calls.txt")).unwrap(), "xx");
}
}
+1 -1
Переглянути файл
@@ -204,7 +204,7 @@ pub fn parse_settings(text: &str) -> Settings {
}
/// `a/b/../c` → `a/c`, without touching the filesystem.
fn normalize(p: &Path) -> PathBuf {
pub(crate) fn normalize(p: &Path) -> PathBuf {
let mut out = PathBuf::new();
for c in p.components() {
match c {
+8
Переглянути файл
@@ -122,6 +122,14 @@ pub fn gradle_external_roots(
gradle::external_roots(root, dirs.iter().map(String::as_str))
}
/// Directories of Maven POMs linked from outside `root`, outermost only.
/// Reads just those explicit paths (no POM parse and no directory walk).
pub fn maven_external_roots(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let dirs = maven::external_pom_dirs(root, linked);
let owned: Vec<String> = dirs.iter().map(|dir| dir.to_string_lossy().into_owned()).collect();
gradle::external_roots(root, owned.iter().map(String::as_str))
}
/// Source, resource and build folders of the last Gradle sync, absolute.
pub fn gradle_source_roots(
root: &Path,
+547 -55
Переглянути файл
@@ -16,6 +16,7 @@ use std::io::Read;
use std::path::{Path, PathBuf};
use std::sync::{Mutex, OnceLock};
pub mod completion;
pub mod sources;
/// Lets the IDE's own Maven runs (Reload, Build) use what the local repository already
@@ -106,7 +107,9 @@ pub struct MavenProject {
pub profiles: Vec<String>,
/// Java feature version the project compiles for (`maven.compiler.release`, `java.version`…).
pub java_version: Option<u32>,
/// Maven launcher: the nearest `mvnw` up to the project root (absolute), else `mvn`.
/// Maven launcher: the nearest `mvnw` (absolute), else `mvn`. Inside the opened
/// project the search stops at its root; outside it, the search stops before
/// that root so a linked project keeps its own wrapper.
pub command: String,
/// Options before the goals (`-s`, `-Dmaven.repo.local=`); filled in by the shell from settings.
pub options: Vec<String>,
@@ -297,7 +300,7 @@ fn read_plugin(e: &Element) -> Option<RawPlugin> {
})
}
fn read_properties(e: Option<&Element>) -> Vec<(String, String)> {
pub(crate) fn read_properties(e: Option<&Element>) -> Vec<(String, String)> {
e.map(|p| p.children.iter().map(|c| (c.name.clone(), c.text.trim().to_owned())).collect()).unwrap_or_default()
}
@@ -709,18 +712,27 @@ pub(crate) fn launch_inputs(
// Profile activation may depend on the presence of an otherwise unrelated file.
let activation = parent == "file" && matches!(el.name.as_str(), "exists" | "missing");
if input || activation {
// Other properties can be overridden by -D, profiles or settings.
// Without Maven's effective roots, conservatively disable reuse for them.
let value = el
.text
.trim()
.replace("${project.basedir}", &dir.to_string_lossy())
.replace("${basedir}", &dir.to_string_lossy());
if value.contains("${") {
return None;
}
if !value.is_empty() {
out.push(dir.join(value));
// A source root under Maven's build directory is generated by the build itself.
// Skip it without resolving the property: `<build><directory>` may be custom, and
// hashing its rewritten files would make every launch look changed.
if !(input && el.text.contains("${project.build.directory}")) {
// Other properties can be overridden by -D, profiles or settings. Without
// Maven's effective model, leave them unresolved to conservatively disable reuse.
let value = el
.text
.trim()
.replace("${project.basedir}", &dir.to_string_lossy())
.replace("${basedir}", &dir.to_string_lossy());
if value.contains("${") {
return None;
}
if !value.is_empty() {
let path = dir.join(value);
// Conventional generated sources under target are also build outputs.
if !(input && path.starts_with(dir.join("target"))) {
out.push(path);
}
}
}
}
for child in &el.children {
@@ -1032,10 +1044,6 @@ pub(crate) fn plugin(
const MAX_PROJECTS: usize = 1000;
fn rel_path(root: &Path, p: &Path) -> String {
p.strip_prefix(root).unwrap_or(p).to_string_lossy().replace('\\', "/")
}
/// The POMs that are not modules of another one: the root `pom.xml`, else the
/// shallowest `pom.xml` of each branch (a folder holding several builds). An
/// aggregate POM directly inside `src/` counts too (issue #1001: `src/pom.xml`
@@ -1049,18 +1057,29 @@ pub fn is_maven_project(root: &Path) -> bool {
!top_level_poms(root).is_empty()
}
/// The `mvnw` nearest to `dir`, looking up to `root`.
/// The `mvnw` nearest to `dir`. Inside `root` the walk stops at `root`. Outside it,
/// the walk stops before entering `root` or any ancestor of `root`: a sibling project
/// keeps its own wrapper, and the search does not continue to the filesystem root.
fn wrapper(root: &Path, dir: &Path) -> String {
let name = if cfg!(windows) { "mvnw.cmd" } else { "mvnw" };
let mut cur = Some(dir);
let root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let dir = dunce::canonicalize(dir).unwrap_or_else(|_| dir.to_path_buf());
let external = !dir.starts_with(&root);
let mut cur = Some(dir.as_path());
while let Some(d) = cur {
if d.join(name).is_file() {
return d.join(name).to_string_lossy().into_owned();
}
if d == root {
if !external && d == root {
break;
}
cur = d.parent();
let Some(parent) = d.parent() else { break };
// `parent` inside the opened project, or an ancestor of it (the common parent,
// then the filesystem root). Stop before inspecting those.
if external && (parent.starts_with(&root) || root.starts_with(parent)) {
break;
}
cur = Some(parent);
}
if cfg!(windows) { "mvn.cmd".into() } else { "mvn".into() }
}
@@ -1104,24 +1123,118 @@ pub(crate) fn load_with(root: &Path, local_repo: Option<PathBuf>, profiles: &[St
load_with_linked(root, local_repo, profiles, &[])
}
/// The POMs of `linked` (project-relative, from Settings) that exist inside `root`.
/// The `pom.xml` files `linked` names. Each entry is used as written (project-relative,
/// `../sibling/pom.xml`, or absolute) and canonicalized; a path that is not an existing
/// `pom.xml` is skipped. Nothing here lists a directory or walks parents looking for builds.
fn linked_poms(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out: Vec<PathBuf> = Vec::new();
for pom in linked {
let pom = normalized_pom(pom);
if pom.is_empty() || pom.starts_with("../") || pom == ".." {
continue;
}
let path = root.join(&pom);
let inside = dunce::canonicalize(&path).is_ok_and(|p| p.starts_with(&base));
if path.is_file() && inside && !out.contains(&path) {
let Some(path) = dunce::canonicalize(base.join(pom.trim())).ok() else { continue };
if is_pom_file(&path) && !out.contains(&path) {
out.push(path);
}
}
out
}
fn is_pom_file(path: &Path) -> bool {
path.is_file()
&& path
.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| if cfg!(windows) { name.eq_ignore_ascii_case("pom.xml") } else { name == "pom.xml" })
}
/// Identity of `path` (both canonical) within `root`: project-relative inside it,
/// otherwise the absolute path with forward slashes.
fn project_rel(root: &Path, path: &Path) -> String {
path.strip_prefix(root).unwrap_or(path).to_string_lossy().replace('\\', "/")
}
/// Canonical `pom.xml` paths named by `linked`, keyed by the spelling to store and to
/// show so unlink removes the same entry settings hold. The first spelling wins.
fn linked_identities(root: &Path, linked: &[String]) -> HashMap<PathBuf, String> {
let mut out = HashMap::new();
for entry in linked {
let entry = entry.trim();
if entry.is_empty() {
continue;
}
let Some(path) = dunce::canonicalize(root.join(entry)).ok() else { continue };
if is_pom_file(&path) {
out.entry(path).or_insert_with(|| entry.replace('\\', "/"));
}
}
out
}
/// Stable, deduplicated form of `poms` for `languages.java.maven.linkedProjects`.
/// An existing `pom.xml` inside `root` is project-relative; one outside it is the
/// canonical absolute path. Spellings of the same file (`../B/pom.xml`, an absolute
/// path, a symlink) collapse to one entry. A path that exists and is not a `pom.xml`
/// is dropped. A `pom.xml` that cannot be resolved yet is kept, so a link is not
/// discarded while its volume is offline. No directory is scanned.
pub fn normalize_linked_poms(root: &Path, poms: &[String]) -> Vec<String> {
let Some(root) = dunce::canonicalize(root).ok() else {
let mut out = Vec::new();
for pom in poms {
let pom = normalized_pom(pom);
if !pom.is_empty() && pom_name_ok(&pom) && !out.contains(&pom) {
out.push(pom);
}
}
return out;
};
let mut out = Vec::new();
let mut seen = HashSet::new();
for pom in poms {
let pom = pom.trim();
if pom.is_empty() {
continue;
}
let key = match dunce::canonicalize(root.join(pom)) {
Ok(path) if is_pom_file(&path) => project_rel(&root, &path),
Ok(_) => continue,
Err(_) => {
let fallback = normalized_pom(pom);
let joined = root.join(pom);
if !pom_name_ok(&fallback) || (joined.exists() && !is_pom_file(&joined)) {
continue;
}
fallback
}
};
if key.is_empty() || !seen.insert(key.clone()) {
continue;
}
out.push(key);
}
out
}
fn pom_name_ok(path: &str) -> bool {
let name = path.rsplit('/').next().unwrap_or(path);
if cfg!(windows) { name.eq_ignore_ascii_case("pom.xml") } else { name == "pom.xml" }
}
/// Directories of linked `pom.xml` files that sit outside `root` (the opened project
/// stays the root). Callers keep the outermost directory only. Modules a POM declares
/// outside its own directory are not included: a linked file cannot widen the indexed
/// tree to an arbitrary path.
pub fn external_pom_dirs(root: &Path, linked: &[String]) -> Vec<PathBuf> {
let base = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let mut out = Vec::new();
for pom in linked_poms(root, linked) {
let Some(dir) = pom.parent() else { continue };
if dir.starts_with(&base) || out.iter().any(|have| have == dir) {
continue;
}
out.push(dir.to_path_buf());
}
out
}
/// Whether `root` holds a Maven build or the settings link one inside it.
pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -> bool {
is_maven_project(root) || !linked_poms(root, &settings.linked_projects).is_empty()
@@ -1130,7 +1243,9 @@ pub fn has_projects(root: &Path, settings: &ide_core::settings::MavenSettings) -
/// The model with the chosen `profiles` (`-P` entries; `!id` deactivates: their modules,
/// dependencies and properties are part of the model, those turned off are not) and the
/// `linked` POMs (Add Maven Projects): builds that no reactor lists, shown as independent
/// top-level projects. A linked POM that turns out to be a module of the root reactor stays that module.
/// top-level projects, including a `pom.xml` outside the opened root. A linked POM that
/// turns out to be a module of the root reactor stays that module. The same canonical file
/// is loaded once, so a cycle or a second spelling does not duplicate it or edit either POM.
pub fn load_with_linked(
root: &Path,
local_repo: Option<PathBuf>,
@@ -1139,7 +1254,10 @@ pub fn load_with_linked(
) -> MavenModel {
let mut resolver = Resolver::new(local_repo, profiles);
let mut model = MavenModel::default();
let mut seen: HashSet<PathBuf> = HashSet::new();
// Canonical POM → its project, for a module listed again by a later aggregator.
let mut seen: HashMap<PathBuf, Option<usize>> = HashMap::new();
let canonical_root = dunce::canonicalize(root).unwrap_or_else(|_| root.to_path_buf());
let linked_ids = linked_identities(&canonical_root, linked);
let mut tops = top_level_poms(root);
for pom in linked_poms(root, linked) {
if !tops.contains(&pom) {
@@ -1147,17 +1265,31 @@ pub fn load_with_linked(
}
}
// (pom, aggregated); the root reactor and its modules come before linked POMs.
let mut stack: Vec<(PathBuf, bool)> = tops.into_iter().rev().map(|p| (p, false)).collect();
while let Some((path, aggregated)) = stack.pop() {
// (pom, listed by): a module carries the top-level project its chain started from and
// whether the aggregator listing it lies inside the opened root.
let mut stack: Vec<(PathBuf, Option<(usize, bool)>)> = tops.into_iter().rev().map(|p| (p, None)).collect();
while let Some((path, listed_by)) = stack.pop() {
let aggregated = listed_by.is_some();
if model.projects.len() >= MAX_PROJECTS {
break;
}
let canonical = dunce::canonicalize(&path).unwrap_or_else(|_| path.clone());
if !seen.insert(canonical) {
if let Some(known) = seen.get(&canonical) {
// Found as a top-level POM first, then listed by an aggregator elsewhere
// (`<module>../</module>` of `parent/pom.xml`): it is that aggregator's module.
// Not when the chain leads back to where it started (a cycle), and not for an
// aggregator outside the opened root: a linked sibling does not take it over.
if let (Some(known), Some((origin, true))) = (*known, listed_by)
&& known != origin
{
model.projects[known].aggregated = true;
}
continue;
}
seen.insert(canonical.clone(), None);
let Some(pom) = resolver.pom(&path) else { continue };
let dir = path.parent().unwrap_or(root).to_path_buf();
let dir_canon = dunce::canonicalize(&dir).unwrap_or_else(|_| dir.clone());
let ctx = resolver.context(&path, 0);
let props = resolver.properties_for(&pom, &ctx, &dir);
let i = |s: &str| interpolate(s, &props);
@@ -1168,7 +1300,7 @@ pub fn load_with_linked(
}
let mut modules: Vec<PathBuf> = Vec::new();
for m in module_names {
let mut mp = dir.join(i(&m));
let mut mp = crate::gradle::normalize(&dir.join(i(&m)));
if mp.is_dir() || mp.extension().is_none_or(|e| e != "xml") {
mp.push("pom.xml");
}
@@ -1232,28 +1364,39 @@ pub fn load_with_linked(
}
}
let artifact_id = i(&pom.artifact_id);
let module_ids: Vec<String> = modules
.iter()
.map(|m| {
let canonical = dunce::canonicalize(m).unwrap_or_else(|_| m.clone());
linked_ids.get(&canonical).cloned().unwrap_or_else(|| project_rel(&canonical_root, &canonical))
})
.collect();
model.projects.push(MavenProject {
project_root: None,
pom: rel_path(root, &path),
dir: rel_path(root, &dir),
pom: linked_ids.get(&canonical).cloned().unwrap_or_else(|| project_rel(&canonical_root, &canonical)),
dir: project_rel(&canonical_root, &dir_canon),
group_id: i(props.get("project.groupId").map(String::as_str).unwrap_or_default()),
version: i(props.get("project.version").map(String::as_str).unwrap_or_default()),
display_name: pom.name.as_deref().map(i).filter(|n| !n.is_empty()).unwrap_or_else(|| artifact_id.clone()),
artifact_id,
packaging,
parent: pom.parent.as_ref().map(|p| format!("{}:{}:{}", p.group_id, p.artifact_id, i(&p.version))),
modules: modules.iter().map(|m| rel_path(root, m)).collect(),
modules: module_ids,
aggregated,
plugins,
dependencies,
profiles: ctx.profiles.iter().map(|(id, _)| id.clone()).collect(),
java_version: jv,
command: wrapper(root, &dir),
command: wrapper(&canonical_root, &dir_canon),
options: Vec::new(),
env: BTreeMap::new(),
});
let index = model.projects.len() - 1;
seen.insert(canonical, Some(index));
let origin = listed_by.map_or(index, |(origin, _)| origin);
let inside = dir_canon.starts_with(&canonical_root);
for m in modules.into_iter().rev() {
stack.push((m, true));
stack.push((m, Some((origin, inside))));
}
}
model
@@ -1341,17 +1484,27 @@ impl UnlinkedProjects {
fn normalized_pom(pom: &str) -> String {
let pom = pom.replace('\\', "/");
let mut parts = Vec::new();
let unc = pom.starts_with("//");
let absolute = pom.starts_with('/');
let mut parts: Vec<&str> = Vec::new();
for part in pom.split('/') {
match part {
"" | "." => {}
".." if parts.last().is_some_and(|p| *p != "..") => {
// Keep a drive letter (`C:`) and a leading `..` that has nothing to climb.
".." if parts.last().is_some_and(|p| *p != ".." && !p.ends_with(':')) => {
parts.pop();
}
part => parts.push(part),
}
}
parts.join("/")
let joined = parts.join("/");
if unc {
format!("//{joined}")
} else if absolute {
format!("/{joined}")
} else {
joined
}
}
/// Every POM of the model (for change detection).
@@ -1536,6 +1689,15 @@ pub enum PomGotoKind {
ManagedDependency,
/// A `<pluginManagement>` entry providing the plugin's version.
ManagedPlugin,
/// The POM of the artifact itself: a project module's, else the local repository's.
ArtifactPom,
}
/// A Maven project of the workspace, for going to a dependency on it.
pub struct ModulePom {
pub group_id: String,
pub artifact_id: String,
pub pom: PathBuf,
}
/// Byte offset of `pos` (0-based line / UTF-16 column), clamped to `text`.
@@ -1586,7 +1748,7 @@ fn property_ref_at(text: &str, at: usize) -> Option<(&str, usize, usize)> {
}
/// The elements containing `at`, from the root down to the innermost one.
fn element_path(root: &Element, at: usize) -> Vec<&Element> {
pub(crate) fn element_path(root: &Element, at: usize) -> Vec<&Element> {
let mut path = vec![root];
loop {
let Some(next) = path.last().unwrap().children.iter().find(|c| c.start <= at && at < c.end) else {
@@ -1615,6 +1777,20 @@ fn coordinates(e: &Element, props: &HashMap<String, String>) -> Option<(String,
/// the local repository when it is not in the project). `None` when there is no
/// resolvable declaration here.
pub fn goto_declaration(text: &str, path: &Path, pos: Pos, local_repo: Option<PathBuf>) -> Option<PomGoto> {
goto_declaration_in(text, path, pos, local_repo, Vec::new)
}
/// [`goto_declaration`], where a `<parent>` and a dependency or plugin whose version no
/// management entry provides (or a management entry itself) go to the artifact's own
/// POM: the project module among `modules` with its coordinates, else the local
/// repository's POM of its version.
pub fn goto_declaration_in(
text: &str,
path: &Path,
pos: Pos,
local_repo: Option<PathBuf>,
modules: impl FnOnce() -> Vec<ModulePom>,
) -> Option<PomGoto> {
let root = xml::parse(text).filter(|e| e.name == "project")?;
let at = offset_at(text, pos);
if let Some((name, _, _)) = property_ref_at(text, at)
@@ -1625,15 +1801,56 @@ pub fn goto_declaration(text: &str, path: &Path, pos: Pos, local_repo: Option<Pa
// A `<dependency>` / `<plugin>` in a declaration position: its version may come
// from a management entry of this POM or an ancestor's.
let elements = element_path(&root, at);
let e = elements.iter().rev().find(|e| e.name == "dependency" || e.name == "plugin")?;
if elements.iter().any(|a| a.name == "dependencyManagement" || a.name == "pluginManagement") {
return None; // Management entries are declarations; there is nothing to go to.
if elements.get(1).is_some_and(|e| e.name == "parent") {
let chain = parent_chain(path, text, local_repo);
let (pom, parent_text) = chain.get(1)?;
let label = elements[1].text_of("artifactId").unwrap_or_default();
return Some(pom_goto(pom.clone(), parent_text, label));
}
let kind = match e.name.as_str() {
"dependency" => PomGotoKind::ManagedDependency,
_ => PomGotoKind::ManagedPlugin,
};
managed_target(text, path, e, kind, local_repo)
let e = elements.iter().rev().find(|e| e.name == "dependency" || e.name == "plugin")?;
// A management entry declares the version: it goes to the artifact (a BOM import, say).
if !elements.iter().any(|a| a.name == "dependencyManagement" || a.name == "pluginManagement") {
let kind = match e.name.as_str() {
"dependency" => PomGotoKind::ManagedDependency,
_ => PomGotoKind::ManagedPlugin,
};
if let Some(found) = managed_target(text, path, e, kind, local_repo.clone()) {
return Some(found);
}
}
artifact_target(text, path, e, local_repo, modules)
}
/// The artifact's own POM for a dependency or plugin element `e`.
fn artifact_target(
text: &str,
path: &Path,
e: &Element,
local_repo: Option<PathBuf>,
modules: impl FnOnce() -> Vec<ModulePom>,
) -> Option<PomGoto> {
let mut resolver = Resolver::new(local_repo.clone(), &[]);
let chain = parent_chain(path, text, local_repo);
let props = properties_at(&mut resolver, &chain, 0);
let (group, artifact) = coordinates(e, &props)?;
let label = format!("{group}:{artifact}");
if let Some(module) = modules().into_iter().find(|m| m.group_id == group && m.artifact_id == artifact) {
let module_text = xml::read_text(&module.pom)?;
return Some(pom_goto(module.pom, &module_text, label));
}
let version = interpolate(&e.text_of("version")?, &props);
let pom = resolver.repo_pom(&group, &artifact, &version)?;
let pom_text = xml::read_text(&pom)?;
Some(pom_goto(pom, &pom_text, label))
}
/// The POM at `path` as a target, at its `<artifactId>`.
fn pom_goto(path: PathBuf, text: &str, label: String) -> PomGoto {
let range = xml::parse(text)
.filter(|e| e.name == "project")
.and_then(|root| root.child("artifactId").map(|a| range_of(text, a)))
.unwrap_or_default();
PomGoto { path, range, kind: PomGotoKind::ArtifactPom, label }
}
/// The parent POMs of the POM in `text` at `path`, child-first (its `<parent>` refs,
@@ -1932,6 +2149,52 @@ mod tests {
assert_eq!(known.goals, ["compile", "help", "testCompile"], "the common plugins keep their table");
}
#[test]
fn launch_inputs_skip_generated_sources_under_the_project_build_directory() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
write(
root,
"pom.xml",
"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version>\
<packaging>pom</packaging><modules><module>app</module></modules></project>",
);
write(
root,
"app/pom.xml",
r#"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1</version></parent>
<artifactId>app</artifactId><build><plugins><plugin>
<groupId>org.codehaus.mojo</groupId><artifactId>build-helper-maven-plugin</artifactId>
<executions><execution><configuration><sources>
<source>${project.build.directory}/generated-sources/java</source>
</sources></configuration></execution></executions>
</plugin></plugins></build></project>"#,
);
let model = load_with_repo(root, None);
let projects: Vec<&MavenProject> = model.projects.iter().collect();
let inputs = launch_inputs(root, &projects, None).expect("portable launch inputs");
assert!(!inputs.contains(&root.join("app/target/generated-sources/java")), "{inputs:?}");
assert!(inputs.contains(&root.join("app/src")), "conventional sources stay fingerprinted");
}
#[test]
fn launch_inputs_disable_reuse_for_file_activation_on_the_build_directory() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
write(
root,
"pom.xml",
r#"<project><groupId>demo</groupId><artifactId>app</artifactId><version>1</version>
<build><directory>${project.basedir}/custom-build</directory></build>
<profiles><profile><id>generated</id><activation><file>
<exists>${project.build.directory}/enable-profile</exists>
</file></activation></profile></profiles></project>"#,
);
let model = load_with_repo(root, None);
let projects: Vec<&MavenProject> = model.projects.iter().collect();
assert!(launch_inputs(root, &projects, None).is_none());
}
fn write(dir: &Path, rel: &str, text: &str) {
let p = dir.join(rel);
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
@@ -2147,6 +2410,169 @@ mod tests {
assert_eq!(top(&load_linked(deep.path(), &settings)), [("a/b/c/tool".into(), false)]);
}
/// A sibling Maven project is linked beside the opened root. Its reactor, including a
/// module that points back at the opened project, is read once. Neither POM is rewritten,
/// and only the linked project's directory is an extra content root — not the common
/// parent and not a module that lives outside that directory.
#[test]
fn external_sibling_keeps_its_directory_and_does_not_edit_poms() {
let tmp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(tmp.path()).unwrap();
let a = base.join("A");
let b = base.join("B");
let c = base.join("C");
std::fs::create_dir_all(a.join("core")).unwrap();
std::fs::create_dir_all(b.join("mod")).unwrap();
std::fs::create_dir_all(&c).unwrap();
let a_pom = "<project><groupId>demo</groupId><artifactId>a</artifactId><version>1</version><packaging>pom</packaging><modules><module>core</module></modules></project>";
let b_pom = "<project><groupId>demo</groupId><artifactId>b</artifactId><version>1</version><packaging>pom</packaging><modules><module>mod</module><module>../A</module><module>../C</module><module>.</module></modules></project>";
let c_pom = "<project><groupId>demo</groupId><artifactId>c</artifactId><version>1</version><modules><module>../B</module></modules></project>";
std::fs::write(a.join("pom.xml"), a_pom).unwrap();
std::fs::write(a.join("core/pom.xml"), "<project><artifactId>core</artifactId></project>").unwrap();
std::fs::write(b.join("pom.xml"), b_pom).unwrap();
std::fs::write(b.join("mod/pom.xml"), "<project><artifactId>bmod</artifactId></project>").unwrap();
std::fs::write(c.join("pom.xml"), c_pom).unwrap();
let wrapper = if cfg!(windows) { "mvnw.cmd" } else { "mvnw" };
std::fs::write(a.join(wrapper), "#!/bin/sh\n").unwrap();
std::fs::write(b.join(wrapper), "#!/bin/sh\n").unwrap();
std::fs::write(base.join(wrapper), "#!/bin/sh\n").unwrap();
let discovered = load_with_linked(&a, None, &[], &[]);
assert_eq!(discovered.projects.iter().map(|p| p.artifact_id.as_str()).collect::<Vec<_>>(), ["a", "core"]);
let stored = vec!["../B/pom.xml".into()];
let linked = load_with_linked(&a, None, &[], &stored);
let b_project = linked.projects.iter().find(|p| p.artifact_id == "b").unwrap();
assert!(!b_project.aggregated);
assert_eq!(b_project.pom, "../B/pom.xml");
assert_eq!(dunce::canonicalize(a.join(&b_project.dir)).unwrap(), b);
assert_eq!(dunce::canonicalize(&b_project.command).unwrap(), dunce::canonicalize(b.join(wrapper)).unwrap());
let a_project = linked.projects.iter().find(|p| p.artifact_id == "a").unwrap();
assert!(!a_project.aggregated);
assert_eq!(a_project.modules, ["core/pom.xml"]);
assert_eq!(dunce::canonicalize(&a_project.command).unwrap(), dunce::canonicalize(a.join(wrapper)).unwrap());
assert_eq!(linked.projects.iter().filter(|p| p.artifact_id == "a").count(), 1);
assert_eq!(linked.projects.iter().filter(|p| p.artifact_id == "b").count(), 1);
assert!(linked.projects.iter().any(|p| p.artifact_id == "c" && p.aggregated));
assert!(linked.projects.iter().any(|p| p.artifact_id == "bmod" && p.aggregated));
let again = load_with_linked(&a, None, &[], &stored);
assert_eq!(
again.projects.iter().map(|p| &p.pom).collect::<Vec<_>>(),
linked.projects.iter().map(|p| &p.pom).collect::<Vec<_>>()
);
let absolute = b.join("pom.xml").to_string_lossy().replace('\\', "/");
let via_a = a.join("../B/pom.xml").to_string_lossy().replace('\\', "/");
let normalized = normalize_linked_poms(
&a,
&[
"../B/pom.xml".into(),
absolute,
via_a,
a.join("pom.xml").to_string_lossy().into_owned(),
b.to_string_lossy().into_owned(),
"stub/pom.xml".into(),
"notes.xml".into(),
],
);
assert_eq!(normalized.len(), 3, "{normalized:?}");
assert_eq!(normalized[0], dunce::canonicalize(b.join("pom.xml")).unwrap().to_string_lossy().replace('\\', "/"));
assert_eq!(normalized[1], "pom.xml");
assert_eq!(normalized[2], "stub/pom.xml");
assert_eq!(crate::maven_external_roots(&a, &normalized), vec![b.clone()]);
assert_eq!(crate::maven_external_roots(&a, &stored), vec![b.clone()]);
let removed = load_with_linked(&a, None, &[], &[]);
assert!(removed.projects.iter().all(|p| p.artifact_id != "b"));
assert_eq!(std::fs::read_to_string(a.join("pom.xml")).unwrap(), a_pom);
assert_eq!(std::fs::read_to_string(b.join("pom.xml")).unwrap(), b_pom);
assert_eq!(std::fs::read_to_string(c.join("pom.xml")).unwrap(), c_pom);
assert!(b.join("mod/pom.xml").is_file());
}
/// The launcher recorded on an external project is that project's wrapper, and
/// running it uses the external directory as the working directory.
#[cfg(unix)]
#[test]
fn external_project_wrapper_runs_in_its_own_directory() {
let tmp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(tmp.path()).unwrap();
let a = base.join("A");
let b = base.join("B");
std::fs::create_dir_all(b.join("src/main/java")).unwrap();
std::fs::create_dir_all(&a).unwrap();
std::fs::write(a.join("pom.xml"), "<project><artifactId>a</artifactId></project>").unwrap();
std::fs::write(
b.join("pom.xml"),
"<project><artifactId>b</artifactId><build><sourceDirectory>src/main/java</sourceDirectory></build></project>",
)
.unwrap();
std::fs::write(b.join("src/main/java/B.java"), "class B {}").unwrap();
let script = b.join("mvnw");
std::fs::write(&script, "#!/bin/sh\npwd > \"$1\"\nprintf 'BUILD SUCCESS\\n'\n").unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap();
}
let model = load_with_linked(&a, None, &[], &["../B/pom.xml".into()]);
let project = model.projects.iter().find(|p| p.artifact_id == "b").unwrap();
let cwd = a.join(&project.dir);
assert_eq!(dunce::canonicalize(&cwd).unwrap(), b);
let marker = tmp.path().join("cwd");
let out = crate::process::run(
&project.command,
&[marker.to_string_lossy().into_owned()],
&cwd,
&std::collections::BTreeMap::new(),
&ide_core::CancellationToken::new(),
|_, _| {},
)
.unwrap();
assert!(out.success, "{}", out.text);
let recorded = dunce::canonicalize(std::fs::read_to_string(&marker).unwrap().trim()).unwrap();
assert_eq!(recorded, b);
}
/// #2877: an aggregator in a subfolder lists the root POM as `../`; linked, it holds every
/// module, the root one included, under the same POM paths the projects have.
#[test]
fn an_aggregator_in_a_subfolder_holds_the_root_pom() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let child = |id: &str, rel: &str| {
format!(
"<project><parent><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><relativePath>{rel}</relativePath></parent><artifactId>{id}</artifactId></project>"
)
};
write(root, "pom.xml", &child("app", "parent/pom.xml"));
write(
root,
"parent/pom.xml",
"<project><groupId>demo</groupId><artifactId>parent</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>../</module><module>../infra</module></modules></project>",
);
write(root, "infra/pom.xml", &child("infra", "../parent/pom.xml"));
let model = load_with_linked(root, None, &[], &["parent/pom.xml".into()]);
let tops: Vec<&str> = model.projects.iter().filter(|p| !p.aggregated).map(|p| p.artifact_id.as_str()).collect();
assert_eq!(tops, ["parent"]);
let parent = model.projects.iter().find(|p| p.artifact_id == "parent").unwrap();
assert_eq!(parent.modules, ["pom.xml", "infra/pom.xml"]);
for pom in &parent.modules {
assert!(model.projects.iter().any(|p| &p.pom == pom), "{pom}");
}
// A cycle (the root lists the aggregator, which lists the root) keeps the root on top.
write(
root,
"pom.xml",
"<project><groupId>demo</groupId><artifactId>app</artifactId><version>1.0</version><packaging>pom</packaging><modules><module>parent</module></modules></project>",
);
let model = load_with_linked(root, None, &[], &[]);
let tops: Vec<&str> = model.projects.iter().filter(|p| !p.aggregated).map(|p| p.artifact_id.as_str()).collect();
assert_eq!(tops, ["app"]);
}
#[test]
fn chosen_profiles_bring_their_modules_and_dependencies() {
let tmp = tempfile::tempdir().unwrap();
@@ -2780,9 +3206,9 @@ mod tests {
let text = std::fs::read_to_string(&path).unwrap();
// The property is declared nowhere along the chain.
assert_eq!(goto_declaration(&text, &path, at(&text, "${missing.key}"), None), None);
// The dependency names its own version: it is the declaration.
// The dependency names its own version and no local repository has its POM.
assert_eq!(goto_declaration(&text, &path, at(&text, "<artifactId>other-lib"), None), None);
// A management entry is a declaration itself, not a reference.
// A management entry declares the version; its artifact's POM is not at hand either.
let parent_path = tmp.path().join("pom.xml");
let parent_text = std::fs::read_to_string(&parent_path).unwrap();
assert_eq!(
@@ -2794,6 +3220,72 @@ mod tests {
assert_eq!(goto_declaration("<project></project>", &path, Pos::new(0, 3), None), None);
}
#[test]
fn artifacts_go_to_their_own_pom_in_the_project_or_the_local_repository() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repo");
let bom_dir = repo.join("com/alibaba/cloud/spring-cloud-alibaba-dependencies/2.2.6.RELEASE");
std::fs::create_dir_all(&bom_dir).unwrap();
let bom = "<project>\n <groupId>com.alibaba.cloud</groupId>\n <artifactId>spring-cloud-alibaba-dependencies</artifactId>\n</project>";
std::fs::write(bom_dir.join("spring-cloud-alibaba-dependencies-2.2.6.RELEASE.pom"), bom).unwrap();
let parent = format!(
r#"{PARENT_HEADER}<properties><alibaba.version>2.2.6.RELEASE</alibaba.version></properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-dependencies</artifactId>
<version>${{alibaba.version}}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>"#
);
let child = format!(
r#"{CHILD_HEADER}<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>core</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</project>"#
);
write_parent_child(tmp.path(), &parent, &child);
let core = tmp.path().join("core/pom.xml");
std::fs::create_dir_all(core.parent().unwrap()).unwrap();
std::fs::write(
&core,
"<project>\n <groupId>com.example</groupId>\n <artifactId>core</artifactId>\n</project>",
)
.unwrap();
let modules =
|| vec![ModulePom { group_id: "com.example".into(), artifact_id: "core".into(), pom: core.clone() }];
// A BOM import in <dependencyManagement> opens the BOM from the local repository.
let parent_path = tmp.path().join("pom.xml");
let parent_text = std::fs::read_to_string(&parent_path).unwrap();
let at_bom = at(&parent_text, "spring-cloud-alibaba-dependencies<");
let goto = goto_declaration_in(&parent_text, &parent_path, at_bom, Some(repo.clone()), modules).unwrap();
assert_eq!(goto.path, bom_dir.join("spring-cloud-alibaba-dependencies-2.2.6.RELEASE.pom"));
assert_eq!(goto.kind, PomGotoKind::ArtifactPom);
assert_eq!(goto.range.start, pos_at(bom, bom.find("<artifactId>").unwrap()));
// A dependency on a project module opens the module's POM.
let path = tmp.path().join("app/pom.xml");
let text = std::fs::read_to_string(&path).unwrap();
let goto =
goto_declaration_in(&text, &path, at(&text, "core</artifactId>"), Some(repo.clone()), modules).unwrap();
assert_eq!((goto.path.as_path(), goto.label.as_str()), (core.as_path(), "com.example:core"));
// The <parent> section opens the parent POM.
let goto = goto_declaration_in(&text, &path, at(&text, "parent</artifactId>"), Some(repo), modules).unwrap();
assert_eq!(goto.path, canon(&parent_path));
assert_eq!(goto.kind, PomGotoKind::ArtifactPom);
}
#[test]
fn a_property_reference_after_non_ascii_text_is_still_found() {
let text = format!(
+694
Переглянути файл
@@ -0,0 +1,694 @@
//! Completion of Maven coordinates in a POM: inside a `<dependency>`, the
//! `<groupId>`, `<artifactId>` and `<version>` values are completed from the
//! local repository layout (`<repo>/<group path>/<artifactId>/<version>/…`).
//! The Maven Central search endpoint tops the lists up remotely; the host op
//! fetches those (cached) and merges them behind the local ones, so this side
//! stays pure text and directory logic and tests build their own repositories.
use super::xml::{self, Element};
use super::{element_path, interpolate, read_properties};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::cmp::Ordering;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
/// The coordinate element a POM completion fills.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum PomCompletionKind {
/// The dependency's `<groupId>`.
GroupId,
/// The dependency's `<artifactId>`.
ArtifactId,
/// The dependency's `<version>`.
Version,
}
/// One suggestion: the coordinate as inserted, and an optional note shown in
/// the lookup (the owning group of an artifactId, its newest installed version).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PomCompletionOption {
/// The text the completion inserts (`org.example.lib`, `example-lib`, `1.2.0`).
pub label: String,
pub detail: Option<String>,
}
/// The completion offered at a caret of a POM: the replaced value span (UTF-16
/// offsets) and the suggestions, local ones first.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub struct PomCompletion {
/// UTF-16 offset where the completed text starts: the value's first character.
pub from: u32,
/// UTF-16 offset where the replaced value ends: past the caret when the caret
/// sits inside an existing value, which is then replaced whole.
pub to: u32,
/// Which coordinate element the caret is in.
pub kind: PomCompletionKind,
/// The suggestions, local repository first.
pub options: Vec<PomCompletionOption>,
}
/// Where the caret sits in a `<dependency>` coordinate element: what is being
/// completed, the typed prefix, and the sibling coordinates the rest need.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PomCompletionTarget {
/// UTF-16 offset of the value's first character (the caret in an empty value).
pub from: u32,
/// UTF-16 offset past the value's last character (the caret in an empty value).
pub to: u32,
/// The value typed so far: the text between `from` and the caret, trimmed.
pub typed: String,
pub kind: PomCompletionKind,
/// The interpolated `<groupId>` sibling, when completing the artifactId and
/// the group is written out plain (no unresolved `${property}`).
pub group: Option<String>,
/// The interpolated `<artifactId>` sibling, when completing the version.
pub artifact: Option<String>,
}
/// A coordinate element's text, with the POM's own `${properties}` replaced;
/// `None` when the element is missing or a property stays unresolved (the
/// coordinates behind it are not known without the parent chain, which the
/// completion does not follow).
fn coordinate(e: Option<&Element>, props: &HashMap<String, String>) -> Option<String> {
let text = interpolate(e?.text.trim(), props);
(!text.is_empty() && !text.contains("${")).then_some(text)
}
fn own_properties(root: &Element) -> HashMap<String, String> {
read_properties(root.child("properties")).into_iter().collect()
}
/// UTF-16 offset -> byte offset, clamped to `text`.
fn utf16_to_byte(text: &str, offset: u32) -> usize {
let mut seen = 0u32;
for (byte, c) in text.char_indices() {
if seen >= offset {
return byte;
}
seen += c.len_utf16() as u32;
}
text.len()
}
/// Byte offset -> UTF-16 offset.
fn utf16_at(text: &str, byte: usize) -> u32 {
text[..byte].encode_utf16().count() as u32
}
/// Byte offset just past the start tag's `>` of the element at `start`.
fn start_tag_end(text: &str, start: usize) -> Option<usize> {
let bytes = text.as_bytes();
let mut i = start.checked_add(1)?;
let mut quote = None;
while i < bytes.len() {
match (quote, bytes[i]) {
(Some(q), b) if b == q => quote = None,
(Some(_), _) => {}
(None, b'"' | b'\'') => quote = Some(bytes[i]),
(None, b'>') => return Some(i + 1),
_ => {}
}
i += 1;
}
None
}
/// What to complete at the caret (`offset`, UTF-16) of the POM `text`: the
/// `<groupId>` / `<artifactId>` / `<version>` element of a `<dependency>` the
/// caret sits in the value of. `None` everywhere else — other elements, carets
/// in an end tag or a `${property}` reference, dependencies without the
/// `<groupId>` (and, for versions, the `<artifactId>`) their completions need.
pub fn completion_target(text: &str, offset: u32) -> Option<PomCompletionTarget> {
let root = xml::parse(text).filter(|e| e.name == "project")?;
let at = utf16_to_byte(text, offset);
// A caret at the very end of the input sits in the element still being
// typed, whose range reaches to the input's end; step back one character
// so `element_path` sees it (the caret's own offset stays `at`).
let probe = if at == text.len() && at > 0 {
let mut p = at - 1;
while p > 0 && !text.is_char_boundary(p) {
p -= 1;
}
p
} else {
at
};
let elements = element_path(&root, probe);
let e = *elements.last()?;
let parent = elements.get(elements.len().checked_sub(2)?)?;
let kind = match e.name.as_str() {
"groupId" => PomCompletionKind::GroupId,
"artifactId" => PomCompletionKind::ArtifactId,
"version" => PomCompletionKind::Version,
_ => return None,
};
if parent.name != "dependency" || at > e.inner_end {
return None;
}
let inner_start = start_tag_end(text, e.start)?;
let value = &text[inner_start..e.inner_end.max(inner_start)];
if value.contains("${") {
return None; // Property references are goto-declaration territory, not completion.
}
let lead = value.len() - value.trim_start().len();
let trail = value.len() - value.trim_end().len();
let (mut from, mut to) = (inner_start + lead, e.inner_end.max(inner_start) - trail);
if from > to {
// A value of whitespace only: complete at the caret.
from = at.min(to);
to = to.max(from);
}
from = from.min(at);
to = to.max(from);
let props = own_properties(&root);
let group = (kind != PomCompletionKind::GroupId).then(|| coordinate(parent.child("groupId"), &props)).flatten();
let artifact =
(kind == PomCompletionKind::Version).then(|| coordinate(parent.child("artifactId"), &props)).flatten();
let typed = text[from..at].trim().to_owned();
Some(PomCompletionTarget { from: utf16_at(text, from), to: utf16_at(text, to), typed, kind, group, artifact })
}
/// The suggestions the local repository at `repo` offers for `target`: group
/// ids installed under it, the artifacts of the target's group, or the versions
/// of the target's artifact. Every scan is bounded (see `SCAN_BUDGET`) and
/// reads only directory listings of `repo` — nothing outside it, no jar
/// contents.
pub fn local_options(repo: &Path, target: &PomCompletionTarget) -> Vec<PomCompletionOption> {
let mut budget = SCAN_BUDGET;
match target.kind {
// `org.spr` completes the `spr` part under `org/`, `org.` the next
// segment under `org/`; every typed segment is validated so POM text
// cannot walk out of the repository.
PomCompletionKind::GroupId => {
let segments: Vec<&str> = target.typed.split('.').collect();
let (partial, base) = segments.split_last().unwrap_or((&"", &[]));
if base.iter().all(|s| segment(s)) {
let dir = base.iter().fold(repo.to_path_buf(), |p, s| p.join(s));
scan_groups(&dir, &base.join("."), partial, &mut budget)
} else {
Vec::new()
}
}
PomCompletionKind::ArtifactId => target
.group
.as_deref()
.and_then(|g| repo_dir(repo, &[g]))
.map_or_else(Vec::new, |dir| scan_artifacts(&dir, &target.typed, &mut budget)),
PomCompletionKind::Version => target
.group
.as_deref()
.zip(target.artifact.as_deref())
.and_then(|(g, a)| repo_dir(repo, &[g, a]))
.map_or_else(Vec::new, |dir| scan_versions(&dir, &target.typed, &mut budget)),
}
}
/// Directory listings one completion may read: the scans are interactive, so
/// they stop before they could be felt, and a huge repository simply yields
/// fewer suggestions (its shallowest first — the walk is breadth-preferring).
const SCAN_BUDGET: u32 = 2048;
/// Suggestion caps per kind: group ids and artifact ids fill a lookup page,
/// versions list an artifact's releases.
const MAX_GROUP_OPTIONS: usize = 60;
const MAX_ARTIFACT_OPTIONS: usize = 60;
const MAX_VERSION_OPTIONS: usize = 100;
/// The cap the merged list (local plus Maven Central's) is truncated to.
pub const MAX_OPTIONS: usize = 100;
/// A directory entry whose name passes the repository layout's alphabet
/// (mirrors `sources::artifact`'s): letters, digits, `._-`. Path separators
/// and `..` never pass, so a POM's `${…}`-free text cannot traverse out of
/// the repository.
fn segment(name: &str) -> bool {
!name.is_empty()
&& name != "."
&& name != ".."
&& name.chars().all(|c| c.is_ascii_alphanumeric() || "._-".contains(c))
}
/// The subdirectories of `dir` with repository-layout names, spending `budget`
/// on the listing; `None` when the budget ran out before `dir` was listed.
fn subdirs(dir: &Path, budget: &mut u32) -> Option<Vec<String>> {
if *budget == 0 {
return None;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return Some(Vec::new());
};
let mut names = Vec::new();
for entry in entries.flatten() {
// The budget bounds file-type checks too: a plain listing answers.
if entry.file_type().map(|t| t.is_dir()).unwrap_or(false)
&& let Ok(name) = entry.file_name().into_string()
&& segment(&name)
{
names.push(name);
}
}
Some(names)
}
/// Whether `dir` holds files — a version directory does (jar, pom, the
/// tracking files), a group path segment never does.
fn holds_files(dir: &Path, budget: &mut u32) -> bool {
if *budget == 0 {
return false;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
entries.flatten().any(|e| e.file_type().map(|t| t.is_file()).unwrap_or(false))
}
/// Whether the version directory `dir` holds the version: its `.pom` or a `.jar`. A failed
/// download leaves only `*.lastUpdated` tracking files behind, for a version that may not
/// even exist (one typed while the POM was being edited).
fn installed(dir: &Path, budget: &mut u32) -> bool {
if *budget == 0 {
return false;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
entries.flatten().any(|e| {
e.file_type().map(|t| t.is_file()).unwrap_or(false)
&& e.file_name().to_str().is_some_and(|n| n.ends_with(".pom") || n.ends_with(".jar"))
})
}
/// Whether `dir` is an artifact directory: it holds versions of one artifact,
/// recognizable by a `maven-metadata*.xml` beside them or a child directory
/// holding files. A group segment's children are directories without files.
fn artifact_dir(dir: &Path, budget: &mut u32) -> bool {
if *budget == 0 {
return false;
}
*budget -= 1;
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
for entry in entries.flatten() {
let Ok(name) = entry.file_name().into_string() else { continue };
if name.starts_with("maven-metadata") {
return true;
}
}
let Some(children) = subdirs(dir, budget) else { return false };
children.iter().any(|c| holds_files(&dir.join(c), budget))
}
/// Whether `dir` is a group directory (holds artifact directories) and whether
/// it also holds deeper group segments worth descending into.
fn group_dir(dir: &Path, budget: &mut u32) -> Option<(bool, bool)> {
let children = subdirs(dir, budget)?;
let (mut group, mut descend) = (false, false);
for child in &children {
if artifact_dir(&dir.join(child), budget) {
group = true;
} else {
descend = true;
}
}
Some((group, descend))
}
/// The group ids installed under `dir` (which the typed prefix `prefix` —
/// "" for the repository root — already names), completing the partial last
/// segment `partial`, shallower groups first.
fn scan_groups(dir: &Path, prefix: &str, partial: &str, budget: &mut u32) -> Vec<PomCompletionOption> {
let mut out = Vec::new();
scan_groups_into(dir, prefix, partial, &mut out, budget);
out.sort_by(|a, b| a.label.cmp(&b.label));
out
}
fn scan_groups_into(dir: &Path, prefix: &str, partial: &str, out: &mut Vec<PomCompletionOption>, budget: &mut u32) {
if out.len() >= MAX_GROUP_OPTIONS {
return;
}
let Some(children) = subdirs(dir, budget) else { return };
for name in children {
if !name.starts_with(partial) {
continue;
}
let path = dir.join(&name);
let label = if prefix.is_empty() { name.clone() } else { format!("{prefix}.{name}") };
let Some((group, descend)) = group_dir(&path, budget) else { return };
if group {
out.push(PomCompletionOption { label: label.clone(), detail: None });
if out.len() >= MAX_GROUP_OPTIONS {
return;
}
}
// Deeper groups (org.example under org.example's own parent) follow the
// shallower ones, so a truncated scan of a huge repository still offers
// every first-level segment.
if descend {
scan_groups_into(&path, &label, "", out, budget);
}
}
}
/// The artifact ids installed under the group directory `dir`, completing the
/// typed prefix, each with its newest installed version as the detail.
fn scan_artifacts(dir: &Path, typed: &str, budget: &mut u32) -> Vec<PomCompletionOption> {
let Some(children) = subdirs(dir, budget) else { return Vec::new() };
let mut out = Vec::new();
for name in children {
if !name.starts_with(typed) || !artifact_dir(&dir.join(&name), budget) {
continue;
}
let detail = newest_version(&dir.join(&name), budget);
out.push(PomCompletionOption { label: name, detail });
}
out.sort_by(|a, b| a.label.cmp(&b.label));
out.truncate(MAX_ARTIFACT_OPTIONS);
out
}
/// The newest version directory of the artifact at `dir`, by Maven's ordering.
fn newest_version(dir: &Path, budget: &mut u32) -> Option<String> {
let children = subdirs(dir, budget)?;
children.into_iter().filter(|name| installed(&dir.join(name), budget)).max_by(|a, b| compare_versions(a, b))
}
/// The versions installed of the artifact at `dir`, completing the typed
/// prefix, newest first.
fn scan_versions(dir: &Path, typed: &str, budget: &mut u32) -> Vec<PomCompletionOption> {
let Some(children) = subdirs(dir, budget) else { return Vec::new() };
let mut out: Vec<_> = children
.into_iter()
.filter(|name| name.starts_with(typed) && installed(&dir.join(name), budget))
.map(|name| PomCompletionOption { label: name, detail: None })
.collect();
out.sort_by(|a, b| compare_versions(&b.label, &a.label));
out.truncate(MAX_VERSION_OPTIONS);
out
}
/// The repository directory of `coordinates` (a group id, or a group and an artifact
/// id): the group's dots are directories, an artifact id is one directory whatever dots
/// it has (`org/eclipse/platform/org.eclipse.core.runtime`). Each segment is validated
/// against the layout's alphabet so POM text cannot walk outside `repo`.
fn repo_dir(repo: &Path, coordinates: &[&str]) -> Option<PathBuf> {
let (group, artifact) = coordinates.split_first()?;
let segments: Vec<&str> = group.split('.').chain(artifact.iter().copied()).collect();
segments.iter().all(|s| segment(s)).then(|| segments.iter().fold(repo.to_path_buf(), |p, s| p.join(s)))
}
/// Maven-ish version order, best-effort (a full `ComparableVersion` is out of
/// proportion for ordering a lookup): numeric runs compare numerically, letter
/// runs by their qualifier rank (`alpha < beta < m < rc < snapshot < release <
/// sp`, unknown ones with releases lexically); missing parts rank as zero.
fn compare_versions(a: &str, b: &str) -> Ordering {
fn tokens(v: &str) -> Vec<String> {
v.split(|c: char| !c.is_ascii_alphanumeric()).filter(|t| !t.is_empty()).map(str::to_ascii_lowercase).collect()
}
fn rank(t: &str) -> Option<u8> {
match t {
"a" | "alpha" => Some(0),
"b" | "beta" => Some(1),
"m" | "milestone" => Some(2),
"rc" | "cr" => Some(3),
"snapshot" => Some(4),
"" | "ga" | "final" => Some(5),
"sp" => Some(6),
_ => None,
}
}
let (a, b) = (tokens(a), tokens(b));
for i in 0..a.len().max(b.len()) {
let (x, y) = (a.get(i).map(String::as_str), b.get(i).map(String::as_str));
let (xn, yn) = (x.and_then(|s| s.parse::<u64>().ok()), y.and_then(|s| s.parse::<u64>().ok()));
let order = match (xn, yn) {
// Numbers compare numerically; a number outranks a qualifier.
(Some(xn), Some(yn)) => xn.cmp(&yn),
(Some(_), None) if y.is_some() => return Ordering::Greater,
(None, Some(_)) if x.is_some() => return Ordering::Less,
_ => {
let xr = x.and_then(rank).unwrap_or(5);
let yr = y.and_then(rank).unwrap_or(5);
if xr != yr { xr.cmp(&yr) } else { x.unwrap_or("").cmp(y.unwrap_or("")) }
}
};
if order != Ordering::Equal {
return order;
}
}
Ordering::Equal
}
#[cfg(test)]
mod tests {
use super::*;
/// A small local repository: every file is laid out as Maven installs it,
/// so the scans recognize group, artifact and version directories.
fn repo(dir: &Path) -> PathBuf {
let repo = dir.join("repository");
for file in [
"org/apache/maven/maven-core/4.0.0-rc-5/maven-core-4.0.0-rc-5.jar",
"org/apache/maven/plugins/maven-compiler-plugin/3.8.1/maven-compiler-plugin-3.8.1.jar",
"org/example/lib/1.0/lib-1.0.jar",
"org/example/lib/1.9.1/lib-1.9.1.jar",
"org/example/lib/1.10/lib-1.10.jar",
"org/example/lib/2.0-beta-1/lib-2.0-beta-1.jar",
"org/example/lib/2.0/lib-2.0.pom",
"net/nu/netty/4.1.100/netty-4.1.100.jar",
"org/eclipse/platform/org.eclipse.core.runtime/3.31.0/org.eclipse.core.runtime-3.31.0.jar",
// Failed downloads (a version typed while editing): tracking files only, not installed.
"org/example/lib/2/lib-2.jar.lastUpdated",
"org/example/lib/3.0/lib-3.0.pom.lastUpdated",
] {
let path = repo.join(file);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(&path, b"").unwrap();
}
repo
}
fn target(text: &str, needle: &str) -> Option<PomCompletionTarget> {
completion_target(text, (text.find(needle).unwrap() + needle.len()) as u32)
}
#[test]
fn completes_each_coordinate_element_of_a_dependency() {
let text = "<project><dependencies><dependency>\
<groupId>org.example</groupId><artifactId>lib</artifactId><version>1.0</version>\
</dependency></dependencies></project>";
let t = target(text, "<groupId>org.example").unwrap();
assert_eq!(t.kind, PomCompletionKind::GroupId);
assert_eq!(t.typed, "org.example");
assert_eq!((t.from, t.to), (44, 55));
assert!(t.group.is_none() && t.artifact.is_none());
let t = target(text, "<artifactId>lib").unwrap();
assert_eq!(t.kind, PomCompletionKind::ArtifactId);
assert_eq!(t.typed, "lib");
assert_eq!(t.group.as_deref(), Some("org.example"));
assert!(t.artifact.is_none());
let t = target(text, "<version>1.0").unwrap();
assert_eq!(t.kind, PomCompletionKind::Version);
assert_eq!(t.typed, "1.0");
assert_eq!(t.group.as_deref(), Some("org.example"));
assert_eq!(t.artifact.as_deref(), Some("lib"));
}
#[test]
fn completes_while_the_element_is_being_typed_and_the_value_is_empty() {
let empty = "<project><dependencies><dependency><groupId></groupId></dependency></dependencies></project>";
let t = target(empty, "<groupId>").unwrap();
assert_eq!(t.kind, PomCompletionKind::GroupId);
assert_eq!(t.typed, "");
assert_eq!(t.from, t.to);
let open = "<project><dependencies><dependency><artifactId>li";
let t = target(open, "li").unwrap();
assert_eq!(t.kind, PomCompletionKind::ArtifactId);
assert_eq!(t.typed, "li");
// The value under construction ends with the input and is replaced whole.
assert_eq!(t.to, open.len() as u32);
}
#[test]
fn completes_only_dependency_coordinates() {
// The same elements outside a <dependency> get nothing: a <plugin> is
// completed by the repository, not coordinates.
let plugin =
"<project><build><plugins><plugin><groupId>org.apache.maven</groupId></plugin></plugins></build></project>";
assert_eq!(target(plugin, "<groupId>org.apache.maven"), None);
// A caret in the end tag, or on a `${property}` value, is not a completion.
let text =
"<project><dependencies><dependency><groupId>org.example</groupId></dependency></dependencies></project>";
assert_eq!(target(text, "</groupId>"), None);
let property = "<project><dependencies><dependency><groupId>${example.group}</groupId></dependency></dependencies></project>";
assert_eq!(target(property, "${example.group}"), None);
// A dependency's groupId is required before its artifacts can be listed.
let no_group =
"<project><dependencies><dependency><artifactId>lib</artifactId></dependency></dependencies></project>";
let t = target(no_group, "<artifactId>lib").unwrap();
assert!(t.group.is_none());
}
#[test]
fn interpolates_the_poms_own_properties_for_siblings() {
let text = "<project>\
<properties><example.group>org.example</example.group><example.version>2.0</example.version></properties>\
<dependencies><dependency><groupId>${example.group}</groupId><artifactId>lib</artifactId><version>${example.version}</version></dependency></dependencies>\
</project>";
// The groupId the caret is typing stays a completion; the siblings resolve.
let t = target(text, "<artifactId>lib").unwrap();
assert_eq!(t.group.as_deref(), Some("org.example"));
// A `${property}` under the caret itself gets nothing to complete.
assert_eq!(target(text, "<version>${example.version}"), None);
}
#[test]
fn local_scan_lists_installed_groups_shallowest_first() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
let options = |typed: &str| {
local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: typed.into(),
kind: PomCompletionKind::GroupId,
group: None,
artifact: None,
},
)
.into_iter()
.map(|o| o.label)
.collect::<Vec<_>>()
};
// Only directories holding artifacts are group ids: `net` and `org`
// alone never receive an install, their deeper levels do.
let org = ["org.apache.maven", "org.apache.maven.plugins", "org.eclipse.platform", "org.example"];
assert_eq!(options(""), [&["net.nu"][..], &org].concat());
assert_eq!(options("org"), org);
assert_eq!(options("org."), org);
assert_eq!(options("org.spr"), Vec::<String>::new());
assert_eq!(options("net."), ["net.nu"]);
}
#[test]
fn local_scan_lists_artifacts_of_the_group_with_their_newest_version() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
let options = |typed: &str| {
local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: typed.into(),
kind: PomCompletionKind::ArtifactId,
group: Some("org.example".into()),
artifact: None,
},
)
.into_iter()
.map(|o| (o.label, o.detail))
.collect::<Vec<_>>()
};
assert_eq!(options(""), [("lib".to_owned(), Some("2.0".to_owned()))]);
assert_eq!(options("net"), Vec::<(String, Option<String>)>::new());
}
#[test]
fn local_scan_lists_versions_of_the_artifact_newest_first() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
let options = |typed: &str| {
local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: typed.into(),
kind: PomCompletionKind::Version,
group: Some("org.example".into()),
artifact: Some("lib".into()),
},
)
.into_iter()
.map(|o| o.label)
.collect::<Vec<_>>()
};
assert_eq!(options(""), ["2.0", "2.0-beta-1", "1.10", "1.9.1", "1.0"]);
assert_eq!(options("1."), ["1.10", "1.9.1", "1.0"]);
// An artifact id with dots is one directory.
let dotted = local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: "".into(),
kind: PomCompletionKind::Version,
group: Some("org.eclipse.platform".into()),
artifact: Some("org.eclipse.core.runtime".into()),
},
);
assert_eq!(dotted.into_iter().map(|o| o.label).collect::<Vec<_>>(), ["3.31.0"]);
}
#[test]
fn a_poms_text_cannot_walk_out_of_the_repository() {
let tmp = tempfile::tempdir().unwrap();
let repo = repo(tmp.path());
for group in ["../../etc", "..", "org/../.."] {
let options = local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: "".into(),
kind: PomCompletionKind::ArtifactId,
group: Some(group.into()),
artifact: None,
},
);
assert!(options.is_empty(), "{group}");
}
}
#[test]
fn the_scan_stops_at_the_option_cap() {
let tmp = tempfile::tempdir().unwrap();
let repo = tmp.path().join("repository");
for i in 0..MAX_GROUP_OPTIONS + 10 {
let group = repo.join(format!("com/example/g{i}"));
std::fs::create_dir_all(group.join("art/1.0")).unwrap();
std::fs::write(group.join("art/1.0/art-1.0.jar"), b"").unwrap();
}
let options = local_options(
&repo,
&PomCompletionTarget {
from: 0,
to: 0,
typed: "".into(),
kind: PomCompletionKind::GroupId,
group: None,
artifact: None,
},
);
assert_eq!(options.len(), MAX_GROUP_OPTIONS);
}
#[test]
fn maven_version_order() {
let mut versions = ["2.0", "1.0", "2.0-beta-1", "2.0-alpha-2", "1.10", "1.9.1", "4.0.0-rc-5", "4.0.0-rc-13"];
versions.sort_by(|a, b| compare_versions(a, b));
assert_eq!(versions, ["1.0", "1.9.1", "1.10", "2.0-alpha-2", "2.0-beta-1", "2.0", "4.0.0-rc-5", "4.0.0-rc-13"]);
}
}
+9
Переглянути файл
@@ -270,6 +270,13 @@ fn run(
/// The first error lines of a failed build, for the tool window.
pub(crate) fn error_summary(text: &str) -> String {
// The JVM's "could not find or load main class" (localized) when `gradlew` has no jar.
if text.contains("org.gradle.wrapper.GradleWrapperMain") {
return "gradle/wrapper/gradle-wrapper.jar is missing (often excluded by a `*.jar` rule in .gitignore). \
Restore it, run `gradle wrapper` with an installed Gradle, or choose Distribution: Local installation \
in Settings › Build, Execution, Deployment › Gradle."
.to_owned();
}
let lines: Vec<&str> = text
.lines()
.filter(|l| {
@@ -909,6 +916,8 @@ mod tests {
let text = "[INFO] x\n[ERROR] Failed to execute goal\n[ERROR] \n[ERROR] Re-run Maven using -X\n";
assert_eq!(error_summary(text), "Failed to execute goal");
assert_eq!(error_summary("one\nlast line\n\n"), "last line");
let missing_jar = "错误: 找不到或无法加载主类 org.gradle.wrapper.GradleWrapperMain\n";
assert!(error_summary(missing_jar).starts_with("gradle/wrapper/gradle-wrapper.jar is missing"));
}
#[test]
+15
Переглянути файл
@@ -27,6 +27,13 @@ pub fn set_text(text: &str, owner: usize) -> std::io::Result<()> {
imp::set_text(text, owner)
}
/// The clipboard's plain text, None when it holds none. macOS only (main thread);
/// Linux reads it asynchronously through GTK in the app, Windows through the WebView.
#[cfg(target_os = "macos")]
pub fn text() -> Option<String> {
imp::text()
}
/// Size of `DROPFILES`: `pFiles`, `pt` (two `LONG`s), `fNC`, `fWide`.
const HEADER: usize = 20;
@@ -373,6 +380,14 @@ mod imp {
})
}
pub fn text() -> Option<String> {
debug_assert!(MainThreadMarker::new().is_some(), "NSPasteboard access requires the main thread");
autoreleasepool(|_| {
// SAFETY: the type is an AppKit constant, valid for the process' lifetime.
NSPasteboard::generalPasteboard().stringForType(unsafe { NSPasteboardTypeString }).map(|s| s.to_string())
})
}
pub fn set_text(text: &str, _owner: usize) -> std::io::Result<()> {
debug_assert!(MainThreadMarker::new().is_some(), "NSPasteboard access requires the main thread");
autoreleasepool(|_| {
+3
Переглянути файл
@@ -67,6 +67,8 @@ pub struct EditorAppearanceSettings {
/// Smooth caret movement curve: `snappy` | `gliding`.
pub caret_movement_style: String,
pub line_numbers: bool,
/// Gutter numbering: `absolute` | `relative` | `hybrid` (absolute on the caret line, distances elsewhere).
pub line_number_mode: String,
pub method_separators: bool,
/// Sub-options of `editor.showWhitespace`.
pub whitespace_leading: bool,
@@ -96,6 +98,7 @@ impl Default for EditorAppearanceSettings {
smooth_caret_movement: false,
caret_movement_style: "snappy".into(),
line_numbers: true,
line_number_mode: "absolute".into(),
method_separators: false,
whitespace_leading: true,
whitespace_inner: true,
+8
Переглянути файл
@@ -8,13 +8,18 @@
use crate::{DebugSessionId, Diagnostic, DocId, RunId, ServerId, TaskId, TerminalId, TextChange};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::collections::BTreeMap;
use std::sync::Arc;
/// What a built-in MCP tool asks the project window for. `AgentRun` runs `command` (a shell
/// line) in the Run tool window's agent tab `name`, on pipes, answered once the process
/// started or failed to.
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
#[serde(tag = "type", rename_all = "camelCase", rename_all_fields = "camelCase")]
pub enum McpUiRequest {
Confirm { client: String, tool: String, subject: String, project: String, detail: Option<String> },
OpenFile { path: crate::IpcPath, line: Option<u32>, column: Option<u32> },
AgentRun { name: String, command: String, cwd: crate::IpcPath, env: BTreeMap<String, String> },
}
#[derive(Clone, Debug, Serialize, Deserialize, Type)]
@@ -72,6 +77,9 @@ pub enum IdeEvent {
WorkspaceEditApplied {
id: u32,
paths: Vec<crate::IpcPath>,
/// Made by an agent through the MCP server's tools: undoable like a refactoring, but
/// not announced as one.
by_agent: bool,
},
/// An open document's language changed (Settings › File Types, Override File
/// Type), or its UI mode (a framework's root marker appeared or went).
+1
Переглянути файл
@@ -21,6 +21,7 @@ pub mod process;
pub mod project_dir;
pub mod project_roots;
pub mod proxy_settings;
pub mod real_path;
pub mod remote_settings;
pub mod settings;
pub mod task;
+52 -1
Переглянути файл
@@ -76,7 +76,8 @@ fn decode_tool_output_with_code_page(bytes: &[u8], code_page: u32) -> String {
text
}
fn encoding_for_code_page(cp: u32) -> Option<&'static encoding_rs::Encoding> {
/// The `encoding_rs` encoding of a Windows code page, when it has one.
pub fn encoding_for_code_page(cp: u32) -> Option<&'static encoding_rs::Encoding> {
Some(match cp {
65001 => encoding_rs::UTF_8,
936 => encoding_rs::GBK,
@@ -101,6 +102,56 @@ fn encoding_for_code_page(cp: u32) -> Option<&'static encoding_rs::Encoding> {
})
}
/// The code page Windows console programs write to a pipe in: the OEM code page (GBK on a
/// Chinese system, Shift_JIS on a Japanese one, EUC-KR on a Korean one, 437 / 850 / 866…
/// elsewhere). `None` when it is UTF-8 (Windows' "Use Unicode UTF-8" option) or off Windows.
pub fn console_code_page() -> Option<u32> {
#[cfg(windows)]
{
// SAFETY: GetOEMCP is an argument-less query with no preconditions.
let cp = unsafe { windows_sys::Win32::Globalization::GetOEMCP() };
(cp != 65001).then_some(cp)
}
#[cfg(not(windows))]
{
None
}
}
/// Whether every character of code page `cp` is one byte, so text in it can be decoded
/// piece by piece.
#[cfg(windows)]
pub fn is_single_byte_code_page(cp: u32) -> bool {
use windows_sys::Win32::Globalization::{CPINFO, GetCPInfo};
// SAFETY: CPINFO is plain data, which GetCPInfo fills in.
unsafe {
let mut info: CPINFO = std::mem::zeroed();
GetCPInfo(cp, &mut info) != 0 && info.MaxCharSize == 1
}
}
/// `bytes` in code page `cp`, decoded by Windows (for the code pages `encoding_rs` lacks,
/// like the OEM pages 437, 850 or 852).
#[cfg(windows)]
pub fn decode_code_page(bytes: &[u8], cp: u32) -> String {
use windows_sys::Win32::Globalization::MultiByteToWideChar;
let Ok(len) = i32::try_from(bytes.len()) else {
return String::from_utf8_lossy(bytes).into_owned();
};
if len == 0 {
return String::new();
}
// SAFETY: `bytes` holds `len` bytes; no output buffer asks for the length needed.
let needed = unsafe { MultiByteToWideChar(cp, 0, bytes.as_ptr(), len, std::ptr::null_mut(), 0) };
if needed <= 0 {
return String::from_utf8_lossy(bytes).into_owned();
}
let mut wide = vec![0u16; needed as usize];
// SAFETY: `wide` holds `needed` units.
let written = unsafe { MultiByteToWideChar(cp, 0, bytes.as_ptr(), len, wide.as_mut_ptr(), needed) };
String::from_utf16_lossy(&wide[..written.max(0) as usize])
}
const STD_PROPERTIES: &[&str] = &["stdout.encoding", "stderr.encoding", "sun.stdout.encoding", "sun.stderr.encoding"];
const FILE_ENCODING: &str = "file.encoding";
const TOOL_OPTIONS: &str = "JAVA_TOOL_OPTIONS";
+53
Переглянути файл
@@ -0,0 +1,53 @@
//! Where a path really lands, and whether it reaches git's own metadata, for writes that must
//! stay inside a project and out of its `.git`.
use std::path::{Path, PathBuf};
/// `joined` after `..`/symlink resolution as far as the file system allows:
/// the canonical path when it exists, else the canonicalized deepest existing
/// ancestor with the still-missing tail resolved textually — so the location
/// of a missing path is still judged where it would really land.
pub fn resolve_real(joined: &Path) -> PathBuf {
let mut tail = Vec::new();
let mut candidate = joined.to_path_buf();
let mut real = loop {
match dunce::canonicalize(&candidate) {
Ok(real) => break real,
Err(_) => {
let Some(last) = candidate.components().next_back() else {
return joined.to_path_buf();
};
tail.push(last.as_os_str().to_owned());
if !candidate.pop() {
return joined.to_path_buf();
}
}
}
};
for part in tail.iter().rev() {
if part == ".." {
real.pop();
} else if part != "." {
real.push(part);
}
}
real
}
/// Whether a path (relative to the project root) goes through git's own `.git` folder or file.
/// git runs what it finds in `.git` (hooks, `gpg.program`, filters), so a content write must
/// never reach it. Names are normalized for trailing dots and spaces and NTFS stream suffixes;
/// the caller also checks the short-name-resolved path via [`resolve_real`].
pub fn names_git_dir(rel: &Path) -> bool {
rel.components().any(|part| {
let name = part.as_os_str().to_string_lossy();
let name = name.split(':').next().unwrap_or_default().trim_end_matches(['.', ' ']);
name.eq_ignore_ascii_case(".git")
})
}
/// A resolved write destination under `root` that reaches git's own `.git` metadata, judged by
/// the name given and by the name the file system resolves it to.
pub fn reaches_git_dir(root: &Path, real: &Path) -> bool {
real.strip_prefix(root).is_ok_and(names_git_dir) || resolve_real(real).strip_prefix(root).is_ok_and(names_git_dir)
}
+100 -3
Переглянути файл
@@ -102,6 +102,11 @@ pub struct AgentSettings {
#[serde(deserialize_with = "or_default")]
#[specta(type = WhileRunning)]
pub while_running: WhileRunning,
/// Whether new tasks start in the agent's mode that approves everything (Claude Code's
/// bypass permissions, Codex's full access) or in one that asks. User settings only.
#[serde(deserialize_with = "or_default")]
#[specta(type = NewTaskApprovals)]
pub new_task_approvals: NewTaskApprovals,
/// Settings new tasks of an agent start with: agent id → option id → value.
#[serde(deserialize_with = "agent_defaults")]
#[specta(type = BTreeMap<String, BTreeMap<String, String>>)]
@@ -127,6 +132,7 @@ impl Default for AgentSettings {
expand_tool_output: false,
send_with: SendKey::Enter,
while_running: WhileRunning::Queue,
new_task_approvals: NewTaskApprovals::Bypass,
defaults: BTreeMap::new(),
mcp_servers: BTreeMap::new(),
}
@@ -194,6 +200,19 @@ where
Ok(serde_json::from_value(serde_json::Value::deserialize(d)?).unwrap_or_default())
}
/// A theme id, with the built-in themes that were removed read as the Islands theme of their kind.
fn theme_id<'de, D>(d: D) -> Result<String, D::Error>
where
D: serde::Deserializer<'de>,
{
let id = String::deserialize(d)?;
Ok(match id.as_str() {
"one-light" => "light".into(),
"one-dark" => "dark".into(),
_ => id,
})
}
/// [`AgentSettings::defaults`], keeping only the text values of each agent's table.
fn agent_defaults<'de, D>(d: D) -> Result<BTreeMap<String, BTreeMap<String, String>>, D::Error>
where
@@ -257,8 +276,19 @@ pub enum WhileRunning {
Steer,
}
/// The approvals new agent tasks start with.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum NewTaskApprovals {
/// The agent's mode that approves everything without asking, when it has one.
#[default]
Bypass,
/// A mode in which the agent asks before edits and commands.
Ask,
}
/// Agents › MCP Server: agents use the IDE's built-in tools.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(default, rename_all = "camelCase")]
pub struct McpServerSettings {
/// Serve the tools on 127.0.0.1 while the IDE runs.
@@ -266,6 +296,35 @@ pub struct McpServerSettings {
/// Run the tools that change the project or start processes without asking first
/// (IDEA's brave mode). Read from the IDE's own settings only, never a project's.
pub run_without_confirmation: bool,
/// Finished agent tabs (`run_command`) the Run tool window shows; older ones are archived.
pub agent_tab_limit: u32,
/// Archived agent tabs kept; older ones are closed.
pub agent_archive_limit: u32,
/// Listen on [`Self::port`] in every run, rather than a free port each time, so HTTP clients and
/// port forwards keep working after a restart.
pub fixed_port: bool,
/// The port of [`Self::fixed_port`].
pub port: u16,
/// Keep the bearer token across runs (in the app data, readable by this user only) rather than
/// make a new one each run.
pub fixed_token: bool,
}
/// The default of [`McpServerSettings::port`].
pub const DEFAULT_MCP_PORT: u16 = 26371;
impl Default for McpServerSettings {
fn default() -> Self {
Self {
enabled: false,
run_without_confirmation: false,
agent_tab_limit: 1,
agent_archive_limit: 999,
fixed_port: true,
port: DEFAULT_MCP_PORT,
fixed_token: false,
}
}
}
/// Absent profiles retain automatic discovery for existing projects.
@@ -557,13 +616,19 @@ pub enum NavigationBarPosition {
pub struct AppearanceSettings {
/// A built-in theme id (`light`, `dark`, `dark-hc`, …), `system` (legacy
/// "Sync with OS"), or a plugin's theme: `plugin:<plugin-id>/<theme-id>`.
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub theme: String,
pub accent: String,
pub ui_font_size: f32,
pub locale: String,
/// Follow the OS light/dark mode with the preferred themes below.
pub sync_with_os: bool,
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub preferred_light_theme: String,
#[serde(deserialize_with = "theme_id")]
#[specta(type = String)]
pub preferred_dark_theme: String,
/// A plugin's file icon theme (`<plugin-id>/<theme-id>`); empty = the built-in icons.
pub file_icon_theme: String,
@@ -1002,11 +1067,28 @@ pub struct PythonSettings {
pub interpreter: String,
/// Activate the interpreter's virtualenv in new terminal sessions.
pub activate_virtualenv: bool,
/// Interpreters the user added to Settings › Python › All Interpreters (executable paths).
/// Detection contributes the rest; `hidden_interpreters` removes some of them.
pub interpreters: Vec<String>,
/// Interpreters the user removed from that list, as executable paths: detection does
/// not list them again.
pub hidden_interpreters: Vec<String>,
/// Hide interpreters that belong to another project (a virtualenv inside its folder).
pub hide_other_project_interpreters: bool,
/// Show each interpreter's path under its name on the interpreters page.
pub show_interpreter_paths: bool,
}
impl Default for PythonSettings {
fn default() -> Self {
Self { interpreter: String::new(), activate_virtualenv: true }
Self {
interpreter: String::new(),
activate_virtualenv: true,
interpreters: Vec::new(),
hidden_interpreters: Vec::new(),
hide_other_project_interpreters: false,
show_interpreter_paths: true,
}
}
}
@@ -1182,7 +1264,9 @@ pub struct MavenSettings {
pub runner_jre: String,
/// Project-relative root POMs removed from the Maven project model.
pub unlinked_projects: Vec<String>,
/// Project-relative POMs outside the root reactor added as independent top-level projects.
/// POMs added as independent top-level projects. Inside the opened project the path is
/// project-relative (`tools/pom.xml`); outside it, the canonical absolute path. Linking
/// does not edit either POM.
pub linked_projects: Vec<String>,
/// Maven home (or its `mvn` executable); empty = the project's `mvnw`, else `mvn` on `PATH`.
pub home: String,
@@ -1422,6 +1506,19 @@ impl Settings {
mod tests {
use super::*;
#[test]
fn removed_one_themes_read_as_islands_themes() {
let settings: Settings = serde_json::from_str(
r#"{"appearance":{"theme":"one-dark","preferredLightTheme":"one-light","preferredDarkTheme":"one-dark"}}"#,
)
.unwrap();
let a = settings.appearance;
assert_eq!((a.theme.as_str(), a.preferred_light_theme.as_str()), ("dark", "light"));
assert_eq!(a.preferred_dark_theme, "dark");
let kept: Settings = serde_json::from_str(r#"{"appearance":{"theme":"solarized-dark"}}"#).unwrap();
assert_eq!(kept.appearance.theme, "solarized-dark");
}
#[test]
fn window_close_settings_default_off_and_round_trip() {
for old in [r#"{}"#, r#"{"general":{"confirmExit":false}}"#] {
+2 -5
Переглянути файл
@@ -61,16 +61,13 @@ pub fn find_command(command: &str, search: &OsStr, root: &Path) -> Option<PathBu
fn is_executable(path: &Path) -> bool {
let Ok(metadata) = path.metadata() else { return false };
if !metadata.is_file() {
return false;
}
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
metadata.permissions().mode() & 0o111 != 0
metadata.is_file() && metadata.permissions().mode() & 0o111 != 0
}
#[cfg(not(unix))]
true
metadata.is_file()
}
/// [`search_path`]'s directories, in order and once each. Only absolute ones: a
+6 -1
Переглянути файл
@@ -30,8 +30,13 @@ ide-http.workspace = true
reqwest = { workspace = true, features = ["query"] }
rustls.workspace = true
uuid.workspace = true
base64.workspace = true
flate2 = "1"
rsa = { version = "0.10.0-rc.18", features = ["sha2", "pkcs5"] }
zeroize = "1"
open = "5"
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "net"] }
tempfile.workspace = true
ide-ssh = { workspace = true, features = ["testing"] }
+188
Переглянути файл
@@ -0,0 +1,188 @@
//! The introspected catalog of a data source, kept on disk: the schemas, tables and views of
//! its databases, so completion knows them from the start of the next run without reading the
//! server again. Names only — no data, no credentials. One file per project and data source in
//! the app data directory.
use crate::TableInfo;
use ide_core::{IdeError, IdeResult};
use serde::{Deserialize, Serialize};
use specta::Type;
use std::path::{Path, PathBuf};
/// A schema of a database and its tables and views.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbCatalogSchema")]
pub struct CatalogSchema {
pub name: String,
pub tables: Vec<TableInfo>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbCatalogDatabase")]
pub struct CatalogDatabase {
pub name: String,
/// When this database was read, in milliseconds since the Unix epoch.
pub read_at: f64,
pub schemas: Vec<CatalogSchema>,
}
/// What introspection has read of a data source.
#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", default)]
#[serde(rename = "DbSourceCatalog")]
pub struct SourceCatalog {
pub databases: Vec<CatalogDatabase>,
}
pub(crate) struct CatalogStore {
dir: PathBuf,
project: String,
/// One writer at a time: an update reads the file, changes a database and writes it back.
writing: tokio::sync::Mutex<()>,
}
fn now_ms() -> f64 {
std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0.0, |d| d.as_millis() as f64)
}
impl CatalogStore {
pub(crate) fn new(dir: PathBuf, root: &Path) -> Self {
Self { dir, project: root.to_string_lossy().into_owned(), writing: tokio::sync::Mutex::new(()) }
}
fn file(&self, id: &str) -> PathBuf {
let name = blake3::hash(format!("{}\0{id}", self.project).as_bytes()).to_hex();
self.dir.join(format!("{}.json", &name.as_str()[..32]))
}
fn read(file: &Path) -> SourceCatalog {
std::fs::read(file).ok().and_then(|bytes| serde_json::from_slice(&bytes).ok()).unwrap_or_default()
}
fn write(file: &Path, catalog: &SourceCatalog) -> IdeResult<()> {
if let Some(dir) = file.parent() {
std::fs::create_dir_all(dir)?;
}
// Written beside the file and moved over it, so a crash leaves the old catalog whole.
let partial = file.with_extension("json.part");
let bytes = serde_json::to_vec(catalog).map_err(|e| IdeError::internal(e.to_string()))?;
std::fs::write(&partial, bytes)?;
std::fs::rename(&partial, file)?;
Ok(())
}
async fn blocking<T: Send + 'static>(task: impl FnOnce() -> T + Send + 'static) -> IdeResult<T> {
tokio::task::spawn_blocking(task).await.map_err(|e| IdeError::internal(e.to_string()))
}
/// The saved catalog of data source `id` (`None`: never introspected).
pub(crate) async fn load(&self, id: &str) -> IdeResult<Option<SourceCatalog>> {
let file = self.file(id);
Self::blocking(move || file.exists().then(|| Self::read(&file))).await
}
/// Saves what was just read of `database`, replacing what was known of it.
pub(crate) async fn put(&self, id: &str, database: &str, schemas: Vec<CatalogSchema>) -> IdeResult<()> {
let _writing = self.writing.lock().await;
let (file, name) = (self.file(id), database.to_owned());
Self::blocking(move || {
let mut catalog = Self::read(&file);
catalog.databases.retain(|d| d.name != name);
catalog.databases.push(CatalogDatabase { name, read_at: now_ms(), schemas });
catalog.databases.sort_by(|a, b| a.name.cmp(&b.name));
Self::write(&file, &catalog)
})
.await?
}
/// Forgets the databases not in `keep` (dropped on the server, or no longer shown).
pub(crate) async fn retain(&self, id: &str, keep: Vec<String>) -> IdeResult<()> {
let _writing = self.writing.lock().await;
let file = self.file(id);
Self::blocking(move || {
if !file.exists() {
return Ok(());
}
let mut catalog = Self::read(&file);
let before = catalog.databases.len();
catalog.databases.retain(|d| keep.contains(&d.name));
if catalog.databases.len() == before { Ok(()) } else { Self::write(&file, &catalog) }
})
.await?
}
/// Deletes the saved catalog of data source `id` (the data source was removed).
pub(crate) fn forget(&self, id: &str) {
let _ = std::fs::remove_file(self.file(id));
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::DbTableKind;
fn schema(name: &str, tables: &[&str]) -> CatalogSchema {
CatalogSchema {
name: name.into(),
tables: tables.iter().map(|t| TableInfo { name: (*t).into(), kind: DbTableKind::Table }).collect(),
}
}
#[tokio::test]
async fn a_catalog_survives_a_restart_per_project_and_data_source() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/shop"));
assert_eq!(store.load("sf").await.unwrap(), None);
store.put("sf", "RAW", vec![schema("PUBLIC", &["EVENTS"])]).await.unwrap();
store
.put("sf", "ANALYTICS", vec![schema("PUBLIC", &["ORDERS", "USERS"]), schema("STAGING", &[])])
.await
.unwrap();
// A new store over the same directory: what the next run of the app sees.
let reopened = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/shop"));
let saved = reopened.load("sf").await.unwrap().unwrap();
assert_eq!(saved.databases.iter().map(|d| d.name.as_str()).collect::<Vec<_>>(), ["ANALYTICS", "RAW"]);
assert_eq!(saved.databases[0].schemas, vec![schema("PUBLIC", &["ORDERS", "USERS"]), schema("STAGING", &[])]);
assert!(saved.databases.iter().all(|d| d.read_at > 0.0));
// Another data source and another project keep their own.
assert_eq!(reopened.load("other").await.unwrap(), None);
let elsewhere = CatalogStore::new(dir.path().join("db-catalogs"), Path::new("/projects/blog"));
assert_eq!(elsewhere.load("sf").await.unwrap(), None);
}
#[tokio::test]
async fn rereading_replaces_a_database_and_dropped_ones_are_forgotten() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().to_path_buf(), Path::new("/p"));
store.put("sf", "RAW", vec![schema("PUBLIC", &["OLD"])]).await.unwrap();
store.put("sf", "GONE", vec![schema("PUBLIC", &["T"])]).await.unwrap();
store.put("sf", "RAW", vec![schema("PUBLIC", &["NEW"])]).await.unwrap();
store.retain("sf", vec!["RAW".into()]).await.unwrap();
let saved = store.load("sf").await.unwrap().unwrap();
assert_eq!(saved.databases.len(), 1);
assert_eq!(saved.databases[0].schemas, vec![schema("PUBLIC", &["NEW"])]);
// Nothing half-written is left beside the file.
let files: Vec<_> = std::fs::read_dir(dir.path()).unwrap().map(|f| f.unwrap().file_name()).collect();
assert_eq!(files.len(), 1);
store.forget("sf");
assert_eq!(store.load("sf").await.unwrap(), None);
store.retain("sf", vec![]).await.unwrap();
assert_eq!(store.load("sf").await.unwrap(), None);
}
#[tokio::test]
async fn a_damaged_file_reads_as_an_empty_catalog() {
let dir = tempfile::tempdir().unwrap();
let store = CatalogStore::new(dir.path().to_path_buf(), Path::new("/p"));
std::fs::write(store.file("sf"), b"{ not json").unwrap();
assert_eq!(store.load("sf").await.unwrap(), Some(SourceCatalog::default()));
store.put("sf", "RAW", vec![schema("PUBLIC", &["T"])]).await.unwrap();
assert_eq!(store.load("sf").await.unwrap().unwrap().databases.len(), 1);
}
}
+100 -11
Переглянути файл
@@ -238,6 +238,13 @@ pub enum Pool {
SqlServer(Bb8Pool<MssqlConnectionManager>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(crate::oracle::OraclePool),
Snowflake(crate::snowflake::SnowflakePool),
}
/// Some servers and proxies (e.g. Aliyun RDS for MySQL 5.6) advertise `CLIENT_DEPRECATE_EOF` but
/// still end result sets with legacy EOF packets, which sqlx then fails to read as OK packets.
fn legacy_eof(e: &sqlx::Error) -> bool {
matches!(e, sqlx::Error::Protocol(msg) if msg.starts_with("OK_Packet too short"))
}
pub(crate) fn db_err(e: sqlx::Error) -> IdeError {
@@ -283,6 +290,16 @@ fn mssql_pool_err(e: bb8::RunError<tiberius::error::Error>) -> IdeError {
}
}
/// Tiberius requires TLS when a TLS backend is enabled. `trust_cert` only skips
/// certificate checks, so Disable has to turn encryption off.
fn apply_sqlserver_ssl(config: &mut tiberius::Config, ssl: DbSslMode) {
match ssl {
DbSslMode::Disable => config.encryption(tiberius::EncryptionLevel::NotSupported),
DbSslMode::Prefer => config.trust_cert(),
DbSslMode::Require | DbSslMode::VerifyCa | DbSslMode::VerifyFull => {}
}
}
const DAYS_1900_01_01: i64 = 693_595;
/// Formats a proleptic Gregorian date given days since `0001-01-01`.
@@ -693,6 +710,9 @@ impl Setup {
if source.driver == DbDriver::Clickhouse && !o.time_zone.trim().is_empty() {
s.params.push(("session_timezone".into(), o.time_zone.trim().into()));
}
if source.driver == DbDriver::Snowflake && !o.time_zone.trim().is_empty() {
s.params.push(("TIMEZONE".into(), o.time_zone.trim().into()));
}
let script = o.startup_script.trim();
if !script.is_empty() {
s.statements.extend(crate::split::split(script, source.driver).into_iter().map(|st| st.text));
@@ -710,6 +730,7 @@ impl Pool {
Pool::SqlServer(_) => DbDriver::SqlServer,
Pool::Clickhouse(_) => DbDriver::Clickhouse,
Pool::Oracle(_) => DbDriver::Oracle,
Pool::Snowflake(_) => DbDriver::Snowflake,
}
}
@@ -731,6 +752,9 @@ impl Pool {
if source.driver == DbDriver::Oracle {
return crate::oracle::OraclePool::connect(source, password, &setup).await.map(Pool::Oracle);
}
if source.driver == DbDriver::Snowflake {
return crate::snowflake::SnowflakePool::connect(source, password, setup, root).await.map(Pool::Snowflake);
}
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let init = Arc::new(setup.statements);
// Runs the session statements (startup script…) on every new connection.
@@ -855,9 +879,16 @@ impl Pool {
if !tz.is_empty() {
opts = opts.timezone(Some(tz.to_owned()));
}
let manager = MysqlManager { options: opts, init: init.clone(), timeout };
let mut manager = MysqlManager { options: opts, init: init.clone(), timeout };
// Open once directly so authentication failures retain their database error.
let first = manager.connect().await.map_err(db_err)?;
let first = match manager.connect().await {
Err(e) if legacy_eof(&e) => {
manager.options = manager.options.clone().deprecate_eof(false);
manager.connect().await
}
first => first,
}
.map_err(db_err)?;
let pool = Bb8Pool::builder().max_size(MAX).connection_timeout(timeout).build_unchecked(manager);
let _ = pool.add(first);
Ok(Pool::Mysql(ServerPool { inner: pool, closed: AtomicBool::new(false) }))
@@ -876,9 +907,7 @@ impl Pool {
if !source.database.is_empty() {
config.database(&source.database);
}
if matches!(source.ssl, DbSslMode::Disable | DbSslMode::Prefer) {
config.trust_cert();
}
apply_sqlserver_ssl(&mut config, source.ssl);
let manager = MssqlConnectionManager(config);
let pool = bb8::Pool::builder()
.max_size(MAX)
@@ -891,7 +920,7 @@ impl Pool {
Ok(Pool::SqlServer(pool))
}
DbDriver::Clickhouse => unreachable!(),
DbDriver::Oracle => unreachable!(),
DbDriver::Oracle | DbDriver::Snowflake => unreachable!(),
}
}
@@ -899,6 +928,7 @@ impl Pool {
match self {
Pool::SqlServer(_) | Pool::Oracle(_) => {}
Pool::Clickhouse(p) => p.close(),
Pool::Snowflake(p) => p.close().await,
Pool::Sqlite(p) => p.close().await,
Pool::Postgres(p) => p.closed.store(true, Ordering::Relaxed),
Pool::Mysql(p) => p.closed.store(true, Ordering::Relaxed),
@@ -910,6 +940,7 @@ impl Pool {
Pool::SqlServer(_) => false,
Pool::Oracle(p) => p.is_closed(),
Pool::Clickhouse(p) => p.is_closed(),
Pool::Snowflake(p) => p.is_closed(),
Pool::Sqlite(p) => p.is_closed(),
Pool::Postgres(p) => p.closed.load(Ordering::Relaxed),
Pool::Mysql(p) => p.closed.load(Ordering::Relaxed),
@@ -921,6 +952,9 @@ impl Pool {
if let Pool::Clickhouse(p) = self {
return Ok(p.execute(sql, max, &CancellationToken::new()).await?.rows);
}
if let Pool::Snowflake(p) = self {
return Ok(p.metadata().execute(sql, max, &CancellationToken::new()).await?.rows);
}
let mut conn = Held::acquire(self).await?;
conn.apply_context(None).await?;
let fetched = conn.fetch_rows(sql, max).await;
@@ -945,6 +979,9 @@ impl Pool {
if let Pool::Clickhouse(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
if let Pool::Snowflake(p) = self {
return p.execute_from(sql, skip, max_rows, cancel).await;
}
let mut conn = Held::acquire(self).await?;
let r = match conn.apply_context(None).await {
Ok(()) => conn.execute_from(self, sql, skip, max_rows, cancel).await,
@@ -962,6 +999,7 @@ impl Pool {
pub async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
match self {
Pool::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Pool::Snowflake(p) => p.transaction(statements).await,
Pool::SqlServer(_) => {
let mut held = Held::acquire(self).await?;
held.apply_context(None).await?;
@@ -1043,6 +1081,8 @@ pub enum Held {
SqlServer(Box<bb8::PooledConnection<'static, MssqlConnectionManager>>),
Clickhouse(crate::clickhouse::ClickhousePool),
Oracle(Box<crate::oracle::OracleConnection>),
/// The shared session, and the schema this holder's statements run in.
Snowflake(crate::snowflake::SnowflakePool, Option<String>),
}
impl Held {
@@ -1059,6 +1099,12 @@ impl Held {
Held::Clickhouse(p.clone())
}
Pool::Oracle(p) => Held::Oracle(Box::new(p.get().await?)),
Pool::Snowflake(p) => {
if p.is_closed() {
return Err(IdeError::new(ErrorKind::Io, "Snowflake connection is closed"));
}
Held::Snowflake(p.clone(), None)
}
})
}
@@ -1103,6 +1149,11 @@ impl Held {
}
pub async fn apply_context(&mut self, schema: Option<&str>) -> IdeResult<()> {
if let Held::Snowflake(_, wanted) = self {
// Applied with each statement, while it holds the shared session.
*wanted = schema.filter(|s| !s.is_empty()).map(str::to_owned);
return Ok(());
}
let driver = match self {
Held::Postgres(_) => DbDriver::Postgres,
Held::Mysql(_) => DbDriver::Mysql,
@@ -1158,7 +1209,7 @@ impl Held {
Held::Mysql(c) => fetch!(&mut ****c, sql, max, Cells::Text),
Held::SqlServer(c) => mssql_fetch(c, sql, 0, max).await,
Held::Oracle(c) => crate::oracle::run(c, sql, 0, max, Cells::Text, &CancellationToken::new()).await,
Held::Clickhouse(_) => unreachable!(),
Held::Clickhouse(_) | Held::Snowflake(..) => unreachable!(),
};
if result.is_ok() {
self.after_statement(sql);
@@ -1180,6 +1231,12 @@ impl Held {
match self {
Held::SqlServer(c) => mssql_command(c, sql).await,
Held::Clickhouse(_) => Err(crate::clickhouse::transaction_error()),
Held::Snowflake(c, schema) => {
if sql.trim().is_empty() {
return Ok(());
}
c.execute_in(schema.as_deref(), sql, 0, 0, &CancellationToken::new()).await.map(drop)
}
Held::Oracle(c) => {
let sql = sql.trim();
if sql.is_empty() {
@@ -1193,7 +1250,9 @@ impl Held {
Held::Sqlite(c) => sqlx::raw_sql(sql).execute(&mut **c).await.map(drop),
Held::Postgres(c) => sqlx::raw_sql(sql).execute(&mut ****c).await.map(drop),
Held::Mysql(c) => sqlx::raw_sql(sql).execute(&mut ****c).await.map(drop),
Held::SqlServer(_) | Held::Clickhouse(_) | Held::Oracle(_) => unreachable!(),
Held::SqlServer(_) | Held::Clickhouse(_) | Held::Oracle(_) | Held::Snowflake(..) => {
unreachable!()
}
}
.map_err(db_err)
}
@@ -1218,7 +1277,7 @@ impl Held {
Held::SqlServer(mut c) => {
c.0.take();
}
Held::Clickhouse(_) => {}
Held::Clickhouse(_) | Held::Snowflake(..) => {}
Held::Oracle(mut c) => c.discard_inner(),
}
}
@@ -1248,6 +1307,9 @@ impl Held {
if let Held::Clickhouse(c) = self {
return c.execute_from(sql, skip, max_rows, cancel).await;
}
if let Held::Snowflake(c, schema) = self {
return c.execute_in(schema.as_deref(), sql, skip, max_rows, cancel).await;
}
self.before_statement(sql);
let started = Instant::now();
// `$id`: the query naming the server process, `$kill`: how another connection stops it.
@@ -1292,7 +1354,7 @@ impl Held {
}
}
Held::Oracle(c) => crate::oracle::run(c, sql, skip, max_rows, Cells::Display, cancel).await?,
Held::Clickhouse(_) => unreachable!(),
Held::Clickhouse(_) | Held::Snowflake(..) => unreachable!(),
};
self.after_statement(sql);
let (columns, rows, affected, has_more, first) = fetched;
@@ -1315,7 +1377,7 @@ pub fn begin_statement(driver: DbDriver) -> &'static str {
DbDriver::Mysql => "START TRANSACTION",
DbDriver::SqlServer => "BEGIN TRAN",
DbDriver::Oracle => "", // opens implicitly with autocommit off
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse => "BEGIN",
DbDriver::Sqlite | DbDriver::Postgres | DbDriver::Clickhouse | DbDriver::Snowflake => "BEGIN",
}
}
@@ -1331,6 +1393,33 @@ mod tests {
use super::*;
use crate::source::{DbOptions, DbProperty};
fn sqlserver_debug(ssl: DbSslMode) -> String {
let mut config = tiberius::Config::new();
apply_sqlserver_ssl(&mut config, ssl);
format!("{config:?}")
}
#[test]
fn sqlserver_disable_does_not_require_tls() {
let debug = sqlserver_debug(DbSslMode::Disable);
assert!(debug.contains("encryption: NotSupported"), "{debug}");
assert!(debug.contains("bypass: false"), "{debug}");
}
#[test]
fn sqlserver_prefer_trusts_the_server_certificate() {
let debug = sqlserver_debug(DbSslMode::Prefer);
assert!(debug.contains("encryption: Required"), "{debug}");
assert!(debug.contains("bypass: true"), "{debug}");
}
#[test]
fn sqlserver_require_keeps_validated_tls() {
let debug = sqlserver_debug(DbSslMode::Require);
assert!(debug.contains("encryption: Required"), "{debug}");
assert!(debug.contains("bypass: false"), "{debug}");
}
#[test]
fn discarded_sqlserver_connections_are_broken() {
let manager = MssqlConnectionManager(tiberius::Config::new());
+219 -23
Переглянути файл
@@ -2,6 +2,7 @@
//! execution ([`conn`]), introspection, table data paging / editing ([`sql`])
//! and statement splitting ([`split`]), for SQLite, PostgreSQL and MySQL/MariaDB.
pub mod catalog;
mod clickhouse;
pub mod conn;
pub mod history;
@@ -9,17 +10,20 @@ mod oracle;
pub mod plan;
pub mod plugin;
pub mod secrets;
mod snowflake;
pub mod source;
pub mod split;
pub mod sql;
mod watch;
pub use watch::SourcesWatch;
pub use catalog::{CatalogDatabase, CatalogSchema, SourceCatalog};
pub use conn::{Held, Pool, QueryResult, ResultColumn};
pub use history::HistoryEntry;
pub use plan::{PlanNode, PlanProperty};
pub use source::{
DataSource, DbColor, DbDriver, DbIntrospection, DbOptions, DbProperty, DbSsh, DbSshAuth, DbSslMode, DbTxMode,
DataSource, DbColor, DbDriver, DbIntrospection, DbOptions, DbProperty, DbSnowflake, DbSnowflakeAuth, DbSsh,
DbSshAuth, DbSslMode, DbTxMode,
};
pub use sql::{CellValue, DbRowEdit, SortKey};
@@ -111,6 +115,21 @@ pub struct SchemaObjects {
pub routines: Vec<RoutineInfo>,
pub packages: Vec<PackageInfo>,
pub sequences: Vec<SequenceInfo>,
/// Objects of kinds only some databases have (Snowflake stages, streams, tasks…).
#[serde(default)]
pub others: Vec<SchemaObject>,
}
/// A schema object the tree lists by name under a folder of its kind.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSchemaObject")]
pub struct SchemaObject {
/// `stage`, `stream`, `task`, `pipe`, `fileFormat`.
pub kind: String,
pub name: String,
/// What the list says about it (a stage's type, a stream's table, a task's schedule).
pub detail: String,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
@@ -206,6 +225,18 @@ pub struct TablePage {
pub offset: f64,
}
/// What a Snowflake session computes with and acts as, and what it may switch to.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbSessionContext")]
pub struct SessionContext {
/// Empty: no warehouse is in use.
pub warehouse: String,
pub role: String,
pub warehouses: Vec<String>,
pub roles: Vec<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
#[serde(rename = "DbDatabases")]
@@ -230,6 +261,8 @@ pub struct DbManager {
/// Directory of the global data sources and their saved passwords ([`DbManager::with_global_sources`]).
global: Option<(PathBuf, SecretStore)>,
history: Arc<HistoryStore>,
/// What introspection read of each data source, kept between runs.
catalogs: catalog::CatalogStore,
/// Records executed statements in the history off the async threads, in order.
recorder: std::sync::mpsc::Sender<(String, String, f64)>,
/// Passwords entered this session (not saved, or not yet), by credential key.
@@ -341,6 +374,9 @@ impl DbManager {
/// `passwords` / `history` are the app-wide files of saved passwords
/// ([`SecretStore`]) and query history ([`HistoryStore`]).
pub fn new(root: PathBuf, passwords: PathBuf, history: PathBuf, ssh: Arc<SshManager>) -> Self {
// Beside the query history, in the app data directory.
let catalogs = history.parent().unwrap_or(&root).join("db-catalogs");
let catalogs = catalog::CatalogStore::new(catalogs, &root);
let history = Arc::new(HistoryStore::new(history, &root));
let (recorder, statements) = std::sync::mpsc::channel::<(String, String, f64)>();
{
@@ -360,6 +396,7 @@ impl DbManager {
saved_passwords: SecretStore::new(passwords, &root),
global: None,
history,
catalogs,
recorder,
root,
pools: Mutex::default(),
@@ -423,6 +460,10 @@ impl DbManager {
.ok_or_else(|| IdeError::not_found(format!("no data source {id}")))?
.resolved()?;
if let Some(db) = database {
// The configured schema belongs to the configured database.
if source.driver == DbDriver::Snowflake && !source.database.eq_ignore_ascii_case(db) {
source.options.default_schema.clear();
}
source.database = db.to_owned();
}
Ok(source)
@@ -457,9 +498,10 @@ impl DbManager {
match sources.iter().find(|s| s.id == o.id) {
Some(n) if *n == o => {}
gone_or_changed => {
let keys = [o.id.clone(), secrets::ssh_key(&o.id)];
let keys = [o.id.clone(), secrets::ssh_key(&o.id), Self::sso_credential(&o.id)];
match gone_or_changed {
None => {
self.catalogs.forget(&o.id);
for k in &keys {
self.session_passwords.lock().remove(k);
self.store_of(&o).delete(k);
@@ -613,7 +655,9 @@ impl DbManager {
ssh_secret: Option<String>,
shared: bool,
) -> IdeResult<(DataSource, Option<Arc<Tunnel>>)> {
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && source.driver != DbDriver::Sqlite) else {
// Snowflake is HTTPS to a named host: a tunnel's local end cannot pass its certificate check.
let tunneled = !matches!(source.driver, DbDriver::Sqlite | DbDriver::Snowflake);
let Some(ssh) = source.ssh.as_ref().filter(|s| s.enabled && tunneled) else {
return Ok((source.clone(), None));
};
let local = |t: &Tunnel| DataSource { host: "127.0.0.1".into(), port: Some(t.port()), ..source.clone() };
@@ -667,7 +711,10 @@ impl DbManager {
None => self.password(source).await,
};
let (target, tunnel) = self.endpoint(source, ssh_secret, false).await?;
let pool = Pool::connect(&target, password.as_deref(), Some(&self.root)).await.map_err(|e| {
self.restore_sso(source).await;
let connected = Pool::connect(&target, password.as_deref(), Some(&self.root)).await;
self.keep_sso(source).await;
let pool = connected.map_err(|e| {
through_tunnel(
e,
source,
@@ -680,10 +727,16 @@ impl DbManager {
DbDriver::Mysql => "SELECT CONCAT(@@version_comment, ' ', VERSION())",
DbDriver::SqlServer => "SELECT @@VERSION",
DbDriver::Oracle => "SELECT banner FROM v$version WHERE banner LIKE 'Oracle%' AND ROWNUM = 1",
DbDriver::Snowflake => "SELECT 'Snowflake ' || CURRENT_VERSION()",
};
let result = match &pool {
// On the session the login opened: metadata has its own, which would sign in again.
Pool::Snowflake(p) => p.execute(version, 1, &CancellationToken::new()).await.map(|r| r.rows),
_ => pool.rows(version, 1).await,
};
let result = pool.rows(version, 1).await;
pool.close().await;
let rows = if source.driver == DbDriver::Clickhouse { result? } else { result.unwrap_or_default() };
let strict = matches!(source.driver, DbDriver::Clickhouse | DbDriver::Snowflake);
let rows = if strict { result? } else { result.unwrap_or_default() };
let v = rows.into_iter().next().and_then(|r| r.into_iter().next().flatten());
Ok(v.unwrap_or_else(|| "Connected".into()))
}
@@ -710,7 +763,10 @@ impl DbManager {
let source = self.source(id)?;
let password = self.password(&source).await;
let (target, tunnel) = self.endpoint(&source, None, true).await?;
let pool = match Pool::connect(&target, password.as_deref(), Some(&self.root)).await {
self.restore_sso(&source).await;
let connected = Pool::connect(&target, password.as_deref(), Some(&self.root)).await;
self.keep_sso(&source).await;
let pool = match connected {
Ok(p) => Arc::new(p),
Err(e) => {
self.drop_unused_tunnel(source_id);
@@ -723,9 +779,10 @@ impl DbManager {
};
// Another caller may have connected meanwhile; keep theirs.
let kept = self.pools.lock().entry(id.to_owned()).or_insert_with(|| pool.clone()).clone();
if Arc::ptr_eq(&kept, &pool)
&& let Some(secs) = source.options.keep_alive_secs.filter(|s| *s > 0)
{
if !Arc::ptr_eq(&kept, &pool) {
// Ours lost the race: close it rather than leave its session open on the server.
pool.close().await;
} else if let Some(secs) = source.options.keep_alive_secs.filter(|s| *s > 0) {
keep_alive(&pool, Duration::from_secs(secs.into()));
}
Ok(kept)
@@ -775,6 +832,23 @@ impl DbManager {
let names = first(pool.rows(sql::sqlserver_databases(), MAX_META_ROWS).await?).collect();
return Ok(Databases { current, names });
}
if pool.driver() == DbDriver::Snowflake {
let current = pool
.rows("SELECT CURRENT_DATABASE()", 1)
.await?
.into_iter()
.next()
.and_then(|r| r.into_iter().next().flatten())
.unwrap_or_default();
// `SHOW` needs no running warehouse and lists what the role may see. Columns: created_on, name…
let names = pool
.rows("SHOW TERSE DATABASES", MAX_META_ROWS)
.await?
.into_iter()
.filter_map(|r| r.into_iter().nth(1).flatten())
.collect();
return Ok(Databases { current, names });
}
if pool.driver() != DbDriver::Postgres {
return Err(IdeError::invalid("this data source does not list databases"));
}
@@ -789,17 +863,95 @@ impl DbManager {
Ok(Databases { current, names })
}
/// The schemas of `database` of data source `id` with their tables and views (Snowflake):
/// what completion needs to follow `database.schema.table` without the tree being opened.
/// The result is also saved ([`DbManager::saved_catalog`]) for the next run.
pub async fn database_catalog(&self, id: &str, database: &str) -> IdeResult<Vec<CatalogSchema>> {
let id = split_target(id).0;
let schemas = match &*self.pool(id).await? {
Pool::Snowflake(p) => snowflake::meta::catalog(p, database).await?,
_ => return Err(IdeError::invalid("this data source does not list other databases")),
};
if let Err(e) = self.catalogs.put(id, database, schemas.clone()).await {
// Completion still has it for this run.
tracing::warn!("could not save the catalog of {database}: {}", e.message);
}
Ok(schemas)
}
/// What earlier introspection of data source `id` saved (`None`: never introspected).
/// Read from disk: the data source is not connected to.
pub async fn saved_catalog(&self, id: &str) -> IdeResult<Option<SourceCatalog>> {
self.catalogs.load(split_target(id).0).await
}
/// Drops the saved databases of data source `id` that are not in `databases` any more.
pub async fn retain_catalog(&self, id: &str, databases: Vec<String>) -> IdeResult<()> {
self.catalogs.retain(split_target(id).0, databases).await
}
/// Credential key of the single sign-on ID token of data source `id`.
fn sso_credential(id: &str) -> String {
format!("{id}#sso")
}
/// Hands the saved single sign-on token of `source` to the driver, so it signs in without the browser.
async fn restore_sso(&self, source: &DataSource) {
let Some(key) = snowflake::sso_key(source) else { return };
if snowflake::sso_token(&key).is_none()
&& let Some(saved) = self.store_of(source).get(&Self::sso_credential(&source.id))
{
snowflake::sso_seed(&key, &saved);
}
}
/// Keeps the token the last sign-in of `source` left in the credential store (or forgets a refused one).
async fn keep_sso(&self, source: &DataSource) {
let Some(key) = snowflake::sso_key(source) else { return };
if source.id.is_empty() {
return;
}
let credential = Self::sso_credential(&source.id);
let store = self.store_of(source);
let (now, saved) = (snowflake::sso_token(&key), store.get(&credential));
if now == saved {
return;
}
match now {
Some(token) => {
if let Err(e) = store.set(&credential, &token) {
tracing::warn!("could not save the single sign-on token: {}", e.message);
}
}
None => store.delete(&credential),
}
}
/// The warehouse and role of the session of data source `id` (Snowflake), and its choices.
pub async fn session_context(&self, id: &str) -> IdeResult<SessionContext> {
match &*self.pool(id).await? {
Pool::Snowflake(p) => snowflake::meta::session(p).await,
_ => Err(IdeError::invalid("this data source has no warehouse or role")),
}
}
/// Every schema (the Schemas tab picks the ones the tree shows); system schemas
/// only with [`DbIntrospection::show_system`].
pub async fn schemas(&self, id: &str) -> IdeResult<Vec<String>> {
let system = self.source(id)?.introspection.show_system;
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::schemas(p, system).await;
}
let rows = pool.rows(sql::schemas(pool.driver(), system), MAX_META_ROWS).await?;
Ok(rows.into_iter().filter_map(|r| r.into_iter().next().flatten()).collect())
}
pub async fn tables(&self, id: &str, schema: &str) -> IdeResult<Vec<TableInfo>> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::tables(p, schema).await;
}
let rows = pool.rows(&sql::tables(pool.driver(), schema), MAX_META_ROWS).await?;
Ok(rows
.into_iter()
@@ -813,6 +965,9 @@ impl DbManager {
pub async fn table_details(&self, id: &str, schema: &str, table: &str) -> IdeResult<TableDetails> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::table_details(p, schema, table).await;
}
let driver = pool.driver();
let columns = pool
.rows(&sql::columns(driver, schema, table), MAX_META_ROWS)
@@ -840,7 +995,7 @@ impl DbManager {
_ => indexes.push(IndexInfo { name, unique: flag(&unique), columns: vec![col] }),
}
}
let fk_rows = if driver == DbDriver::Clickhouse {
let fk_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
Vec::new()
} else {
pool.rows(&sql::table_foreign_keys(driver, schema, table), MAX_META_ROWS).await.unwrap_or_default()
@@ -857,7 +1012,7 @@ impl DbManager {
.collect();
}
foreign_keys.retain(|k| k.columns.len() == k.ref_columns.len());
let trigger_rows = if driver == DbDriver::Clickhouse {
let trigger_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
Vec::new()
} else {
pool.rows(&sql::triggers(driver, schema, table), MAX_META_ROWS).await.unwrap_or_else(|e| {
@@ -884,6 +1039,9 @@ impl DbManager {
/// The routines and sequences of `schema`.
pub async fn schema_objects(&self, id: &str, schema: &str) -> IdeResult<SchemaObjects> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
return snowflake::meta::schema_objects(p, schema).await;
}
let driver = pool.driver();
let mut out = SchemaObjects::default();
if let Some(q) = sql::routines(driver, schema) {
@@ -933,6 +1091,14 @@ impl DbManager {
/// The definition of a routine (every overload), package (spec and body), trigger or sequence.
pub async fn object_ddl(&self, id: &str, schema: &str, kind: DbObjectKind, name: &str) -> IdeResult<String> {
let pool = self.pool(id).await?;
if let Pool::Snowflake(p) = &*pool {
let defs = snowflake::meta::object_ddl(p, schema, kind, name).await?;
if defs.is_empty() {
return Err(IdeError::not_found(format!("{schema}.{name}")));
}
let defs: Vec<String> = defs.iter().map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).collect();
return Ok(defs.join("\n"));
}
if kind == DbObjectKind::Package {
let mut defs: Vec<String> = Vec::new();
for body in [false, true] {
@@ -1033,6 +1199,26 @@ impl DbManager {
.next();
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::Snowflake => {
let path = match &*pool {
Pool::Snowflake(p) => p.database_ident().map(|db| format!("{db}.")).unwrap_or_default(),
_ => String::new(),
};
let target = sql::quote_literal(driver, &format!("{path}{}", sql::qualified(driver, schema, name)));
let mut ddl = None;
// Views answer to their own object type on some accounts.
for kind in ["TABLE", "VIEW"] {
match pool.rows(&format!("SELECT GET_DDL('{kind}', {target})"), 1).await {
Ok(rows) => {
ddl = cells(rows, 0).into_iter().next();
break;
}
Err(_) if kind == "TABLE" => {}
Err(e) => return Err(e),
}
}
ddl.map(|d| format!("{};\n", d.trim_end().trim_end_matches(';'))).ok_or_else(missing)
}
DbDriver::Postgres => {
let rel = pool.rows(&sql::postgres_relation(schema, name), 1).await?.into_iter().next();
let (oid, kind) = match rel.as_deref() {
@@ -1081,7 +1267,14 @@ impl DbManager {
let pool = self.pool(id).await?;
let driver = pool.driver();
let mut tables: Vec<CompletionTable> = Vec::new();
for r in pool.rows(sql::completion_columns(driver), MAX_META_ROWS).await? {
let rows = match &*pool {
Pool::Snowflake(p) => {
tables = snowflake::meta::completion(p).await?;
Vec::new()
}
_ => pool.rows(sql::completion_columns(driver), MAX_META_ROWS).await?,
};
for r in rows {
let mut it = r.into_iter().map(Option::unwrap_or_default);
let mut next = || it.next().unwrap_or_default();
let (schema, name, kind) = (next(), next(), next());
@@ -1106,7 +1299,7 @@ impl DbManager {
}
}
// Foreign keys are a nicety (JOIN completion): a server that refuses the catalog query still completes.
let fk_rows = if driver == DbDriver::Clickhouse {
let fk_rows = if matches!(driver, DbDriver::Clickhouse | DbDriver::Snowflake) {
Vec::new()
} else {
pool.rows(sql::foreign_keys(driver), MAX_META_ROWS).await.unwrap_or_else(|e| {
@@ -1298,18 +1491,21 @@ impl DbManager {
let Some(session) = session else { return Ok(()) };
let mut held = session.lock().await;
let Some(mut conn) = held.take() else { return Ok(()) };
let result = async {
conn.command(if commit { "COMMIT" } else { "ROLLBACK" }).await?;
conn.restore_default().await
}
.await;
match result {
Ok(()) => Ok(()),
Err(e) => {
if let Err(e) = conn.command(if commit { "COMMIT" } else { "ROLLBACK" }).await {
// Discarding a Snowflake connection doesn't end its session, which the data source's
// other targets share: the transaction stays open, so it stays held to be ended again.
if matches!(conn, conn::Held::Snowflake(..)) {
*held = Some(conn);
} else {
conn.discard().await;
Err(e)
}
return Err(e);
}
if let Err(e) = conn.restore_default().await {
conn.discard().await;
return Err(e);
}
Ok(())
}
#[allow(clippy::too_many_arguments)]
+98
Переглянути файл
@@ -39,6 +39,7 @@ pub fn explain_statement(driver: DbDriver, statement: &str) -> String {
DbDriver::Mysql => format!("EXPLAIN {s}"),
DbDriver::SqlServer => format!("SET SHOWPLAN_TEXT ON; {s}"),
DbDriver::Oracle => format!("EXPLAIN PLAN FOR {s}"),
DbDriver::Snowflake => format!("EXPLAIN USING TABULAR {s}"),
}
}
@@ -77,6 +78,10 @@ pub fn parse(driver: DbDriver, result: &QueryResult) -> IdeResult<Vec<PlanNode>>
Ok(mysql(&names, &result.rows))
}
DbDriver::SqlServer => Ok(Vec::new()),
DbDriver::Snowflake => {
let names: Vec<&str> = result.columns.iter().map(|c| c.name.as_str()).collect();
Ok(snowflake(&names, &result.rows))
}
}
}
@@ -197,6 +202,42 @@ fn oracle(rows: &[Vec<Option<String>>]) -> Vec<PlanNode> {
out
}
/// Snowflake's `EXPLAIN USING TABULAR`: one row per operation with its parents' ids; the
/// first row totals the partitions and bytes the statement would scan.
fn snowflake(names: &[&str], rows: &[Vec<Option<String>>]) -> Vec<PlanNode> {
let column = |name: &str| names.iter().position(|n| n.eq_ignore_ascii_case(name));
let cell = |r: &[Option<String>], name: &str| column(name).and_then(|i| r.get(i).cloned().flatten());
let mut ids: Vec<String> = Vec::new();
let mut out = Vec::new();
for r in rows {
let operation = cell(r, "operation").unwrap_or_default();
let id = cell(r, "id").unwrap_or_default();
// `[0]`, `[1, 2]`: the first is the operation this one feeds.
let parent = cell(r, "parentOperators")
.and_then(|p| p.trim_matches(['[', ']']).split(',').next().map(|p| p.trim().to_owned()))
.and_then(|p| ids.iter().position(|x| *x == p))
.map(|i| i as u32);
let mut params = cell(r, "objects").unwrap_or_default();
if let Some(alias) = cell(r, "alias").filter(|a| !a.is_empty()) {
params = format!("{params} {alias}").trim().to_owned();
}
let mut properties = Vec::new();
for (label, name) in [
("Expressions", "expressions"),
("Partitions scanned", "partitionsAssigned"),
("Partitions total", "partitionsTotal"),
("Bytes scanned", "bytesAssigned"),
] {
if let Some(value) = cell(r, name).filter(|v| !v.is_empty()) {
properties.push(prop(label, value));
}
}
out.push(PlanNode { parent, operation, params, properties });
ids.push(id);
}
out
}
/// MySQL's tabular `EXPLAIN`: one row per table access, flat.
fn mysql(names: &[&str], rows: &[Vec<Option<String>>]) -> Vec<PlanNode> {
let col = |r: &[Option<String>], n: &str| {
@@ -377,4 +418,61 @@ mod tests {
assert_eq!((plan[1].operation.as_str(), plan[1].params.as_str()), ("Index Lookup", "table: c; key: PRIMARY;"));
assert!(plan[0].properties.iter().any(|p| p.name == "rows" && p.value == "20"));
}
#[test]
fn reads_snowflake_tabular_plans() {
let names = [
"step",
"id",
"parentOperators",
"operation",
"objects",
"alias",
"expressions",
"partitionsTotal",
"partitionsAssigned",
"bytesAssigned",
];
let row = |cells: [Option<&'static str>; 10]| cells.to_vec();
let r = result(
&names,
vec![
row([None, None, None, Some("GlobalStats"), None, None, None, Some("12"), Some("3"), Some("4096")]),
row([Some("1"), Some("0"), None, Some("Result"), None, None, Some("O.ID"), None, None, None]),
row([
Some("1"),
Some("1"),
Some("[0]"),
Some("Filter"),
None,
None,
Some("O.ID > 3"),
None,
None,
None,
]),
row([
Some("1"),
Some("2"),
Some("[1]"),
Some("TableScan"),
Some("DB.PUBLIC.ORDERS"),
Some("O"),
Some("ID"),
Some("12"),
Some("3"),
Some("4096"),
]),
],
);
let plan = parse(DbDriver::Snowflake, &r).unwrap();
assert_eq!(
plan.iter().map(|n| n.operation.as_str()).collect::<Vec<_>>(),
["GlobalStats", "Result", "Filter", "TableScan"]
);
assert_eq!(plan.iter().map(|n| n.parent).collect::<Vec<_>>(), [None, None, Some(1), Some(2)]);
assert_eq!(plan[3].params, "DB.PUBLIC.ORDERS O");
assert!(plan[3].properties.contains(&PlanProperty { name: "Partitions scanned".into(), value: "3".into() }));
assert!(plan[0].properties.contains(&PlanProperty { name: "Bytes scanned".into(), value: "4096".into() }));
}
}
+1 -1
Переглянути файл
@@ -171,7 +171,7 @@ impl DbManager {
}
/// Completes cleanup after removal from the file.
pub async fn forget_plugin_source(&self, id: &str) {
for key in [id.to_owned(), crate::secrets::ssh_key(id)] {
for key in [id.to_owned(), crate::secrets::ssh_key(id), format!("{id}#sso")] {
self.session_passwords.lock().remove(&key);
self.saved_passwords.delete(&key);
}
+121
Переглянути файл
@@ -0,0 +1,121 @@
//! Where a Snowflake account is reached, and the account name its login and JWT carry.
use crate::source::{DataSource, DbSslMode};
use ide_core::{IdeError, IdeResult};
use reqwest::Url;
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct Endpoint {
pub base: Url,
/// Upper-case account name: the first label of the account identifier.
pub account: String,
}
/// Reads the data source's host: an account identifier (`myorg-myaccount`,
/// `xy12345.eu-central-1`), a host name (`myorg-myaccount.snowflakecomputing.com`) or the
/// account URL.
pub(crate) fn endpoint(source: &DataSource) -> IdeResult<Endpoint> {
let raw = source.host.trim();
let raw = raw.strip_prefix("https://").or_else(|| raw.strip_prefix("http://")).unwrap_or(raw);
let raw = raw.split(['/', '?', '#']).next().unwrap_or_default().trim_end_matches('.');
let (host, port) = match raw.rsplit_once(':') {
Some((host, port)) if port.chars().all(|c| c.is_ascii_digit()) && !port.is_empty() => {
(host, port.parse::<u16>().ok())
}
_ => (raw, None),
};
let valid = |c: char| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.');
if host.is_empty() || host == "localhost" && source.ssl != DbSslMode::Disable || !host.chars().all(valid) {
return Err(IdeError::invalid(
"enter the Snowflake account identifier, for example myorg-myaccount or xy12345.eu-central-1",
));
}
let account = host.split('.').next().unwrap_or_default().to_ascii_uppercase();
let literal = host.parse::<std::net::IpAddr>().is_ok() || host == "localhost";
let full = if literal || host.to_ascii_lowercase().contains(".snowflakecomputing.") {
host.to_owned()
} else {
// Host names cannot hold underscores; Snowflake serves such accounts under the dashed name.
format!("{}.snowflakecomputing.com", host.replace('_', "-"))
};
let scheme = if source.ssl == DbSslMode::Disable { "http" } else { "https" };
let port = port.or(source.port).filter(|p| *p != 443 || scheme == "http");
let authority = match port {
Some(port) => format!("{full}:{port}"),
None => full,
};
let base = Url::parse(&format!("{scheme}://{authority}/"))
.map_err(|_| IdeError::invalid("invalid Snowflake account identifier"))?;
Ok(Endpoint { base, account })
}
#[cfg(test)]
mod tests {
use super::*;
use crate::source::DbDriver;
fn of(host: &str) -> Endpoint {
endpoint(&DataSource { driver: DbDriver::Snowflake, host: host.into(), ..Default::default() }).unwrap()
}
#[test]
fn account_identifiers_become_hosts() {
let e = of("myorg-myaccount");
assert_eq!(e.base.as_str(), "https://myorg-myaccount.snowflakecomputing.com/");
assert_eq!(e.account, "MYORG-MYACCOUNT");
let e = of("xy12345.eu-central-1");
assert_eq!(e.base.as_str(), "https://xy12345.eu-central-1.snowflakecomputing.com/");
assert_eq!(e.account, "XY12345");
let e = of("xy12345.us-east-2.aws");
assert_eq!(e.base.as_str(), "https://xy12345.us-east-2.aws.snowflakecomputing.com/");
assert_eq!(e.account, "XY12345");
}
#[test]
fn underscores_stay_in_the_account_name_but_not_in_the_host() {
let e = of("my_org-my_account");
assert_eq!(e.base.as_str(), "https://my-org-my-account.snowflakecomputing.com/");
assert_eq!(e.account, "MY_ORG-MY_ACCOUNT");
}
#[test]
fn hosts_and_urls_are_accepted() {
for host in [
"myorg-myaccount.snowflakecomputing.com",
"https://myorg-myaccount.snowflakecomputing.com",
"https://myorg-myaccount.snowflakecomputing.com/console/login?x=1",
"MyOrg-MyAccount.snowflakecomputing.com:443",
] {
let e = of(host);
assert_eq!(e.base.host_str().unwrap().to_ascii_lowercase(), "myorg-myaccount.snowflakecomputing.com");
assert_eq!(e.base.port(), None);
assert_eq!(e.account, "MYORG-MYACCOUNT");
}
let e = of("xy12345.eu-west-1.privatelink.snowflakecomputing.com");
assert_eq!(e.base.as_str(), "https://xy12345.eu-west-1.privatelink.snowflakecomputing.com/");
assert_eq!(e.account, "XY12345");
assert_eq!(of("acct.snowflakecomputing.cn").base.as_str(), "https://acct.snowflakecomputing.cn/");
}
#[test]
fn rejects_missing_and_malformed_identifiers() {
for host in ["", " ", "localhost", "my account", "user@host", "/path"] {
let source = DataSource { driver: DbDriver::Snowflake, host: host.into(), ..Default::default() };
let error = endpoint(&source).unwrap_err();
assert!(error.message.contains("account identifier"), "{host}: {}", error.message);
}
}
#[test]
fn plain_http_is_only_for_an_explicit_ssl_off() {
let source = DataSource {
driver: DbDriver::Snowflake,
host: "127.0.0.1".into(),
port: Some(8080),
ssl: DbSslMode::Disable,
..Default::default()
};
let e = endpoint(&source).unwrap();
assert_eq!(e.base.as_str(), "http://127.0.0.1:8080/");
}
}
+229
Переглянути файл
@@ -0,0 +1,229 @@
//! Single sign-on in the web browser (Snowflake's `externalbrowser` authenticator): Snowflake
//! names the identity provider's page, the browser signs in there and is sent back to a
//! listener on this computer with a one-time token, which the login request then presents
//! together with the proof key Snowflake issued for this attempt.
use super::client::{Api, Request};
use super::{APP, error, uuid};
use ide_core::{IdeError, IdeResult, error::ErrorKind};
use parking_lot::Mutex;
use serde_json::{Value, json};
use std::collections::HashMap;
use std::sync::LazyLock;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
const AUTHENTICATOR: &str = "session/authenticator-request";
/// How long the user has to finish signing in.
const WAIT: Duration = Duration::from_secs(120);
const DONE_PAGE: &str = "<!doctype html><html><head><meta charset=\"utf-8\"><title>OpenIDEA</title></head>\
<body style=\"font-family: sans-serif; margin: 3em\"><h3>Signed in to Snowflake</h3>\
<p>You can close this tab and return to OpenIDEA.</p></body></html>";
/// One browser sign-in at a time: connections opened together share its outcome through [`cached`].
pub(crate) static FLOW: LazyLock<tokio::sync::Mutex<()>> = LazyLock::new(|| tokio::sync::Mutex::new(()));
/// ID tokens Snowflake issued after a browser sign-in, by account address and user. They let
/// further sessions of this run open without the browser. Kept in memory only.
static ID_TOKENS: LazyLock<Mutex<HashMap<String, String>>> = LazyLock::new(|| Mutex::new(HashMap::new()));
pub(crate) fn cached(key: &str) -> Option<String> {
ID_TOKENS.lock().get(key).cloned()
}
pub(crate) fn remember(key: &str, token: &str) {
ID_TOKENS.lock().insert(key.to_owned(), token.to_owned());
}
pub(crate) fn forget(key: &str) {
ID_TOKENS.lock().remove(key);
}
/// What opens the sign-in page; tests put a stand-in for the browser here.
#[cfg(test)]
type Opener = dyn Fn(&str, u16) -> IdeResult<()> + Send + Sync;
#[cfg(test)]
pub(crate) static TEST_OPENER: LazyLock<Mutex<Option<Box<Opener>>>> = LazyLock::new(|| Mutex::new(None));
fn open(url: &str, port: u16) -> IdeResult<()> {
#[cfg(test)]
if let Some(opener) = TEST_OPENER.lock().as_ref() {
return opener(url, port);
}
let _ = port;
open::that_detached(url)
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: cannot open the web browser: {e}")))
}
/// The one-time token and proof key of a completed browser sign-in.
pub(crate) async fn authenticate(
api: &Api,
account: &str,
user: &str,
timeout: Duration,
) -> IdeResult<(String, String)> {
let listener = TcpListener::bind("127.0.0.1:0")
.await
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: cannot listen for the browser sign-in: {e}")))?;
let port = listener.local_addr().map_err(|e| IdeError::new(ErrorKind::Io, e.to_string()))?.port();
let body = json!({ "data": {
"ACCOUNT_NAME": account,
"LOGIN_NAME": user,
"AUTHENTICATOR": "EXTERNALBROWSER",
"BROWSER_MODE_REDIRECT_PORT": port.to_string(),
"CLIENT_APP_ID": APP,
"CLIENT_APP_VERSION": env!("CARGO_PKG_VERSION"),
} });
let envelope = api.send(Request::post(AUTHENTICATOR, body).query("requestId", uuid()).timeout(timeout)).await?;
if !envelope.success {
return Err(api.scrub.clean(error::server(envelope.code.as_deref(), envelope.message.as_deref())));
}
let text = |name: &str| envelope.data.get(name).and_then(Value::as_str).map(str::to_owned);
let (Some(url), Some(proof_key)) = (text("ssoUrl"), text("proofKey")) else {
return Err(IdeError::new(
ErrorKind::Protocol,
"Snowflake did not name a sign-in page; check that the account allows browser sign-in",
));
};
if !(url.starts_with("https://") || url.starts_with("http://")) {
return Err(IdeError::new(ErrorKind::Protocol, "Snowflake named a sign-in page that is not a web address"));
}
api.scrub.add(&proof_key);
open(&url, port)?;
let token = tokio::time::timeout(WAIT, callback(&listener)).await.map_err(|_| {
IdeError::new(ErrorKind::TimedOut, "Snowflake: the browser sign-in was not finished within 2 minutes")
})??;
api.scrub.add(&token);
Ok((token, proof_key))
}
/// Waits for the browser to come back with the token (other requests, like the tab's icon, are refused).
async fn callback(listener: &TcpListener) -> IdeResult<String> {
loop {
let (mut stream, _) = listener
.accept()
.await
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: browser sign-in failed: {e}")))?;
let Some(request) = read(&mut stream).await else { continue };
let line = request.head.lines().next().unwrap_or_default();
let mut parts = line.split_whitespace();
let (method, target) = (parts.next().unwrap_or_default(), parts.next().unwrap_or_default());
let origin = request.header("origin").unwrap_or("*").to_owned();
let cors = format!(
"Access-Control-Allow-Origin: {origin}\r\nVary: Origin\r\nAccess-Control-Allow-Methods: POST, GET\r\n\
Access-Control-Allow-Headers: {}\r\n",
request.header("access-control-request-headers").unwrap_or("content-type")
);
if method == "OPTIONS" {
respond(&mut stream, "200 OK", &cors, "").await;
continue;
}
let token = match method {
"GET" => target.split_once('?').and_then(|(_, query)| parameter(query, "token")),
"POST" => serde_json::from_str::<Value>(&request.body)
.ok()
.and_then(|v| v.get("token").and_then(Value::as_str).map(str::to_owned))
.or_else(|| parameter(&request.body, "token")),
_ => None,
};
match token.filter(|t| !t.is_empty()) {
Some(token) => {
respond(&mut stream, "200 OK", &format!("{cors}Content-Type: text/html; charset=utf-8\r\n"), DONE_PAGE)
.await;
return Ok(token);
}
None => respond(&mut stream, "404 Not Found", "", "").await,
}
}
}
struct Received {
head: String,
body: String,
}
impl Received {
fn header(&self, name: &str) -> Option<&str> {
self.head.lines().skip(1).find_map(|line| {
let (key, value) = line.split_once(':')?;
key.eq_ignore_ascii_case(name).then(|| value.trim())
})
}
}
async fn read(stream: &mut TcpStream) -> Option<Received> {
const LIMIT: usize = 256 * 1024;
let mut bytes = Vec::new();
let mut buffer = [0; 8192];
loop {
let count = tokio::time::timeout(Duration::from_secs(10), stream.read(&mut buffer)).await.ok()?.ok()?;
if count == 0 || bytes.len() > LIMIT {
return None;
}
bytes.extend_from_slice(&buffer[..count]);
if let Some(end) = bytes.windows(4).position(|w| w == b"\r\n\r\n") {
let head = String::from_utf8_lossy(&bytes[..end]).into_owned();
let length: usize = head
.lines()
.find_map(|line| {
line.to_ascii_lowercase().strip_prefix("content-length:").and_then(|v| v.trim().parse().ok())
})
.unwrap_or(0);
if length > LIMIT {
return None;
}
if bytes.len() >= end + 4 + length {
return Some(Received { head, body: String::from_utf8_lossy(&bytes[end + 4..]).into_owned() });
}
}
}
}
async fn respond(stream: &mut TcpStream, status: &str, headers: &str, body: &str) {
let response =
format!("HTTP/1.1 {status}\r\n{headers}Content-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = stream.write_all(response.as_bytes()).await;
let _ = stream.shutdown().await;
}
/// The value of `name` in a URL query or form body.
fn parameter(query: &str, name: &str) -> Option<String> {
query.split('&').find_map(|pair| {
let (key, value) = pair.split_once('=')?;
(key == name).then(|| decode(value))
})
}
fn decode(text: &str) -> String {
let bytes = text.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let hex = |i: usize| bytes.get(i).and_then(|&c| (c as char).to_digit(16));
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'%'
&& let (Some(high), Some(low)) = (hex(i + 1), hex(i + 2))
{
out.push((high * 16 + low) as u8);
i += 3;
continue;
}
out.push(if bytes[i] == b'+' { b' ' } else { bytes[i] });
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_the_token_from_a_query_or_form() {
assert_eq!(parameter("token=abc%2Bdef%3D%3D&x=1", "token").as_deref(), Some("abc+def=="));
assert_eq!(parameter("x=1&token=t", "token").as_deref(), Some("t"));
assert_eq!(parameter("tokens=1", "token"), None);
assert_eq!(parameter("", "token"), None);
}
}
+225
Переглянути файл
@@ -0,0 +1,225 @@
//! The HTTP side of the Snowflake session protocol: one JSON envelope per response.
//! Every error leaves through [`Scrubber::clean`], so no credential reaches a message or a log.
use ide_core::{IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use reqwest::{Client, Method, StatusCode, Url};
use serde::Deserialize;
use serde_json::Value;
use std::sync::Arc;
use std::time::Duration;
/// Replaces known secrets (passphrase, password, tokens) in text that leaves the driver.
#[derive(Clone, Default)]
pub(crate) struct Scrubber(Arc<RwLock<Vec<String>>>);
impl Scrubber {
pub(crate) fn add(&self, secret: &str) {
// Short values would blank out ordinary words.
if secret.len() >= 4 {
let mut secrets = self.0.write();
if !secrets.iter().any(|s| s == secret) {
secrets.push(secret.to_owned());
}
}
}
pub(crate) fn clean(&self, mut error: IdeError) -> IdeError {
for secret in self.0.read().iter() {
if error.message.contains(secret.as_str()) {
error.message = error.message.replace(secret.as_str(), "[redacted]");
}
}
error
}
}
/// A response body: `data` on success, `code` and `message` on failure.
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
pub(crate) struct Envelope {
pub success: bool,
#[serde(deserialize_with = "code")]
pub code: Option<String>,
pub message: Option<String>,
pub data: Value,
}
/// Codes arrive as strings, numbers or null.
fn code<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result<Option<String>, D::Error> {
Ok(match Value::deserialize(deserializer)? {
Value::String(s) if !s.is_empty() => Some(s),
Value::Number(n) => Some(n.to_string()),
_ => None,
})
}
impl Envelope {
pub(crate) fn has_code(&self, codes: &[&str]) -> bool {
self.code.as_deref().is_some_and(|code| codes.contains(&code))
}
}
#[derive(Clone)]
pub(crate) struct Api {
http: Client,
base: Url,
pub scrub: Scrubber,
}
pub(crate) struct Request<'a> {
pub method: Method,
pub path: &'a str,
pub query: Vec<(&'a str, String)>,
/// The session or master token.
pub token: Option<&'a str>,
pub body: Option<Value>,
pub timeout: Option<Duration>,
}
impl<'a> Request<'a> {
pub(crate) fn post(path: &'a str, body: Value) -> Self {
Self { method: Method::POST, path, query: Vec::new(), token: None, body: Some(body), timeout: None }
}
pub(crate) fn get(path: &'a str) -> Self {
Self { method: Method::GET, path, query: Vec::new(), token: None, body: None, timeout: None }
}
pub(crate) fn token(mut self, token: &'a str) -> Self {
self.token = Some(token);
self
}
pub(crate) fn query(mut self, name: &'a str, value: impl Into<String>) -> Self {
self.query.push((name, value.into()));
self
}
pub(crate) fn timeout(mut self, timeout: Duration) -> Self {
self.timeout = Some(timeout);
self
}
}
impl Api {
pub(crate) fn new(http: Client, base: Url) -> Self {
Self { http, base, scrub: Scrubber::default() }
}
/// The account's address, for keys that tell accounts apart.
pub(crate) fn address(&self) -> String {
format!("{}:{}", self.host(), self.base.port_or_known_default().unwrap_or_default())
}
fn host(&self) -> &str {
self.base.host_str().unwrap_or_default()
}
/// Sends `request` and reads the envelope, whatever it reports.
pub(crate) async fn send(&self, request: Request<'_>) -> IdeResult<Envelope> {
let url = self.base.join(request.path).map_err(|_| IdeError::internal("invalid Snowflake request path"))?;
let mut builder = self
.http
.request(request.method, url)
.query(&request.query)
.header(reqwest::header::ACCEPT, "application/snowflake");
if let Some(token) = request.token {
builder = builder.header(reqwest::header::AUTHORIZATION, format!("Snowflake Token=\"{token}\""));
}
if let Some(body) = &request.body {
builder = builder.header(reqwest::header::CONTENT_TYPE, "application/json").body(body.to_string());
}
if let Some(timeout) = request.timeout {
builder = builder.timeout(timeout);
}
let response = builder.send().await.map_err(|e| self.transport(e))?;
let status = response.status();
let body = response.bytes().await.map_err(|e| self.transport(e))?;
match serde_json::from_slice::<Envelope>(&body) {
Ok(envelope) if status.is_success() || envelope.code.is_some() || envelope.message.is_some() => {
Ok(envelope)
}
_ => Err(self.scrub.clean(self.status_error(status))),
}
}
/// Downloads a result chunk from cloud storage (no Snowflake credentials: the URL is presigned).
pub(crate) async fn chunk(&self, url: &str, headers: &[(String, String)]) -> IdeResult<Vec<u8>> {
let mut builder = self.http.get(url);
for (name, value) in headers {
builder = builder.header(name.as_str(), value.as_str());
}
let response = builder.send().await.map_err(|e| self.transport(e))?;
let status = response.status();
if !status.is_success() {
return Err(IdeError::new(
ErrorKind::Io,
format!("Snowflake: cannot download a result chunk (HTTP {})", status.as_u16()),
));
}
Ok(response.bytes().await.map_err(|e| self.transport(e))?.to_vec())
}
fn status_error(&self, status: StatusCode) -> IdeError {
let host = self.host();
match status {
StatusCode::FORBIDDEN | StatusCode::NOT_FOUND => IdeError::invalid(format!(
"Snowflake has no account at {host} (HTTP {}): check the account identifier",
status.as_u16()
)),
s if s.is_server_error() || s == StatusCode::TOO_MANY_REQUESTS => IdeError::new(
ErrorKind::Io,
format!("Snowflake is temporarily unavailable (HTTP {}); try again", s.as_u16()),
),
s => IdeError::new(ErrorKind::Protocol, format!("Snowflake: unexpected response (HTTP {})", s.as_u16())),
}
}
fn transport(&self, error: reqwest::Error) -> IdeError {
use std::error::Error;
let kind = if error.is_timeout() { ErrorKind::TimedOut } else { ErrorKind::Io };
let connect = error.is_connect();
let error = error.without_url();
let mut message = error.to_string();
let mut source = error.source();
while let Some(error) = source {
let detail = error.to_string();
if !message.contains(&detail) {
message.push_str(": ");
message.push_str(&detail);
}
source = error.source();
}
let hint = if connect { "; check the account identifier and the network connection" } else { "" };
self.scrub.clean(IdeError::new(kind, format!("Snowflake ({}): {message}{hint}", self.host())))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn envelopes_read_any_code_shape() {
let e: Envelope =
serde_json::from_str(r#"{"success":false,"code":"390100","message":"no","data":null}"#).unwrap();
assert_eq!(e.code.as_deref(), Some("390100"));
assert!(e.has_code(&["390100"]));
let e: Envelope = serde_json::from_str(r#"{"success":false,"code":390100}"#).unwrap();
assert_eq!(e.code.as_deref(), Some("390100"));
let e: Envelope = serde_json::from_str(r#"{"success":true,"code":null,"data":{"a":1}}"#).unwrap();
assert!(e.success && e.code.is_none());
assert_eq!(e.data["a"], 1);
}
#[test]
fn scrubbing_removes_every_known_secret() {
let scrub = Scrubber::default();
scrub.add("s3cret-passphrase");
scrub.add("eyJ.token.value");
scrub.add("ab");
let e = scrub.clean(IdeError::invalid("failed with s3cret-passphrase and eyJ.token.value, ab stays"));
assert_eq!(e.message, "failed with [redacted] and [redacted], ab stays");
}
}
+161
Переглянути файл
@@ -0,0 +1,161 @@
//! Every SQL command of Snowflake's reference
//! (<https://docs.snowflake.com/en/sql-reference/sql-all>) through the parts of the client a
//! statement passes: the script splitter, the read-only guard and the wire. The samples are in
//! `tests/fixtures/snowflake/commands.json`, one per documented command; the SQL assistance of
//! the UI reads the same file (`src/features/database/sql/snowflakeCommands.test.ts`).
//!
//! The samples exercise the client, not Snowflake's grammar: a server is needed to prove a
//! statement valid, which the ignored live test does for the commands that only read.
use super::tests::{CLOSED_OK, LOGIN_OK, bind, ok, result, serve, source};
use super::{first_word, is_write};
use crate::conn::Pool;
use crate::source::DbDriver;
use crate::split::split;
use ide_core::CancellationToken;
use serde::Deserialize;
use serde_json::json;
#[derive(Deserialize)]
pub(crate) struct Command {
pub command: String,
pub sql: String,
/// Runs on a read-only data source: it reads, or changes the session only.
pub reads: bool,
}
#[derive(Deserialize)]
struct Fixture {
commands: Vec<Command>,
}
pub(crate) fn commands() -> Vec<Command> {
let fixture: Fixture =
serde_json::from_str(include_str!("../../tests/fixtures/snowflake/commands.json")).expect("the fixture parses");
fixture.commands
}
/// `PUT` and `GET` move files on the client's side.
fn transfers(c: &Command) -> bool {
matches!(first_word(&c.sql).as_str(), "PUT" | "GET")
}
#[test]
fn the_fixture_covers_the_reference() {
let commands = commands();
assert!(commands.len() >= 786, "{} commands", commands.len());
let mut names: Vec<&str> = commands.iter().map(|c| c.command.as_str()).collect();
names.sort_unstable();
names.dedup();
assert_eq!(names.len(), commands.len(), "a command is listed twice");
for family in ["ALTER", "CREATE", "DESCRIBE", "DROP", "SHOW", "UNDROP", "USE", "GRANT", "REVOKE", "EXECUTE"] {
assert!(names.iter().any(|n| n.starts_with(family)), "no {family} command");
}
}
#[test]
fn every_command_is_one_statement() {
let commands = commands();
for c in &commands {
let alone = split(&c.sql, DbDriver::Snowflake);
assert_eq!(alone.len(), 1, "{}: split into {} statements", c.command, alone.len());
assert_eq!(alone[0].text, c.sql.trim(), "{}", c.command);
// Terminated, commented and followed by another statement.
let script = format!("-- {}; a comment\n{};\n// next; one\nselect 1;", c.command, c.sql);
let parts: Vec<String> = split(&script, DbDriver::Snowflake).into_iter().map(|s| s.text).collect();
assert_eq!(parts.len(), 2, "{}: {parts:?}", c.command);
assert!(parts[0].ends_with(c.sql.trim()), "{}: {:?}", c.command, parts[0]);
assert_eq!(parts[1].trim_start_matches("// next; one\n"), "select 1", "{}", c.command);
}
// The whole reference as one script.
let script = commands.iter().map(|c| format!("{};\n", c.sql)).collect::<String>();
let parts = split(&script, DbDriver::Snowflake);
assert_eq!(parts.len(), commands.len());
for (part, c) in parts.iter().zip(&commands) {
assert_eq!(part.text, c.sql.trim(), "{}", c.command);
}
}
#[test]
fn every_command_is_known_to_read_or_to_write() {
for c in commands() {
assert_eq!(!is_write(&c.sql), c.reads, "{}: {}", c.command, c.sql);
assert!(!first_word(&c.sql).is_empty(), "{}", c.command);
// Comments before a statement do not change what it is.
let commented = format!("/* note; */ -- one\n// two\n{}", c.sql);
assert_eq!(!is_write(&commented), c.reads, "{} after comments", c.command);
}
}
#[tokio::test]
async fn every_command_reaches_the_server_as_written_and_its_answer_is_read() {
let commands = commands();
let (listener, port) = bind().await;
let status = json!({ "success": true, "data": { "rowtype": [{ "name": "status", "type": "text" }],
"rowset": [["Statement executed successfully."]], "queryResultFormat": "json", "statementTypeId": 0x5000 } })
.to_string();
let rows = result(
json!([{ "name": "name", "type": "text" }, { "name": "n", "type": "fixed", "scale": 0 }]),
json!([["a", "1"], ["b", null]]),
);
let changed =
json!({ "success": true, "data": { "rowtype": [{ "name": "number of rows inserted", "type": "fixed" }],
"rowset": [["3"]], "queryResultFormat": "json", "statementTypeId": 0x3100, "total": 1 } })
.to_string();
// Login (which names the schema), then one answer per command.
let mut replies = vec![ok(LOGIN_OK)];
let mut entered = true;
for c in commands.iter().filter(|c| !transfers(c)) {
if !entered && first_word(&c.sql) != "USE" {
replies.push(ok(status.clone()));
}
// These leave the session wherever they went: its schema is entered again afterwards.
entered = !matches!(first_word(&c.sql).as_str(), "USE" | "CALL" | "EXECUTE");
let dml = matches!(first_word(&c.sql).as_str(), "INSERT" | "UPDATE" | "DELETE" | "MERGE");
replies.push(ok(if c.reads && first_word(&c.sql) != "USE" {
rows.clone()
} else if dml {
changed.clone()
} else {
status.clone()
}));
}
replies.push(ok(CLOSED_OK));
let server = serve(listener, replies);
let pool = Pool::connect(&source(port), None, None).await.unwrap();
let cancel = CancellationToken::new();
for c in &commands {
if transfers(c) {
// Refused before anything is sent: the files would not move.
let refused = pool.execute(&c.sql, 100, &cancel).await.unwrap_err();
assert!(refused.message.contains("Snowflake CLI"), "{}: {}", c.command, refused.message);
continue;
}
let answer = pool.execute(&c.sql, 100, &cancel).await;
let answer = answer.unwrap_or_else(|e| panic!("{}: {}", c.command, e.message));
let word = first_word(&c.sql);
if c.reads && word != "USE" {
assert_eq!(
answer.rows,
[[Some("a".to_owned()), Some("1".to_owned())], [Some("b".to_owned()), None]],
"{}",
c.command
);
assert_eq!(answer.affected, None, "{}", c.command);
} else if matches!(word.as_str(), "INSERT" | "UPDATE" | "DELETE" | "MERGE") {
assert_eq!(answer.affected, Some(3.0), "{}", c.command);
} else {
assert!(answer.columns.is_empty() && answer.rows.is_empty(), "{}: a command shows no grid", c.command);
}
}
pool.close().await;
let seen = server.await.unwrap();
let sent = seen[1..seen.len() - 1]
.iter()
.map(|r| r.json()["sqlText"].as_str().unwrap_or_default().to_owned())
.collect::<Vec<_>>();
let sent: Vec<&str> = sent.iter().map(String::as_str).collect();
let context = "USE SCHEMA \"ANALYTICS\".\"PUBLIC\"";
let written: Vec<&str> = sent.iter().copied().filter(|sql| *sql != context).collect();
assert_eq!(written, commands.iter().filter(|c| !transfers(c)).map(|c| c.sql.as_str()).collect::<Vec<_>>());
}
+98
Переглянути файл
@@ -0,0 +1,98 @@
//! Snowflake error codes as errors the UI can act on: `PermissionDenied` asks for the
//! data source's secret again, everything else shows its message.
use ide_core::{IdeError, error::ErrorKind};
/// The session token expired; the master token renews it.
pub(crate) const SESSION_EXPIRED: &str = "390112";
/// The session is gone (master token expired, session closed by the server): log in again.
pub(crate) const SESSION_GONE: &[&str] = &["390111", "390114", "390104"];
/// The statement still runs; its result is polled.
pub(crate) const IN_PROGRESS: &[&str] = &["333333", "333334"];
const KEY_HINT: &str = "Check that the public key of this private key is assigned to the user \
(ALTER USER … SET RSA_PUBLIC_KEY), and that the account identifier and user name are right.";
/// What to do about a failed login or statement, by Snowflake error code.
fn hint(code: &str, message: &str) -> Option<&'static str> {
Some(match code {
"390144" => KEY_HINT,
"394300" => "Snowflake found no such user: check the user name and the account identifier.",
"394302" => "The login token arrived too late: check this computer's clock.",
"394303" => "The login token had expired: check this computer's clock.",
"394304" => {
"The public key assigned to the user does not match this private key: compare the fingerprint below \
with RSA_PUBLIC_KEY_FP of DESC USER."
}
"394307" => "The account in the login token does not match: check the account identifier.",
"390189" => "The role is not granted to this user: change the role of the data source.",
"390201" => "Check the database, schema and warehouse of the data source, and that the role may use them.",
"000606" => "Set a warehouse in the data source, or run USE WAREHOUSE <name>.",
"390422" | "390420" => "A network policy of the account blocks this address.",
_ if message.contains("is not allowed to access Snowflake") => {
"A network policy of the account blocks this address."
}
_ => return None,
})
}
/// Whether the login was refused for reasons a different private key or account setting
/// fixes, so the key's fingerprint helps.
pub(crate) fn key_rejected(code: &str) -> bool {
code == "390144" || code.starts_with("3943")
}
fn kind(code: &str) -> ErrorKind {
match code {
// Wrong password or token; expired or revoked tokens.
"390100" | "390303" | "390318" | "394400" | "394401" | "394402" => ErrorKind::PermissionDenied,
"000604" | "000605" => ErrorKind::Cancelled,
"000630" => ErrorKind::TimedOut,
_ => ErrorKind::InvalidInput,
}
}
/// The error of a response that reports failure.
pub(crate) fn server(code: Option<&str>, message: Option<&str>) -> IdeError {
let code = code.unwrap_or_default();
let message = message.map(str::trim).filter(|m| !m.is_empty()).unwrap_or("the request failed");
let mut text =
if code.is_empty() { format!("Snowflake: {message}") } else { format!("Snowflake [{code}]: {message}") };
if let Some(hint) = hint(code, message) {
text.push('\n');
text.push_str(hint);
}
IdeError::new(kind(code), text)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn wrong_credentials_ask_again_and_other_failures_explain() {
let e = server(Some("390100"), Some("Incorrect username or password was specified."));
assert_eq!(e.kind, ErrorKind::PermissionDenied);
assert_eq!(e.message, "Snowflake [390100]: Incorrect username or password was specified.");
let e = server(Some("390144"), Some("JWT token is invalid. [abc]"));
assert_eq!(e.kind, ErrorKind::InvalidInput);
assert!(e.message.starts_with("Snowflake [390144]: JWT token is invalid. [abc]\nCheck that the public key"));
assert!(key_rejected("390144") && key_rejected("394304") && !key_rejected("390100"));
let e = server(Some("000606"), Some("No active warehouse selected in the current session."));
assert!(e.message.ends_with("run USE WAREHOUSE <name>."), "{}", e.message);
let e = server(Some("002003"), Some("SQL compilation error:\nObject 'T' does not exist or not authorized."));
assert_eq!(e.kind, ErrorKind::InvalidInput);
assert_eq!(
e.message,
"Snowflake [002003]: SQL compilation error:\nObject 'T' does not exist or not authorized."
);
}
#[test]
fn network_policy_and_bare_failures() {
let e = server(Some("390432"), Some("IP 203.0.113.9 is not allowed to access Snowflake."));
assert!(e.message.ends_with("A network policy of the account blocks this address."));
assert_eq!(server(None, None).message, "Snowflake: the request failed");
assert_eq!(server(Some("000604"), Some("SQL execution canceled")).kind, ErrorKind::Cancelled);
}
}
+265
Переглянути файл
@@ -0,0 +1,265 @@
//! Key-pair authentication: reads the user's RSA private key file and signs the login JWT.
//! Parsing, decryption and signing are RustCrypto's (`rsa`, `pkcs8`, `pkcs5`).
use base64::Engine;
use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD};
use ide_core::{IdeError, IdeResult, error::ErrorKind};
use rsa::RsaPrivateKey;
use rsa::pkcs1::DecodeRsaPrivateKey;
use rsa::pkcs1v15::SigningKey;
use rsa::pkcs8::{EncodePublicKey, EncryptedPrivateKeyInfoRef, PrivateKeyInfoRef, SecretDocument};
use rsa::sha2::{Digest, Sha256};
use rsa::signature::{SignatureEncoding, Signer};
use rsa::traits::PublicKeyParts;
use zeroize::Zeroizing;
/// Snowflake rejects a JWT that lives longer than an hour.
const LIFETIME_SECS: u64 = 59 * 60;
const MIN_BITS: usize = 2048;
const CONVERT: &str = "openssl pkcs8 -topk8 -v2 aes-256-cbc -in <key file> -out rsa_key.p8";
pub(crate) struct KeyPair {
key: RsaPrivateKey,
}
fn invalid(message: impl Into<String>) -> IdeError {
IdeError::invalid(message)
}
/// The UI answers `PermissionDenied` by asking for the secret, here the key's passphrase.
fn passphrase_error(message: &str) -> IdeError {
IdeError::new(ErrorKind::PermissionDenied, message)
}
/// Reads the private key file at `path` (`~/` is the home directory).
pub(crate) fn load(path: &str, passphrase: Option<&str>) -> IdeResult<KeyPair> {
let path = path.trim();
if path.is_empty() {
return Err(invalid("select the private key file of the Snowflake user"));
}
let file = ide_ssh::expand(path);
let bytes = Zeroizing::new(
std::fs::read(&file)
.map_err(|e| invalid(format!("cannot read the private key file {}: {e}", file.display())))?,
);
let Ok(text) = std::str::from_utf8(&bytes) else {
return Err(invalid(format!(
"{} is not a PEM file; convert a DER key with: openssl pkcs8 -inform DER -outform PEM -in <key file> -out rsa_key.p8",
file.display()
)));
};
parse(text, passphrase)
}
pub(crate) fn parse(pem: &str, passphrase: Option<&str>) -> IdeResult<KeyPair> {
let pem = pem.trim();
if pem.contains("Proc-Type: 4,ENCRYPTED") {
return Err(invalid(format!(
"the private key uses the legacy encrypted PEM format; convert it to PKCS#8 with: {CONVERT}"
)));
}
if pem.contains("BEGIN OPENSSH PRIVATE KEY") {
return Err(invalid(
"the private key is in OpenSSH format; convert it to PKCS#8 with: ssh-keygen -p -m PKCS8 -f <key file>",
));
}
let (label, document) = SecretDocument::from_pem(pem).map_err(|_| {
if pem.contains("PRIVATE KEY-----") {
invalid("the private key file is damaged")
} else {
invalid("the private key file holds no PEM private key (expected -----BEGIN PRIVATE KEY-----)")
}
})?;
let key = match label {
"PRIVATE KEY" => from_pkcs8(document.as_bytes(), "the private key file is damaged")?,
"ENCRYPTED PRIVATE KEY" => {
let Some(passphrase) = passphrase.filter(|p| !p.is_empty()) else {
return Err(passphrase_error("the private key is encrypted; enter its passphrase"));
};
let encrypted = EncryptedPrivateKeyInfoRef::try_from(document.as_bytes())
.map_err(|_| invalid("the encrypted private key file is damaged"))?;
let decrypted = encrypted.decrypt(passphrase).map_err(|error| match error {
rsa::pkcs8::Error::EncryptedPrivateKey(rsa::pkcs8::pkcs5::Error::DecryptFailed) => {
passphrase_error("incorrect passphrase for the private key")
}
rsa::pkcs8::Error::EncryptedPrivateKey(_) => invalid(format!(
"the private key is encrypted with an unsupported algorithm; re-encrypt it with: {CONVERT}"
)),
_ => invalid("the encrypted private key file is damaged"),
})?;
// A wrong passphrase can leave valid padding around bytes that are no key.
from_pkcs8(decrypted.as_bytes(), "incorrect passphrase for the private key").map_err(|error| {
if error.message.starts_with("incorrect passphrase") { passphrase_error(&error.message) } else { error }
})?
}
"RSA PRIVATE KEY" => RsaPrivateKey::from_pkcs1_der(document.as_bytes())
.map_err(|_| invalid("the private key file is damaged"))?,
"PUBLIC KEY" | "RSA PUBLIC KEY" => {
return Err(invalid("this file holds the public key; select the private key file"));
}
"EC PRIVATE KEY" => return Err(not_rsa()),
other => return Err(invalid(format!("unsupported private key format: {other}"))),
};
let bits = key.size() * 8;
if bits < MIN_BITS {
return Err(invalid(format!("the RSA key has {bits} bits; Snowflake requires at least {MIN_BITS}")));
}
Ok(KeyPair { key })
}
fn not_rsa() -> IdeError {
invalid("the private key is not an RSA key; Snowflake key-pair authentication needs an RSA key")
}
fn from_pkcs8(der: &[u8], malformed: &str) -> IdeResult<RsaPrivateKey> {
let info = PrivateKeyInfoRef::try_from(der).map_err(|_| invalid(malformed))?;
if info.algorithm.oid != rsa::pkcs1::ALGORITHM_OID {
return Err(not_rsa());
}
RsaPrivateKey::try_from(info).map_err(|_| invalid(malformed))
}
impl KeyPair {
/// `SHA256:<base64>` of the public key, as `DESC USER` shows it in `RSA_PUBLIC_KEY_FP`.
pub(crate) fn fingerprint(&self) -> IdeResult<String> {
let der = self
.key
.to_public_key()
.to_public_key_der()
.map_err(|e| IdeError::internal(format!("cannot encode the public key: {e}")))?;
Ok(format!("SHA256:{}", STANDARD.encode(Sha256::digest(der.as_bytes()))))
}
/// The login token for `user` of `account` (both as Snowflake stores them: upper case).
pub(crate) fn jwt(&self, account: &str, user: &str, now_secs: u64) -> IdeResult<String> {
let subject = format!("{}.{}", account.to_uppercase(), user.to_uppercase());
let header = URL_SAFE_NO_PAD.encode(br#"{"alg":"RS256","typ":"JWT"}"#);
let claims = serde_json::json!({
"iss": format!("{subject}.{}", self.fingerprint()?),
"sub": subject,
"iat": now_secs,
"exp": now_secs + LIFETIME_SECS,
});
let message = format!("{header}.{}", URL_SAFE_NO_PAD.encode(claims.to_string()));
let signature = SigningKey::<Sha256>::new(self.key.clone())
.try_sign(message.as_bytes())
.map_err(|e| IdeError::internal(format!("cannot sign the login token: {e}")))?;
Ok(format!("{message}.{}", URL_SAFE_NO_PAD.encode(signature.to_bytes())))
}
}
#[cfg(test)]
mod tests {
use super::*;
use rsa::pkcs1v15::{Signature, VerifyingKey};
use rsa::signature::Verifier;
const PLAIN: &str = include_str!("../../tests/fixtures/snowflake/rsa_key.p8");
const ENCRYPTED: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_encrypted.p8");
const SCRYPT: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_scrypt.p8");
const PKCS1: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_pkcs1.pem");
const LEGACY: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_legacy_encrypted.pem");
const SHORT: &str = include_str!("../../tests/fixtures/snowflake/rsa_key_1024.p8");
const EC: &str = include_str!("../../tests/fixtures/snowflake/ec_key.p8");
const PUBLIC: &str = include_str!("../../tests/fixtures/snowflake/rsa_key.pub");
/// Computed with OpenSSL (see the fixtures' README).
const FINGERPRINT: &str = "SHA256:ajjL6Bf6l1FIIyTcJli4i2AWPYDVJGQnSZr+9q9ClgI=";
const PASSPHRASE: &str = "test-passphrase";
fn error(pem: &str, passphrase: Option<&str>) -> IdeError {
match parse(pem, passphrase) {
Ok(_) => panic!("the key was accepted"),
Err(error) => error,
}
}
#[test]
fn every_supported_format_yields_the_same_key() {
for (pem, passphrase) in [
(PLAIN, None),
(PLAIN, Some("ignored")),
(ENCRYPTED, Some(PASSPHRASE)),
(SCRYPT, Some(PASSPHRASE)),
(PKCS1, None),
] {
assert_eq!(parse(pem, passphrase).unwrap().fingerprint().unwrap(), FINGERPRINT);
}
assert_eq!(parse(&format!("\n\n{PLAIN}\n\n"), None).unwrap().fingerprint().unwrap(), FINGERPRINT);
}
#[test]
fn an_encrypted_key_asks_for_its_passphrase() {
for passphrase in [None, Some("")] {
let e = error(ENCRYPTED, passphrase);
assert_eq!(e.kind, ErrorKind::PermissionDenied);
assert!(e.message.contains("enter its passphrase"), "{}", e.message);
}
}
#[test]
fn a_wrong_passphrase_is_named_and_asks_again() {
for pem in [ENCRYPTED, SCRYPT] {
for wrong in ["wrong", "test-passphrase ", "Test-Passphrase"] {
let e = error(pem, Some(wrong));
assert_eq!(e.kind, ErrorKind::PermissionDenied, "{}", e.message);
assert_eq!(e.message, "incorrect passphrase for the private key");
}
}
}
#[test]
fn unusable_files_say_what_is_wrong() {
let cases = [
(LEGACY, "legacy encrypted PEM"),
(SHORT, "1024 bits"),
(EC, "not an RSA key"),
(PUBLIC, "public key"),
("", "no PEM private key"),
("not a key", "no PEM private key"),
("-----BEGIN OPENSSH PRIVATE KEY-----\nAAAA\n-----END OPENSSH PRIVATE KEY-----", "OpenSSH format"),
("-----BEGIN PRIVATE KEY-----\nAAAA\n-----END PRIVATE KEY-----", "damaged"),
];
for (pem, expected) in cases {
let e = error(pem, Some(PASSPHRASE));
assert_eq!(e.kind, ErrorKind::InvalidInput, "{expected}: {}", e.message);
assert!(e.message.contains(expected), "{expected}: {}", e.message);
}
}
#[test]
fn missing_and_unreadable_files_are_reported_with_their_path() {
let e = load("", None).err().unwrap();
assert!(e.message.contains("select the private key file"));
let e = load("/nonexistent/openidea/rsa_key.p8", None).err().unwrap();
assert_eq!(e.kind, ErrorKind::InvalidInput);
assert!(e.message.contains("/nonexistent/openidea/rsa_key.p8"), "{}", e.message);
let dir = tempfile::tempdir().unwrap();
let der = dir.path().join("key.der");
std::fs::write(&der, [0x30, 0x82, 0xff, 0xfe, 0x00]).unwrap();
let e = load(der.to_str().unwrap(), None).err().unwrap();
assert!(e.message.contains("not a PEM file"), "{}", e.message);
let good = dir.path().join("rsa_key.p8");
std::fs::write(&good, ENCRYPTED).unwrap();
assert_eq!(load(good.to_str().unwrap(), Some(PASSPHRASE)).unwrap().fingerprint().unwrap(), FINGERPRINT);
}
#[test]
fn the_token_carries_snowflake_claims_and_a_valid_rs256_signature() {
let pair = parse(PLAIN, None).unwrap();
let token = pair.jwt("myorg-myaccount", "[email protected]", 1_700_000_000).unwrap();
let parts: Vec<&str> = token.split('.').collect();
assert_eq!(parts.len(), 3);
let decode = |part: &str| URL_SAFE_NO_PAD.decode(part).unwrap();
let header: serde_json::Value = serde_json::from_slice(&decode(parts[0])).unwrap();
assert_eq!(header, serde_json::json!({"alg": "RS256", "typ": "JWT"}));
let claims: serde_json::Value = serde_json::from_slice(&decode(parts[1])).unwrap();
assert_eq!(claims["sub"], "[email protected]");
assert_eq!(claims["iss"], format!("[email protected].{FINGERPRINT}"));
assert_eq!(claims["iat"], 1_700_000_000u64);
assert_eq!(claims["exp"], 1_700_000_000u64 + 59 * 60);
let verifier = VerifyingKey::<Sha256>::new(pair.key.to_public_key());
let signature = Signature::try_from(decode(parts[2]).as_slice()).unwrap();
verifier.verify(format!("{}.{}", parts[0], parts[1]).as_bytes(), &signature).unwrap();
assert!(verifier.verify(format!("{}.{}x", parts[0], parts[1]).as_bytes(), &signature).is_err());
}
}
+511
Переглянути файл
@@ -0,0 +1,511 @@
//! The Database tree's metadata, read with `SHOW` and `DESCRIBE`: they need no running
//! warehouse, list only what the session's role may see, and are asked one level at a time.
use super::SnowflakePool;
use crate::sql::{quote_ident, quote_literal};
use crate::{
ColumnInfo, CompletionColumn, CompletionTable, DbDriver, DbObjectKind, DbTableKind, ForeignKey, RoutineInfo,
SchemaObject, SchemaObjects, SequenceInfo, TableDetails, TableInfo,
};
use ide_core::{CancellationToken, IdeError, IdeResult};
use std::collections::HashMap;
/// `SHOW … LIMIT` takes at most this many rows; longer lists continue `FROM` the last name.
const PAGE: usize = 10_000;
const MAX_ROWS: usize = 100_000;
const DRIVER: DbDriver = DbDriver::Snowflake;
/// A `SHOW` / `DESCRIBE` result, read by column name (their column order varies by version).
struct Shown {
columns: Vec<String>,
rows: Vec<Vec<Option<String>>>,
}
impl Shown {
fn index(&self, name: &str) -> Option<usize> {
self.columns.iter().position(|c| c.eq_ignore_ascii_case(name))
}
fn values<'a>(&'a self, name: &str) -> impl Iterator<Item = Option<&'a str>> + 'a {
let index = self.index(name);
self.rows.iter().map(move |row| index.and_then(|i| row.get(i)).and_then(|v| v.as_deref()))
}
fn records(&self) -> impl Iterator<Item = Record<'_>> {
self.rows.iter().map(|row| Record { shown: self, row })
}
}
struct Record<'a> {
shown: &'a Shown,
row: &'a [Option<String>],
}
impl<'a> Record<'a> {
fn get(&self, name: &str) -> Option<&'a str> {
self.shown.index(name).and_then(|i| self.row.get(i)).and_then(|v| v.as_deref())
}
fn text(&self, name: &str) -> String {
self.get(name).unwrap_or_default().to_owned()
}
}
async fn show(pool: &SnowflakePool, sql: &str) -> IdeResult<Shown> {
let result = pool.metadata().execute(sql, MAX_ROWS, &CancellationToken::new()).await?;
Ok(Shown { columns: result.columns.into_iter().map(|c| c.name).collect(), rows: result.rows })
}
/// A `SHOW` that lists by name, page after page.
async fn show_all(pool: &SnowflakePool, sql: &str) -> IdeResult<Shown> {
let mut all = show(pool, &format!("{sql} LIMIT {PAGE}")).await?;
let mut last_page = all.rows.len();
while last_page >= PAGE && all.rows.len() < MAX_ROWS {
let Some(last) = all.values("name").last().flatten().map(str::to_owned) else { break };
let next = show(pool, &format!("{sql} LIMIT {PAGE} FROM {}", quote_literal(DRIVER, &last))).await?;
last_page = next.rows.len();
all.rows.extend(next.rows);
}
Ok(all)
}
/// ` IN DATABASE "DB"`, or nothing for the session's current database.
fn in_database(pool: &SnowflakePool) -> String {
pool.database_ident().map(|db| format!(" IN DATABASE {db}")).unwrap_or_default()
}
/// `"DB"."SCHEMA"` (the schema as the tree names it: exactly).
fn schema_path(pool: &SnowflakePool, schema: &str) -> String {
match pool.database_ident() {
Some(db) => format!("{db}.{}", quote_ident(DRIVER, schema)),
None => quote_ident(DRIVER, schema),
}
}
fn object_path(pool: &SnowflakePool, schema: &str, name: &str) -> String {
format!("{}.{}", schema_path(pool, schema), quote_ident(DRIVER, name))
}
pub(crate) async fn schemas(pool: &SnowflakePool, system: bool) -> IdeResult<Vec<String>> {
let shown = show_all(pool, &format!("SHOW TERSE SCHEMAS{}", in_database(pool))).await?;
Ok(shown
.values("name")
.flatten()
.filter(|name| system || *name != "INFORMATION_SCHEMA")
.map(str::to_owned)
.collect())
}
fn table_kind(kind: &str) -> DbTableKind {
let kind = kind.to_ascii_uppercase().replace('_', " ");
if kind == "MATERIALIZED VIEW" {
DbTableKind::MaterializedView
} else if kind.ends_with("VIEW") {
DbTableKind::View
} else {
DbTableKind::Table
}
}
/// Tables of every kind (dynamic, external, hybrid…) and views of `schema`.
pub(crate) async fn tables(pool: &SnowflakePool, schema: &str) -> IdeResult<Vec<TableInfo>> {
let shown = show_all(pool, &format!("SHOW TERSE OBJECTS IN SCHEMA {}", schema_path(pool, schema))).await?;
Ok(shown
.records()
.filter_map(|r| {
Some(TableInfo { name: r.get("name")?.to_owned(), kind: table_kind(r.get("kind").unwrap_or("TABLE")) })
})
.collect())
}
pub(crate) async fn table_details(pool: &SnowflakePool, schema: &str, table: &str) -> IdeResult<TableDetails> {
let path = object_path(pool, schema, table);
let described = show(pool, &format!("DESCRIBE TABLE {path}")).await?;
let yes = |v: Option<&str>| v.is_some_and(|v| v.eq_ignore_ascii_case("Y"));
let columns = described
.records()
.filter(|r| r.get("kind").is_none_or(|kind| kind.eq_ignore_ascii_case("COLUMN")))
.filter_map(|r| {
Some(ColumnInfo {
name: r.get("name")?.to_owned(),
data_type: r.text("type"),
nullable: yes(r.get("null?")),
default: r.get("default").filter(|d| !d.is_empty()).map(str::to_owned),
primary_key: yes(r.get("primary key")),
})
})
.collect();
// Views have no keys; a role without the privilege still sees the columns.
let foreign_keys = match show(pool, &format!("SHOW IMPORTED KEYS IN TABLE {path}")).await {
Ok(keys) => foreign_keys(&keys),
Err(_) => Vec::new(),
};
Ok(TableDetails { columns, indexes: Vec::new(), foreign_keys, triggers: Vec::new() })
}
fn foreign_keys(shown: &Shown) -> Vec<ForeignKey> {
let mut rows: Vec<Record<'_>> = shown.records().collect();
let sequence = |r: &Record<'_>| r.get("key_sequence").and_then(|n| n.parse::<u32>().ok()).unwrap_or(0);
rows.sort_by_key(|a| (a.text("fk_name"), sequence(a)));
let mut keys: Vec<(String, ForeignKey)> = Vec::new();
for r in rows {
let name = r.text("fk_name");
if !keys.last().is_some_and(|(last, _)| *last == name) {
keys.push((
name,
ForeignKey {
schema: r.text("fk_schema_name"),
table: r.text("fk_table_name"),
columns: Vec::new(),
ref_schema: r.text("pk_schema_name"),
ref_table: r.text("pk_table_name"),
ref_columns: Vec::new(),
},
));
}
let key = &mut keys.last_mut().expect("pushed above").1;
key.columns.push(r.text("fk_column_name"));
key.ref_columns.push(r.text("pk_column_name"));
}
keys.into_iter().map(|(_, key)| key).collect()
}
/// Rows of `SHOW TERSE OBJECTS` for database `db` (quoted; `None`: the session's), each with
/// its `schema_name`. One listing for the database when that answers in full; schema by schema
/// (as the tree lists them) when it is refused or is cut off at a page, since a page
/// continues by object name and names repeat across schemas.
async fn database_objects(pool: &SnowflakePool, db: Option<&str>, schemas: &[String]) -> IdeResult<Shown> {
let scope = db.map(|db| format!(" IN DATABASE {db}")).unwrap_or_default();
match show(pool, &format!("SHOW TERSE OBJECTS{scope} LIMIT {PAGE}")).await {
Ok(whole) if whole.rows.len() < PAGE && whole.index("schema_name").is_some() => return Ok(whole),
Ok(_) => {}
Err(e) => tracing::warn!("listing a whole database failed, listing its schemas instead: {}", e.message),
}
let mut all = Shown { columns: vec!["name".into(), "kind".into(), "schema_name".into()], rows: Vec::new() };
let (mut listed, mut refused) = (0, None);
for schema in schemas {
let path = match db {
Some(db) => format!("{db}.{}", quote_ident(DRIVER, schema)),
None => quote_ident(DRIVER, schema),
};
// A schema the role may not list: the others still complete.
let shown = match show_all(pool, &format!("SHOW TERSE OBJECTS IN SCHEMA {path}")).await {
Ok(shown) => shown,
Err(e) => {
refused.get_or_insert(e);
continue;
}
};
listed += 1;
let (Some(name), Some(kind)) = (shown.index("name"), shown.index("kind")) else { continue };
all.rows.extend(shown.rows.into_iter().map(|mut row| {
let kind = row.get_mut(kind).and_then(Option::take);
let name = row.get_mut(name).and_then(Option::take);
vec![name, kind, Some(schema.clone())]
}));
if all.rows.len() >= MAX_ROWS {
break;
}
}
// Nothing could be listed at all: that is a failure, not an empty database.
match refused {
Some(e) if listed == 0 => Err(e),
_ => Ok(all),
}
}
/// Every schema of `database` with its tables and views, for completion across databases.
pub(crate) async fn catalog(pool: &SnowflakePool, database: &str) -> IdeResult<Vec<crate::CatalogSchema>> {
let db = quote_ident(DRIVER, database);
let schemas = show_all(pool, &format!("SHOW TERSE SCHEMAS IN DATABASE {db}")).await?;
let names: Vec<String> =
schemas.values("name").flatten().filter(|name| *name != "INFORMATION_SCHEMA").map(str::to_owned).collect();
let mut out: Vec<crate::CatalogSchema> =
names.iter().map(|name| crate::CatalogSchema { name: name.clone(), tables: Vec::new() }).collect();
let at: HashMap<String, usize> = out.iter().enumerate().map(|(i, s)| (s.name.clone(), i)).collect();
let objects = database_objects(pool, Some(&db), &names).await?;
for r in objects.records() {
let (Some(&index), Some(name)) = (at.get(r.get("schema_name").unwrap_or_default()), r.get("name")) else {
continue;
};
out[index].tables.push(TableInfo { name: name.to_owned(), kind: table_kind(r.get("kind").unwrap_or("TABLE")) });
}
for schema in &mut out {
schema.tables.sort_by(|a, b| a.name.cmp(&b.name));
schema.tables.dedup_by(|a, b| a.name == b.name);
}
Ok(out)
}
/// The session's warehouse and role, and the ones it may switch to.
pub(crate) async fn session(pool: &SnowflakePool) -> IdeResult<crate::SessionContext> {
// The consoles' session: `USE WAREHOUSE` / `USE ROLE` there is what the pickers show.
let sql = "SELECT CURRENT_WAREHOUSE(), CURRENT_ROLE(), CURRENT_AVAILABLE_ROLES()";
let current = pool.execute(sql, 1, &CancellationToken::new()).await?.rows;
let cell = |i: usize| current.first().and_then(|r| r.get(i).cloned().flatten()).unwrap_or_default();
let mut roles: Vec<String> = serde_json::from_str(&cell(2)).unwrap_or_default();
roles.sort();
let warehouses = match show(pool, "SHOW WAREHOUSES").await {
Ok(shown) => shown.values("name").flatten().map(str::to_owned).collect(),
Err(_) => Vec::new(),
};
Ok(crate::SessionContext { warehouse: cell(0), role: cell(1), warehouses, roles })
}
/// `NAME(NUMBER, VARCHAR) RETURN NUMBER` as its argument types and result type.
fn signature(arguments: &str) -> (String, Option<String>) {
let inner = arguments.split_once('(').map_or(arguments, |(_, rest)| rest);
// The result type may hold parentheses of its own (`TABLE (…)`): split at the marker.
let (types, returns) = match inner.split_once(") RETURN ") {
Some((types, returns)) => (types, Some(returns.trim().to_owned())),
None => (inner.strip_suffix(')').unwrap_or(inner), None),
};
(types.trim().to_owned(), returns.filter(|r| !r.is_empty()))
}
async fn routines(
pool: &SnowflakePool,
schema: &str,
kind: DbObjectKind,
like: Option<&str>,
) -> IdeResult<Vec<RoutineInfo>> {
let what = if kind == DbObjectKind::Procedure { "PROCEDURES" } else { "USER FUNCTIONS" };
let like = like.map(|name| format!(" LIKE {}", quote_literal(DRIVER, &escape_like(name)))).unwrap_or_default();
let shown = show(pool, &format!("SHOW {what}{like} IN SCHEMA {}", schema_path(pool, schema))).await?;
Ok(shown
.records()
.filter(|r| !r.get("is_builtin").is_some_and(|b| b.eq_ignore_ascii_case("Y")))
.filter_map(|r| {
let (arguments, returns) = signature(r.get("arguments").unwrap_or_default());
Some(RoutineInfo {
name: r.get("name")?.to_owned(),
kind,
arguments,
returns: returns.filter(|_| kind == DbObjectKind::Function),
})
})
.collect())
}
/// `LIKE` patterns match `_` and `%` loosely; escape them to match the name itself.
fn escape_like(name: &str) -> String {
name.replace('\\', "\\\\").replace('_', "\\_").replace('%', "\\%")
}
/// Kinds of the tree's other folders, the `SHOW` that lists them, and the column that says most about one.
const OTHERS: &[(&str, &str, &str)] = &[
("stage", "STAGES", "type"),
("stream", "STREAMS", "table_name"),
("task", "TASKS", "schedule"),
("pipe", "PIPES", "definition"),
("fileFormat", "FILE FORMATS", "type"),
];
pub(crate) async fn schema_objects(pool: &SnowflakePool, schema: &str) -> IdeResult<SchemaObjects> {
let mut out = SchemaObjects::default();
// A role may see a schema's tables and none of these: each list stands alone.
for kind in [DbObjectKind::Function, DbObjectKind::Procedure] {
out.routines.extend(routines(pool, schema, kind, None).await.unwrap_or_default());
}
out.routines.sort_by(|a, b| (&a.name, &a.arguments).cmp(&(&b.name, &b.arguments)));
if let Ok(shown) = show(pool, &format!("SHOW SEQUENCES IN SCHEMA {}", schema_path(pool, schema))).await {
out.sequences = shown
.values("name")
.flatten()
.map(|name| SequenceInfo { name: name.to_owned(), data_type: "NUMBER".into() })
.collect();
}
for (kind, what, detail) in OTHERS {
let Ok(shown) = show(pool, &format!("SHOW {what} IN SCHEMA {}", schema_path(pool, schema))).await else {
continue;
};
out.others.extend(shown.records().filter_map(|r| {
let detail: String = r.text(detail).split_whitespace().collect::<Vec<_>>().join(" ");
Some(SchemaObject {
kind: (*kind).to_owned(),
name: r.get("name")?.to_owned(),
detail: detail.chars().take(120).collect(),
})
}));
}
Ok(out)
}
/// The definitions of a routine's overloads, or of a sequence.
pub(crate) async fn object_ddl(
pool: &SnowflakePool,
schema: &str,
kind: DbObjectKind,
name: &str,
) -> IdeResult<Vec<String>> {
let path = object_path(pool, schema, name);
let mut out = Vec::new();
match kind {
DbObjectKind::Sequence => out.extend(ddl(pool, "SEQUENCE", path).await?),
DbObjectKind::Function | DbObjectKind::Procedure => {
let object = if kind == DbObjectKind::Procedure { "PROCEDURE" } else { "FUNCTION" };
for routine in routines(pool, schema, kind, Some(name)).await?.into_iter().filter(|r| r.name == name) {
// Overloads are told apart by their argument types (names and defaults dropped).
let types: Vec<&str> = routine
.arguments
.split(',')
.map(|a| a.trim().trim_start_matches("DEFAULT ").trim())
.filter(|a| !a.is_empty())
.collect();
out.extend(ddl(pool, object, format!("{path}({})", types.join(", "))).await?);
}
}
DbObjectKind::Trigger | DbObjectKind::Package => {
return Err(IdeError::invalid("Snowflake has no triggers or packages"));
}
}
Ok(out)
}
async fn ddl(pool: &SnowflakePool, object: &str, target: String) -> IdeResult<Option<String>> {
let sql = format!("SELECT GET_DDL('{object}', {})", quote_literal(DRIVER, &target));
let rows = pool.metadata().execute(&sql, 1, &CancellationToken::new()).await?.rows;
Ok(rows.into_iter().next().and_then(|r| r.into_iter().next().flatten()))
}
/// A column type of `SHOW COLUMNS` (`{"type":"FIXED","precision":38,"scale":0,…}`) the way SQL spells it.
fn column_type(json: &str) -> String {
let Ok(value) = serde_json::from_str::<serde_json::Value>(json) else { return json.to_owned() };
let kind = value.get("type").and_then(|t| t.as_str()).unwrap_or_default().to_ascii_uppercase();
let number = |name: &str| value.get(name).and_then(|n| n.as_i64());
match kind.as_str() {
"FIXED" => match (number("precision"), number("scale")) {
(Some(p), Some(s)) => format!("NUMBER({p},{s})"),
_ => "NUMBER".into(),
},
"REAL" => "FLOAT".into(),
"TEXT" => match number("length") {
Some(n) if n > 0 && n < 16_777_216 => format!("VARCHAR({n})"),
_ => "VARCHAR".into(),
},
"TIME" | "TIMESTAMP_NTZ" | "TIMESTAMP_LTZ" | "TIMESTAMP_TZ" => match number("scale") {
Some(s) => format!("{kind}({s})"),
None => kind,
},
_ => kind,
}
}
/// Above this many tables, a database's columns are too many to read at once.
const SMALL_DATABASE: usize = 1_000;
/// Every table of the target's database for SQL completion, with the columns of all of them
/// when the database is small, else of the session's schema only. A table listed without
/// columns is one whose columns were not read.
pub(crate) async fn completion(pool: &SnowflakePool) -> IdeResult<Vec<CompletionTable>> {
let names = schemas(pool, false).await?;
let objects = database_objects(pool, pool.database_ident().as_deref(), &names).await?;
let mut tables: Vec<CompletionTable> = objects
.records()
.filter(|r| r.get("schema_name") != Some("INFORMATION_SCHEMA"))
.filter_map(|r| {
Some(CompletionTable {
schema: r.text("schema_name"),
name: r.get("name")?.to_owned(),
kind: table_kind(r.get("kind").unwrap_or("TABLE")),
columns: Vec::new(),
})
})
.collect();
tables.sort_by(|a, b| (&a.schema, &a.name).cmp(&(&b.schema, &b.name)));
tables.dedup_by(|a, b| a.schema == b.schema && a.name == b.name);
let scope = if tables.len() <= SMALL_DATABASE {
Some(in_database(pool))
} else {
let current = show(pool, "SELECT CURRENT_SCHEMA()").await?;
let schema = current.rows.first().and_then(|r| r.first().cloned().flatten()).filter(|s| !s.is_empty());
schema.map(|schema| format!(" IN SCHEMA {}", schema_path(pool, &schema)))
};
let Some(scope) = scope else { return Ok(tables) };
// Columns are a help, not a need: without them the tables still complete.
let Ok(shown) = show(pool, &format!("SHOW COLUMNS{scope}")).await else { return Ok(tables) };
let at: HashMap<(String, String), usize> =
tables.iter().enumerate().map(|(i, t)| ((t.schema.clone(), t.name.clone()), i)).collect();
for r in shown.records() {
let Some(&index) = at.get(&(r.text("schema_name"), r.text("table_name"))) else { continue };
tables[index].columns.push(CompletionColumn {
name: r.text("column_name"),
data_type: column_type(r.get("data_type").unwrap_or_default()),
nullable: !r
.get("null?")
.is_some_and(|v| v.eq_ignore_ascii_case("NOT_NULL") || v.eq_ignore_ascii_case("N")),
primary_key: false,
has_default: r.get("default").is_some_and(|d| !d.is_empty())
|| r.get("autoincrement").is_some_and(|a| !a.is_empty()),
});
}
Ok(tables)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn signatures_split_into_arguments_and_result() {
assert_eq!(signature("ADD_ONE(NUMBER) RETURN NUMBER"), ("NUMBER".into(), Some("NUMBER".into())));
assert_eq!(
signature("PARSE(VARCHAR, NUMBER(38,0)) RETURN TABLE (A NUMBER)"),
("VARCHAR, NUMBER(38,0)".into(), Some("TABLE (A NUMBER)".into()))
);
assert_eq!(signature("NOW_UTC() RETURN TIMESTAMP_NTZ"), (String::new(), Some("TIMESTAMP_NTZ".into())));
assert_eq!(signature(""), (String::new(), None));
}
#[test]
fn column_types_read_like_sql() {
assert_eq!(column_type(r#"{"type":"FIXED","precision":38,"scale":0,"nullable":true}"#), "NUMBER(38,0)");
assert_eq!(column_type(r#"{"type":"TEXT","length":100,"byteLength":400,"nullable":false}"#), "VARCHAR(100)");
assert_eq!(column_type(r#"{"type":"TEXT","length":16777216}"#), "VARCHAR");
assert_eq!(column_type(r#"{"type":"TIMESTAMP_NTZ","precision":0,"scale":9}"#), "TIMESTAMP_NTZ(9)");
assert_eq!(column_type(r#"{"type":"VARIANT","nullable":true}"#), "VARIANT");
assert_eq!(column_type(r#"{"type":"REAL"}"#), "FLOAT");
assert_eq!(column_type("not json"), "not json");
}
#[test]
fn object_kinds_become_tree_kinds() {
for kind in ["TABLE", "DYNAMIC TABLE", "EXTERNAL TABLE", "EVENT TABLE", "HYBRID TABLE"] {
assert_eq!(table_kind(kind), DbTableKind::Table, "{kind}");
}
assert_eq!(table_kind("VIEW"), DbTableKind::View);
assert_eq!(table_kind("MATERIALIZED_VIEW"), DbTableKind::MaterializedView);
assert_eq!(table_kind("MATERIALIZED VIEW"), DbTableKind::MaterializedView);
}
#[test]
fn imported_keys_group_by_constraint_in_column_order() {
let columns = [
"pk_schema_name",
"pk_table_name",
"pk_column_name",
"fk_schema_name",
"fk_table_name",
"fk_column_name",
"key_sequence",
"fk_name",
];
let row = |values: [&str; 8]| values.iter().map(|v| Some((*v).to_owned())).collect::<Vec<_>>();
let shown = Shown {
columns: columns.iter().map(|c| (*c).to_owned()).collect(),
rows: vec![
row(["PUBLIC", "REGIONS", "CODE", "PUBLIC", "ORDERS", "REGION_CODE", "2", "FK_REGION"]),
row(["PUBLIC", "CUSTOMERS", "ID", "PUBLIC", "ORDERS", "CUSTOMER_ID", "1", "FK_CUSTOMER"]),
row(["PUBLIC", "REGIONS", "COUNTRY", "PUBLIC", "ORDERS", "COUNTRY", "1", "FK_REGION"]),
],
};
let keys = foreign_keys(&shown);
assert_eq!(keys.len(), 2);
assert_eq!(
(keys[0].ref_table.as_str(), keys[0].columns.as_slice()),
("CUSTOMERS", &["CUSTOMER_ID".to_owned()][..])
);
assert_eq!(keys[1].columns, ["COUNTRY", "REGION_CODE"]);
assert_eq!(keys[1].ref_columns, ["COUNTRY", "CODE"]);
}
}
+999
Переглянути файл
@@ -0,0 +1,999 @@
//! Snowflake over HTTPS: the session protocol of Snowflake's own drivers (login, statements,
//! cancellation, token renewal) with results in JSON. One session per pool; statements run
//! one at a time on it, so session state (`USE`, `ALTER SESSION`, transactions) holds.
mod account;
mod browser;
mod client;
mod error;
mod keypair;
pub(crate) mod meta;
mod rows;
use crate::conn::{QueryResult, Setup};
use crate::source::{DataSource, DbSnowflakeAuth};
use client::{Api, Envelope, Request};
use ide_core::{CancellationToken, IdeError, IdeResult, error::ErrorKind};
use parking_lot::RwLock;
use rows::RowType;
use serde::Deserialize;
use serde_json::{Value, json};
use std::collections::BTreeMap;
use std::collections::HashMap;
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::{Arc, LazyLock, Weak};
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use tokio::sync::Mutex;
const APP: &str = "OpenIDEA";
const LOGIN: &str = "session/v1/login-request";
const QUERY: &str = "queries/v1/query-request";
const ABORT: &str = "queries/v1/abort-request";
const RENEW: &str = "session/token-request";
const SESSION: &str = "session";
/// DML statement types (`INSERT`, `UPDATE`, `DELETE`, `MERGE`…): their result is a row of counts.
const DML: std::ops::Range<i64> = 0x3000..0x4000;
const SELECT: i64 = 0x1000;
/// A connection target (a database of a data source) on the Snowflake session its data source's
/// targets share: one login serves every database, and each statement runs under its target's
/// database and schema.
#[derive(Clone)]
pub struct SnowflakePool {
shared: Arc<State>,
/// The target's database and the data source's schema (empty: the user's defaults).
database: String,
schema: String,
read_only: Arc<AtomicBool>,
closed: CancellationToken,
/// Which of the data source's sessions this handle runs on.
lane: usize,
}
static CONNECTING: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
/// The key the single sign-on ID token of `source` is cached under (`None`: another method).
pub(crate) fn sso_key(source: &DataSource) -> Option<String> {
if source.snowflake().auth != DbSnowflakeAuth::Browser {
return None;
}
let endpoint = account::endpoint(source).ok()?;
let address = format!(
"{}:{}",
endpoint.base.host_str().unwrap_or_default(),
endpoint.base.port_or_known_default().unwrap_or_default()
);
Some(format!("{address}|{}", source.user.trim().to_uppercase()))
}
/// The ID token a browser sign-in left, which signs in again without the browser while Snowflake accepts it.
pub(crate) fn sso_token(key: &str) -> Option<String> {
browser::cached(key)
}
/// Offers a saved ID token for the next sign-in (one already in use is kept).
pub(crate) fn sso_seed(key: &str, token: &str) {
if browser::cached(key).is_none() {
browser::remember(key, token);
}
}
/// Live sessions by data source and connection settings, so the targets of one data source (its
/// databases, the tree, consoles) opened with the same settings share one.
static SESSIONS: LazyLock<parking_lot::Mutex<HashMap<String, Weak<State>>>> =
LazyLock::new(|| parking_lot::Mutex::new(HashMap::new()));
struct State {
api: Api,
login: Login,
/// The consoles' session and the tree's ([`CONSOLE`], [`METADATA`]): a long statement in a
/// console does not hold up the tree. The second one logs in when it is first used.
lanes: [Lane; 2],
/// Open targets on this session; the last one to close ends it.
users: AtomicUsize,
key: String,
closed: CancellationToken,
}
/// One Snowflake session.
#[derive(Default)]
struct Lane {
tokens: RwLock<Option<Tokens>>,
/// Statements share the session, so they take turns.
turn: Mutex<()>,
sequence: AtomicU64,
/// The `USE` statement the session is known to be under (`None`: a statement may have
/// moved it; empty: as the login left it, without a database or schema).
context: parking_lot::Mutex<Option<String>>,
}
/// What [`SnowflakePool::statement`] does when the session is gone and it signs in again.
#[derive(Clone, Copy)]
enum Lost<'a> {
/// Runs the statement again on the new session, after this `USE` when there is one.
Enter(Option<&'a str>),
/// Fails: the statement was part of a transaction that ended with the session.
Fail,
}
const CONSOLE: usize = 0;
const METADATA: usize = 1;
struct Login {
account: String,
user: String,
credentials: Credentials,
database: String,
schema: String,
warehouse: String,
role: String,
parameters: Vec<(String, String)>,
timeout: Duration,
}
enum Credentials {
KeyPair { file: String, passphrase: Option<String> },
Password(String),
Token(String),
OAuth(String),
Browser,
}
#[derive(Clone)]
struct Tokens {
session: String,
master: String,
}
#[derive(Debug, Default, Deserialize)]
#[serde(default, rename_all = "camelCase")]
struct ResultData {
rowtype: Vec<RowType>,
rowset: Vec<Vec<Value>>,
rowset_base64: Option<String>,
chunks: Vec<Chunk>,
chunk_headers: BTreeMap<String, String>,
qrmk: Option<String>,
query_result_format: Option<String>,
statement_type_id: Option<i64>,
get_result_url: Option<String>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(default, rename_all = "camelCase")]
struct Chunk {
url: String,
row_count: usize,
}
fn uuid() -> String {
uuid::Uuid::new_v4().to_string()
}
fn now_secs() -> u64 {
SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_secs())
}
fn closed() -> IdeError {
IdeError::new(ErrorKind::Io, "Snowflake connection is closed")
}
/// A session parameter as JSON: Snowflake types booleans and numbers.
fn parameter(value: &str) -> Value {
if value.eq_ignore_ascii_case("true") {
Value::Bool(true)
} else if value.eq_ignore_ascii_case("false") {
Value::Bool(false)
} else if let Ok(n) = value.parse::<i64>() {
json!(n)
} else {
Value::String(value.to_owned())
}
}
/// Settings of the Advanced tab the driver sets itself.
fn reserved(name: &str) -> bool {
matches!(
name.to_ascii_uppercase().as_str(),
"ROWS_PER_RESULTSET" | "MULTI_STATEMENT_COUNT" | "CLIENT_STORE_TEMPORARY_CREDENTIAL"
)
}
/// A name typed in the dialog, as SQL: typed in one case it is an unquoted name, which folds
/// to upper case; anything else is taken as the exact name.
fn typed(name: &str) -> String {
let plain = name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '$')
&& !name.starts_with(|c: char| c.is_ascii_digit());
let one_case = !name.chars().any(|c| c.is_ascii_lowercase()) || !name.chars().any(|c| c.is_ascii_uppercase());
let name = if plain && one_case { name.to_ascii_uppercase() } else { name.to_owned() };
crate::sql::quote_ident(crate::DbDriver::Snowflake, &name)
}
fn first_word(sql: &str) -> String {
let mut sql = sql.trim_start();
loop {
if let Some(rest) = sql.strip_prefix("--").or_else(|| sql.strip_prefix("//")) {
sql = rest.split_once('\n').map_or("", |(_, rest)| rest).trim_start();
} else if let Some(rest) = sql.strip_prefix("/*") {
sql = rest.split_once("*/").map_or("", |(_, rest)| rest).trim_start();
} else {
break;
}
}
sql.split(|c: char| !c.is_ascii_alphanumeric() && c != '_').next().unwrap_or_default().to_ascii_uppercase()
}
/// Whether `sql` may change data or objects (a read-only data source refuses it).
/// Snowflake answers a `USE` it will not perform with one message for every cause; this adds
/// the causes.
fn explain_use(sql: &str, mut error: IdeError) -> IdeError {
if first_word(sql) == "USE" && error.message.contains("002043") {
error.message.push_str(
"\nThe role, warehouse, database or schema does not exist, or is not granted: a role must be \
granted to your user (SHOW GRANTS TO USER <name>), the others to the current role.",
);
}
error
}
/// Whether `sql` may change the account. Statements that read, and those that change only the
/// session (`USE`, `SET`, `UNSET`), run on a read-only data source.
fn is_write(sql: &str) -> bool {
match first_word(sql).as_str() {
"SELECT" | "SHOW" | "DESCRIBE" | "DESC" | "EXPLAIN" | "LIST" | "LS" | "USE" | "SET" | "UNSET" | "VALUES"
| "TABLE" | "" => false,
// `WITH name AS PROCEDURE (…) … CALL name(…)` runs a procedure; other `WITH`s select.
"WITH" => {
sql.to_ascii_uppercase().split_whitespace().collect::<Vec<_>>().windows(2).any(|w| w == ["AS", "PROCEDURE"])
}
_ => true,
}
}
fn gunzip(bytes: Vec<u8>) -> IdeResult<Vec<u8>> {
if !bytes.starts_with(&[0x1f, 0x8b]) {
return Ok(bytes);
}
let mut out = Vec::new();
std::io::Read::read_to_end(&mut flate2::read::GzDecoder::new(bytes.as_slice()), &mut out)
.map_err(|e| IdeError::new(ErrorKind::Protocol, format!("Snowflake: cannot read a result chunk: {e}")))?;
Ok(out)
}
impl SnowflakePool {
pub(crate) async fn connect(
source: &DataSource,
secret: Option<&str>,
setup: Setup,
root: Option<&std::path::Path>,
) -> IdeResult<Self> {
let settings = source.snowflake();
let endpoint = account::endpoint(source)?;
let user = source.user.trim();
if user.is_empty() {
return Err(IdeError::invalid("enter the Snowflake user name"));
}
for (name, _) in &setup.params {
if reserved(name) {
return Err(IdeError::invalid(format!("the Snowflake driver manages the {name} parameter")));
}
}
let secret = secret.filter(|s| !s.is_empty());
let ask = |what: &str| IdeError::new(ErrorKind::PermissionDenied, format!("enter the {what}"));
let credentials = match settings.auth {
DbSnowflakeAuth::KeyPair => {
Credentials::KeyPair { file: settings.private_key_file.clone(), passphrase: secret.map(str::to_owned) }
}
DbSnowflakeAuth::Password => Credentials::Password(secret.ok_or_else(|| ask("password"))?.to_owned()),
DbSnowflakeAuth::Token => {
Credentials::Token(secret.ok_or_else(|| ask("programmatic access token"))?.to_owned())
}
DbSnowflakeAuth::OAuth => Credentials::OAuth(secret.ok_or_else(|| ask("OAuth access token"))?.to_owned()),
DbSnowflakeAuth::Browser => Credentials::Browser,
};
let timeout = setup.connect_timeout.unwrap_or(Duration::from_secs(30));
let _ = rustls::crypto::ring::default_provider().install_default();
let builder = reqwest::Client::builder()
.connect_timeout(timeout)
.pool_max_idle_per_host(4)
.redirect(reqwest::redirect::Policy::none())
.user_agent(concat!("OpenIDEA/", env!("CARGO_PKG_VERSION")));
let proxy = ide_core::proxy_settings::proxy();
let http = ide_http::client::with_proxy(builder, &proxy)
.build()
.map_err(|e| IdeError::new(ErrorKind::Io, format!("Snowflake: cannot start the HTTP client: {e}")))?;
let api = Api::new(http, endpoint.base);
if let Some(secret) = secret {
api.scrub.add(secret);
}
let database = source.database.trim().to_owned();
let schema = source.options.default_schema.trim().to_owned();
let target = |shared: Arc<State>| Self {
shared,
database: database.clone(),
schema: schema.clone(),
read_only: Arc::new(AtomicBool::new(source.read_only)),
closed: CancellationToken::new(),
lane: CONSOLE,
};
// Everything that makes a session what it is, the secret included (hashed: the key is only compared).
// The data source and its project too: its databases share a session, but another data source
// with the same settings, or the same one in another project (a plugin's source has the same id
// in every project), gets its own: its statements and transactions are not this one's. A browser
// sign-in is still shared through the cached ID token.
let key = blake3::hash(
format!(
"{:?}\0{}\0{}\0{user}\0{:?}\0{}\0{}\0{}\0{}\0{:?}\0{:?}",
root,
source.id,
api.address(),
settings.auth,
secret.unwrap_or_default(),
settings.private_key_file,
settings.warehouse.trim(),
settings.role.trim(),
setup.params,
setup.statements
)
.as_bytes(),
)
.to_hex()
.to_string();
// One connect at a time: targets opened together (the tree, consoles, completion) find the
// session the first of them signs in to, instead of each signing in (and each opening the
// browser with single sign-on).
let _connecting = CONNECTING.lock().await;
let live = SESSIONS.lock().get(&key).and_then(Weak::upgrade).filter(|s| !s.closed.is_cancelled());
if let Some(shared) = live {
shared.users.fetch_add(1, Ordering::SeqCst);
return Ok(target(shared));
}
let pool = target(Arc::new(State {
api,
login: Login {
account: endpoint.account,
user: user.to_owned(),
credentials,
database: database.clone(),
schema: schema.clone(),
warehouse: settings.warehouse.trim().to_owned(),
role: settings.role.trim().to_owned(),
parameters: setup.params,
timeout,
},
lanes: Default::default(),
users: AtomicUsize::new(1),
key: key.clone(),
closed: CancellationToken::new(),
}));
pool.login().await?;
// The login asked for this target's database and schema.
*pool.lane().context.lock() = Some(pool.context(None).unwrap_or_default());
let cancel = CancellationToken::new();
// The startup script may write whatever the read-only setting says.
pool.read_only.store(false, Ordering::Relaxed);
for sql in setup.statements {
if let Err(error) = pool.execute(&sql, 0, &cancel).await {
pool.close().await;
return Err(error);
}
}
pool.read_only.store(source.read_only, Ordering::Relaxed);
SESSIONS.lock().insert(key, Arc::downgrade(&pool.shared));
Ok(pool)
}
/// The target's database as SQL (`None`: the session's current database).
pub(crate) fn database_ident(&self) -> Option<String> {
(!self.database.is_empty()).then(|| typed(&self.database))
}
/// The `USE` statement for this target with `schema` active (`None`: the data source's schema).
fn context(&self, schema: Option<&str>) -> Option<String> {
let quote = |name: &str| crate::sql::quote_ident(crate::DbDriver::Snowflake, name);
let schema = match schema.filter(|s| !s.is_empty()) {
// Picked from the list of schemas: the exact name.
Some(picked) => Some(quote(picked)),
None if self.schema.is_empty() => None,
None => Some(typed(&self.schema)),
};
match (self.database.is_empty(), schema) {
(false, Some(schema)) => Some(format!("USE SCHEMA {}.{schema}", typed(&self.database))),
(false, None) => Some(format!("USE DATABASE {}", typed(&self.database))),
(true, Some(schema)) => Some(format!("USE SCHEMA {schema}")),
(true, None) => None,
}
}
/// Ends the session on the server (best effort) and refuses further statements.
pub async fn close(&self) {
if self.closed.is_cancelled() {
return;
}
self.closed.cancel();
if self.shared.users.fetch_sub(1, Ordering::SeqCst) != 1 {
return;
}
self.shared.closed.cancel();
{
let mut sessions = SESSIONS.lock();
if sessions.get(&self.shared.key).is_some_and(|s| std::ptr::eq(s.as_ptr(), Arc::as_ptr(&self.shared))) {
sessions.remove(&self.shared.key);
}
}
for lane in &self.shared.lanes {
let tokens = lane.tokens.write().take();
if let Some(tokens) = tokens {
let request = Request::post(SESSION, json!({}))
.query("delete", "true")
.query("requestId", uuid())
.token(&tokens.session)
.timeout(Duration::from_secs(5));
let _ = self.shared.api.send(request).await;
}
}
}
fn lane(&self) -> &Lane {
&self.shared.lanes[self.lane]
}
/// This target on the session the tree reads metadata with.
pub(crate) fn metadata(&self) -> Self {
// A second browser sign-in would interrupt the user: without an ID token to sign in
// with, metadata takes turns with the consoles.
let second = !matches!(self.shared.login.credentials, Credentials::Browser)
|| browser::cached(&self.browser_key()).is_some();
Self { lane: if second { METADATA } else { CONSOLE }, ..self.clone() }
}
fn browser_key(&self) -> String {
format!("{}|{}", self.shared.api.address(), self.shared.login.user.to_uppercase())
}
pub fn is_closed(&self) -> bool {
self.closed.is_cancelled() || self.shared.closed.is_cancelled()
}
/// The fields of the login request that prove who the user is.
async fn proof(&self) -> IdeResult<(Vec<(&'static str, Value)>, Option<String>)> {
let login = &self.shared.login;
Ok(match &login.credentials {
Credentials::KeyPair { file, passphrase } => {
let (file, passphrase) = (file.clone(), passphrase.clone());
let (account, user) = (login.account.clone(), login.user.clone());
// Decrypting the key derives its encryption key, which takes a while by design.
let (token, fingerprint) = tokio::task::spawn_blocking(move || {
let pair = keypair::load(&file, passphrase.as_deref())?;
Ok::<_, IdeError>((pair.jwt(&account, &user, now_secs())?, pair.fingerprint()?))
})
.await
.map_err(|e| IdeError::internal(e.to_string()))??;
self.shared.api.scrub.add(&token);
(vec![("AUTHENTICATOR", json!("SNOWFLAKE_JWT")), ("TOKEN", json!(token))], Some(fingerprint))
}
Credentials::Password(password) => (vec![("PASSWORD", json!(password))], None),
Credentials::Token(token) => {
(vec![("AUTHENTICATOR", json!("PROGRAMMATIC_ACCESS_TOKEN")), ("TOKEN", json!(token))], None)
}
Credentials::OAuth(token) => (vec![("AUTHENTICATOR", json!("OAUTH")), ("TOKEN", json!(token))], None),
Credentials::Browser => unreachable!("browser sign-in has its own login"),
})
}
async fn login(&self) -> IdeResult<Tokens> {
if matches!(self.shared.login.credentials, Credentials::Browser) {
return self.browser_login().await;
}
let (proof, fingerprint) = self.proof().await?;
self.sign_in(proof, fingerprint, None).await
}
/// Signs in through the web browser, or without it while an ID token of an earlier
/// sign-in of this run is still accepted.
async fn browser_login(&self) -> IdeResult<Tokens> {
let login = &self.shared.login;
let key = self.browser_key();
let _flow = browser::FLOW.lock().await;
if let Some(id_token) = browser::cached(&key) {
self.shared.api.scrub.add(&id_token);
let proof = vec![("AUTHENTICATOR", json!("ID_TOKEN")), ("TOKEN", json!(id_token))];
match self.sign_in(proof, None, Some(&key)).await {
Ok(tokens) => return Ok(tokens),
// The network is down: the browser would not get further either.
Err(error) if matches!(error.kind, ErrorKind::Io | ErrorKind::TimedOut) => return Err(error),
Err(_) => browser::forget(&key),
}
}
let (token, proof_key) =
browser::authenticate(&self.shared.api, &login.account, &login.user, login.timeout).await?;
let proof =
vec![("AUTHENTICATOR", json!("EXTERNALBROWSER")), ("TOKEN", json!(token)), ("PROOF_KEY", json!(proof_key))];
self.sign_in(proof, None, Some(&key)).await
}
/// The login request. `id_token_key`: ask for an ID token and keep it under this key.
async fn sign_in(
&self,
proof: Vec<(&'static str, Value)>,
fingerprint: Option<String>,
id_token_key: Option<&str>,
) -> IdeResult<Tokens> {
let login = &self.shared.login;
let mut parameters: serde_json::Map<String, Value> =
login.parameters.iter().map(|(name, value)| (name.to_ascii_uppercase(), parameter(value))).collect();
if id_token_key.is_some() {
parameters.insert("CLIENT_STORE_TEMPORARY_CREDENTIAL".into(), Value::Bool(true));
}
let mut data = json!({
"CLIENT_APP_ID": APP,
"CLIENT_APP_VERSION": env!("CARGO_PKG_VERSION"),
"ACCOUNT_NAME": login.account,
"LOGIN_NAME": login.user,
"CLIENT_ENVIRONMENT": { "APPLICATION": APP, "OS": std::env::consts::OS },
"SESSION_PARAMETERS": parameters,
});
for (name, value) in proof {
data[name] = value;
}
let mut request = Request::post(LOGIN, json!({ "data": data }))
.query("requestId", uuid())
.query("request_guid", uuid())
.timeout(login.timeout);
for (name, value) in [
("databaseName", &login.database),
("schemaName", &login.schema),
("warehouse", &login.warehouse),
("roleName", &login.role),
] {
if !value.is_empty() {
request = request.query(name, value.as_str());
}
}
let envelope = self.shared.api.send(request).await?;
if !envelope.success {
let code = envelope.code.as_deref().unwrap_or_default();
let mut failure = error::server(envelope.code.as_deref(), envelope.message.as_deref());
if let Some(fingerprint) = fingerprint.filter(|_| error::key_rejected(code)) {
failure.message.push_str(&format!("\nFingerprint of this private key: {fingerprint}"));
}
return Err(self.shared.api.scrub.clean(failure));
}
let text = |name: &str| envelope.data.get(name).and_then(Value::as_str).map(str::to_owned);
let (Some(session), Some(master)) = (text("token"), text("masterToken")) else {
return Err(IdeError::new(ErrorKind::Protocol, "Snowflake: the login response has no session token"));
};
self.shared.api.scrub.add(&session);
self.shared.api.scrub.add(&master);
if let (Some(key), Some(id_token)) = (id_token_key, text("idToken")) {
self.shared.api.scrub.add(&id_token);
browser::remember(key, &id_token);
}
let tokens = Tokens { session, master };
if self.is_closed() {
return Err(closed());
}
// A new session: nothing is known about where it is until a statement says.
*self.lane().context.lock() = None;
*self.lane().tokens.write() = Some(tokens.clone());
Ok(tokens)
}
/// A fresh session token from the master token; a new login when that is refused.
async fn renew(&self, stale: &Tokens) -> IdeResult<Tokens> {
let request = Request::post(RENEW, json!({ "oldSessionToken": stale.session, "requestType": "RENEW" }))
.query("requestId", uuid())
.token(&stale.master)
.timeout(self.shared.login.timeout);
if let Ok(envelope) = self.shared.api.send(request).await
&& envelope.success
{
let text = |name: &str| envelope.data.get(name).and_then(Value::as_str).map(str::to_owned);
if let Some(session) = text("sessionToken") {
let master = text("masterToken").unwrap_or_else(|| stale.master.clone());
self.shared.api.scrub.add(&session);
self.shared.api.scrub.add(&master);
let tokens = Tokens { session, master };
*self.lane().tokens.write() = Some(tokens.clone());
return Ok(tokens);
}
}
self.login().await
}
async fn tokens(&self) -> IdeResult<Tokens> {
let current = self.lane().tokens.read().clone();
match current {
Some(tokens) => Ok(tokens),
None if self.is_closed() => Err(closed()),
None => self.login().await,
}
}
pub async fn execute(&self, sql: &str, max_rows: usize, cancel: &CancellationToken) -> IdeResult<QueryResult> {
self.execute_from(sql, 0, max_rows, cancel).await
}
/// [`execute`](Self::execute), leaving out the first `skip` rows (a later page): whole
/// result chunks before the page are not downloaded.
pub async fn execute_from(
&self,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
self.execute_in(None, sql, skip, max_rows, cancel).await
}
/// [`execute_from`](Self::execute_from) with unqualified names resolving in `schema`
/// (a console's active schema; `None`: the data source's). The switch and the statement
/// hold the session together, so consoles with different schemas do not cross.
pub async fn execute_in(
&self,
schema: Option<&str>,
sql: &str,
skip: usize,
max_rows: usize,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
if self.read_only.load(Ordering::Relaxed) && is_write(sql) {
return Err(IdeError::invalid(format!(
"read-only data source: {} statements are not allowed",
first_word(sql)
)));
}
if matches!(first_word(sql).as_str(), "PUT" | "GET") {
// The server only plans these; the client moves the files, which this driver does not do.
return Err(IdeError::invalid(format!(
"{} copies files between this computer and a stage, which OpenIDEA does not do yet: \
run it with the Snowflake CLI (snow sql) or SnowSQL",
first_word(sql)
)));
}
let _turn = tokio::select! {
biased;
() = cancel.cancelled() => return Err(IdeError::cancelled()),
() = self.closed.cancelled() => return Err(closed()),
turn = self.lane().turn.lock() => turn,
};
// A `USE` needs no schema to run in, and may be what makes the schema reachable. Ending a
// transaction needs none either, and must not fail because the schema is gone meanwhile.
let context = match first_word(sql).as_str() {
"USE" | "COMMIT" | "ROLLBACK" => None,
_ => {
self.enter(schema, cancel).await?;
self.context(schema)
}
};
// These can move the session to another database or schema.
let moves = matches!(first_word(sql).as_str(), "USE" | "CALL" | "EXECUTE");
let result = self.statement(sql, skip, max_rows, Lost::Enter(context.as_deref()), cancel).await;
if moves {
*self.lane().context.lock() = None;
}
result.map_err(|error| explain_use(sql, error))
}
/// Puts the session under this target's database and `schema` when it is not there already.
async fn enter(&self, schema: Option<&str>, cancel: &CancellationToken) -> IdeResult<()> {
if let Some(wanted) = self.context(schema)
&& self.lane().context.lock().as_deref() != Some(wanted.as_str())
{
*self.lane().context.lock() = None;
// Refused (the session's role may not use this database or schema): the statement
// does not run, since the session is still wherever it was, which may be another
// target's database. Tried again with the next statement.
if let Err(error) = self.statement(&wanted, 0, 0, Lost::Enter(None), cancel).await {
return Err(IdeError {
message: format!("{wanted} failed, so the statement was not run: {}", error.message),
..error
});
}
*self.lane().context.lock() = Some(wanted);
}
Ok(())
}
/// Runs `statements` in one transaction; each must change one row when flagged
/// (table editor submits). Nothing else runs on the session meanwhile.
pub(crate) async fn transaction(&self, statements: &[(String, bool)]) -> IdeResult<u64> {
if self.read_only.load(Ordering::Relaxed) {
return Err(IdeError::invalid("read-only data source: changes are not allowed"));
}
let _turn = self.lane().turn.lock().await;
let cancel = CancellationToken::new();
self.enter(None, &cancel).await?;
self.statement("BEGIN", 0, 0, Lost::Enter(self.context(None).as_deref()), &cancel).await?;
let changes = async {
let mut total = 0;
for (sql, expect_one) in statements {
let changed = self.statement(sql, 0, 0, Lost::Fail, &cancel).await?.affected.unwrap_or(0.0) as u64;
if *expect_one && changed != 1 {
return Err(IdeError::invalid(format!(
"{sql}: expected to change 1 row, changed {changed}; nothing was saved"
)));
}
total += changed;
}
Ok(total)
}
.await;
match changes {
Ok(total) => {
self.statement("COMMIT", 0, 0, Lost::Fail, &cancel).await?;
Ok(total)
}
Err(error) => {
let _ = self.statement("ROLLBACK", 0, 0, Lost::Fail, &cancel).await;
Err(error)
}
}
}
/// One statement on the session; the caller holds the session's turn. `lost`: what to do when
/// the session is gone and a new one is signed in to.
async fn statement(
&self,
sql: &str,
skip: usize,
max_rows: usize,
lost: Lost<'_>,
cancel: &CancellationToken,
) -> IdeResult<QueryResult> {
let started = Instant::now();
let mut tokens = self.tokens().await?;
let mut renewed = false;
loop {
let request_id = uuid();
let run = self.run(sql, skip, max_rows, &tokens, &request_id, started);
let outcome = tokio::select! {
biased;
() = cancel.cancelled() => None,
() = self.closed.cancelled() => None,
outcome = run => Some(outcome),
};
let Some(outcome) = outcome else {
if self.is_closed() {
return Err(closed());
}
return Err(match self.abort(sql, &request_id, &tokens).await {
Ok(()) => IdeError::cancelled(),
Err(error) => IdeError::new(
ErrorKind::Cancelled,
format!("cancelled locally; Snowflake did not confirm the cancellation: {}", error.message),
),
});
};
match outcome {
Ok(Outcome::Done(result)) => return Ok(*result),
Ok(Outcome::Expired { gone }) if !renewed => {
renewed = true;
if !gone {
tokens = self.renew(&tokens).await?;
continue;
}
if let Lost::Fail = lost {
// The transaction ended with the session: nothing of it was saved.
let _ = self.login().await;
return Err(IdeError::new(
ErrorKind::Io,
"Snowflake: the session was lost during the transaction; nothing was saved",
));
}
// The new session is under the login's database and schema, which may be another
// target's: back to this one's before the statement runs again.
tokens = self.login().await?;
if let Lost::Enter(Some(wanted)) = lost {
self.reenter(wanted, &tokens, cancel).await?;
}
}
Ok(Outcome::Expired { .. }) => {
return Err(IdeError::new(ErrorKind::Io, "Snowflake: the session expired; reconnect"));
}
Err(error) => return Err(self.shared.api.scrub.clean(error)),
}
}
}
/// [`enter`](Self::enter) on a session just signed in to (its `tokens`), within [`statement`](Self::statement).
async fn reenter(&self, wanted: &str, tokens: &Tokens, cancel: &CancellationToken) -> IdeResult<()> {
let request_id = uuid();
let outcome = tokio::select! {
biased;
() = cancel.cancelled() => return Err(IdeError::cancelled()),
() = self.closed.cancelled() => return Err(closed()),
outcome = self.run(wanted, 0, 0, tokens, &request_id, Instant::now()) => outcome,
};
match outcome {
Ok(Outcome::Done(_)) => {
*self.lane().context.lock() = Some(wanted.to_owned());
Ok(())
}
Ok(Outcome::Expired { .. }) => {
Err(IdeError::new(ErrorKind::Io, "Snowflake: the session expired; reconnect"))
}
Err(error) => {
let error = self.shared.api.scrub.clean(error);
Err(IdeError {
message: format!("{wanted} failed, so the statement was not run: {}", error.message),
..error
})
}
}
}
async fn run(
&self,
sql: &str,
skip: usize,
max_rows: usize,
tokens: &Tokens,
request_id: &str,
started: Instant,
) -> IdeResult<Outcome> {
let wanted = skip.saturating_add(max_rows).saturating_add(1);
let body = json!({
"sqlText": sql,
"asyncExec": false,
"sequenceId": self.lane().sequence.fetch_add(1, Ordering::Relaxed) + 1,
"querySubmissionTime": SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_millis() as u64),
// The grid's page, plus one row that tells whether more exist.
"parameters": { "ROWS_PER_RESULTSET": wanted },
});
let request = Request::post(QUERY, body)
.query("requestId", request_id)
.query("request_guid", uuid())
.token(&tokens.session);
let mut envelope = self.shared.api.send(request).await?;
while envelope.has_code(error::IN_PROGRESS) {
let Some(url) = envelope.data.get("getResultUrl").and_then(Value::as_str).map(str::to_owned) else {
return Err(IdeError::new(ErrorKind::Protocol, "Snowflake: a running statement has no result address"));
};
tokio::time::sleep(Duration::from_millis(200)).await;
envelope = self.shared.api.send(Request::get(url.trim_start_matches('/')).token(&tokens.session)).await?;
}
if let Some(expired) = expiry(&envelope) {
return Ok(expired);
}
if !envelope.success {
return Err(error::server(envelope.code.as_deref(), envelope.message.as_deref()));
}
let execution_ms = started.elapsed().as_secs_f64() * 1000.0;
let data: ResultData = serde_json::from_value(envelope.data)
.map_err(|e| IdeError::new(ErrorKind::Protocol, format!("Snowflake: unreadable result: {e}")))?;
let arrow = data.query_result_format.as_deref().is_some_and(|f| !f.eq_ignore_ascii_case("json"));
if arrow || (data.rowset_base64.as_deref().is_some_and(|r| !r.is_empty()) && data.rowset.is_empty()) {
return Err(IdeError::new(
ErrorKind::Protocol,
"Snowflake sent the result in a format other than JSON, which this driver does not read",
));
}
let (rows, has_more) = self.rows(&data, skip, max_rows).await?;
let kind = data.statement_type_id.unwrap_or(SELECT);
let status_only = kind != SELECT && data.rowtype.len() == 1 && data.rowtype[0].name == "status";
let (columns, rows, affected, has_more) = if DML.contains(&kind) {
let changed: f64 = data
.rowset
.first()
.map(|row| row.iter().filter_map(|v| v.as_str().and_then(|s| s.parse::<f64>().ok())).sum())
.unwrap_or(0.0);
(Vec::new(), Vec::new(), Some(changed), false)
} else if status_only {
(Vec::new(), Vec::new(), Some(0.0), false)
} else {
(data.rowtype.iter().map(RowType::column).collect(), rows, None, has_more)
};
Ok(Outcome::Done(Box::new(QueryResult {
statement: sql.to_owned(),
columns,
rows,
affected,
has_more,
elapsed_ms: started.elapsed().as_secs_f64() * 1000.0,
execution_ms,
})))
}
/// The page of rows `skip..skip + max_rows`, and whether rows follow it.
async fn rows(
&self,
data: &ResultData,
mut skip: usize,
max_rows: usize,
) -> IdeResult<(Vec<Vec<Option<String>>>, bool)> {
let mut rows = Vec::new();
let mut more = false;
let mut take = |batch: Vec<Vec<Value>>, skip: &mut usize| -> IdeResult<bool> {
for row in batch {
if *skip > 0 {
*skip -= 1;
continue;
}
if rows.len() >= max_rows {
return Ok(true);
}
if row.len() != data.rowtype.len() {
return Err(IdeError::new(
ErrorKind::Protocol,
"Snowflake: a result row does not match the result's columns",
));
}
rows.push(
row.into_iter().zip(&data.rowtype).map(|(value, column)| rows::cell(column, value)).collect(),
);
}
Ok(false)
};
if take(data.rowset.clone(), &mut skip)? {
more = true;
}
let headers: Vec<(String, String)> = if !data.chunk_headers.is_empty() {
data.chunk_headers.iter().map(|(name, value)| (name.clone(), value.clone())).collect()
} else if let Some(key) = &data.qrmk {
vec![
("x-amz-server-side-encryption-customer-algorithm".into(), "AES256".into()),
("x-amz-server-side-encryption-customer-key".into(), key.clone()),
]
} else {
Vec::new()
};
for chunk in &data.chunks {
if more {
break;
}
if chunk.row_count > 0 && skip >= chunk.row_count {
skip -= chunk.row_count;
continue;
}
let body = gunzip(self.shared.api.chunk(&chunk.url, &headers).await?)?;
// A chunk is its rows separated by commas, without the enclosing brackets.
let mut text = Vec::with_capacity(body.len() + 2);
text.push(b'[');
text.extend_from_slice(&body);
text.push(b']');
let batch: Vec<Vec<Value>> = serde_json::from_slice(&text)
.map_err(|e| IdeError::new(ErrorKind::Protocol, format!("Snowflake: unreadable result chunk: {e}")))?;
if take(batch, &mut skip)? {
more = true;
}
}
Ok((rows, more))
}
async fn abort(&self, sql: &str, request_id: &str, tokens: &Tokens) -> IdeResult<()> {
let request = Request::post(ABORT, json!({ "sqlText": sql, "requestId": request_id }))
.query("requestId", uuid())
.token(&tokens.session)
.timeout(Duration::from_secs(5));
let envelope = self.shared.api.send(request).await?;
if envelope.success {
Ok(())
} else {
Err(self.shared.api.scrub.clean(error::server(envelope.code.as_deref(), envelope.message.as_deref())))
}
}
}
enum Outcome {
Done(Box<QueryResult>),
/// The session token no longer works (`gone`: nor does the master token).
Expired {
gone: bool,
},
}
fn expiry(envelope: &Envelope) -> Option<Outcome> {
if envelope.has_code(&[error::SESSION_EXPIRED]) {
Some(Outcome::Expired { gone: false })
} else if envelope.has_code(error::SESSION_GONE) {
Some(Outcome::Expired { gone: true })
} else {
None
}
}
#[cfg(test)]
mod commands;
#[cfg(test)]
mod tests;
+210
Переглянути файл
@@ -0,0 +1,210 @@
//! Result decoding: Snowflake's JSON result format sends every cell as text in an internal
//! form (dates as day numbers, timestamps as epoch seconds); these become the grid's text.
use crate::conn::ResultColumn;
use serde::Deserialize;
use serde_json::Value;
/// A column of the result's `rowtype`.
#[derive(Debug, Clone, Default, Deserialize)]
#[serde(default)]
pub(crate) struct RowType {
pub name: String,
#[serde(rename = "type")]
pub kind: String,
pub precision: Option<i64>,
pub scale: Option<i64>,
pub length: Option<i64>,
}
/// `VARCHAR` without a length: Snowflake reports its maximum.
const MAX_TEXT: i64 = 16_777_216;
impl RowType {
pub(crate) fn column(&self) -> ResultColumn {
let kind = self.kind.to_ascii_lowercase();
let scaled = |name: &str| match self.scale {
Some(scale) => format!("{name}({scale})"),
None => name.to_owned(),
};
let type_name = match kind.as_str() {
"fixed" => match (self.precision, self.scale) {
(Some(p), Some(s)) => format!("NUMBER({p},{s})"),
_ => "NUMBER".into(),
},
"real" => "FLOAT".into(),
"text" => match self.length {
Some(n) if n > 0 && n < MAX_TEXT => format!("VARCHAR({n})"),
_ => "VARCHAR".into(),
},
"time" => scaled("TIME"),
"timestamp_ntz" | "timestamp_ltz" | "timestamp_tz" => scaled(&kind.to_ascii_uppercase()),
other => other.to_ascii_uppercase(),
};
ResultColumn { name: self.name.clone(), type_name }
}
}
/// The grid text of one cell (`None`: SQL NULL).
pub(crate) fn cell(column: &RowType, value: Value) -> Option<String> {
let text = match value {
Value::Null => return None,
Value::String(text) => text,
other => other.to_string(),
};
let scale = column.scale.unwrap_or(0).clamp(0, 9) as usize;
let shown = match column.kind.to_ascii_lowercase().as_str() {
"boolean" => Some(if matches!(text.as_str(), "1" | "true" | "TRUE") { "true" } else { "false" }.to_owned()),
"date" => text.parse::<i64>().ok().map(date),
"time" => nanos(&text).map(|n| time(n.rem_euclid(DAY), scale)),
"timestamp_ntz" => nanos(&text).map(|n| timestamp(n, scale)),
"timestamp_ltz" => nanos(&text).map(|n| format!("{} Z", timestamp(n, scale))),
"timestamp_tz" => timestamp_tz(&text, scale),
"binary" => Some(format!("0x{text}")),
_ => None,
};
Some(shown.unwrap_or(text))
}
const SECOND: i128 = 1_000_000_000;
const DAY: i128 = 86_400 * SECOND;
/// `seconds[.fraction]` as nanoseconds.
fn nanos(text: &str) -> Option<i128> {
let text = text.trim();
let (negative, digits) = match text.strip_prefix('-') {
Some(rest) => (true, rest),
None => (false, text),
};
let (whole, fraction) = digits.split_once('.').unwrap_or((digits, ""));
if whole.is_empty() || !whole.bytes().all(|b| b.is_ascii_digit()) || !fraction.bytes().all(|b| b.is_ascii_digit()) {
return None;
}
let mut fraction = fraction.to_owned();
fraction.truncate(9);
while fraction.len() < 9 {
fraction.push('0');
}
let value = whole.parse::<i128>().ok()? * SECOND + fraction.parse::<i128>().ok()?;
Some(if negative { -value } else { value })
}
/// Days since 1970-01-01 as `YYYY-MM-DD` (proleptic Gregorian calendar).
fn date(days: i64) -> String {
let z = days + 719_468;
let era = z.div_euclid(146_097);
let doe = z.rem_euclid(146_097);
let yoe = (doe - doe / 1_460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let day = doy - (153 * mp + 2) / 5 + 1;
let month = if mp < 10 { mp + 3 } else { mp - 9 };
let year = yoe + era * 400 + i64::from(month <= 2);
if year < 0 { format!("-{:04}-{month:02}-{day:02}", -year) } else { format!("{year:04}-{month:02}-{day:02}") }
}
fn time(nanos_of_day: i128, scale: usize) -> String {
let seconds = nanos_of_day / SECOND;
let mut text = format!("{:02}:{:02}:{:02}", seconds / 3600, seconds / 60 % 60, seconds % 60);
if scale > 0 {
let fraction = format!("{:09}", nanos_of_day % SECOND);
text.push('.');
text.push_str(&fraction[..scale]);
}
text
}
fn timestamp(nanos: i128, scale: usize) -> String {
format!("{} {}", date(nanos.div_euclid(DAY) as i64), time(nanos.rem_euclid(DAY), scale))
}
/// `seconds[.fraction] offset`, the offset in minutes plus 1440.
fn timestamp_tz(text: &str, scale: usize) -> Option<String> {
let (instant, offset) = text.trim().split_once(' ')?;
let offset = offset.trim().parse::<i128>().ok()? - 1440;
let local = nanos(instant)? + offset * 60 * SECOND;
let sign = if offset < 0 { '-' } else { '+' };
Some(format!("{} {sign}{:02}:{:02}", timestamp(local, scale), offset.abs() / 60, offset.abs() % 60))
}
#[cfg(test)]
mod tests {
use super::*;
fn column(kind: &str, scale: Option<i64>) -> RowType {
RowType { name: "C".into(), kind: kind.into(), scale, ..Default::default() }
}
fn shown(kind: &str, scale: Option<i64>, value: &str) -> String {
cell(&column(kind, scale), Value::String(value.into())).unwrap()
}
#[test]
fn names_types_the_way_snowflake_spells_them() {
let named = |kind: &str, precision, scale, length| {
RowType { name: "C".into(), kind: kind.into(), precision, scale, length }.column().type_name
};
assert_eq!(named("fixed", Some(38), Some(0), None), "NUMBER(38,0)");
assert_eq!(named("fixed", Some(10), Some(2), None), "NUMBER(10,2)");
assert_eq!(named("real", None, None, None), "FLOAT");
assert_eq!(named("text", None, None, Some(100)), "VARCHAR(100)");
assert_eq!(named("text", None, None, Some(16_777_216)), "VARCHAR");
assert_eq!(named("text", None, None, Some(134_217_728)), "VARCHAR");
assert_eq!(named("timestamp_ntz", None, Some(9), None), "TIMESTAMP_NTZ(9)");
assert_eq!(named("timestamp_tz", None, Some(3), None), "TIMESTAMP_TZ(3)");
assert_eq!(named("time", None, Some(0), None), "TIME(0)");
for kind in ["boolean", "date", "variant", "object", "array", "binary", "geography", "vector"] {
assert_eq!(named(kind, None, None, None), kind.to_ascii_uppercase());
}
}
#[test]
fn null_and_plain_values_pass_through() {
assert_eq!(cell(&column("fixed", Some(0)), Value::Null), None);
assert_eq!(shown("fixed", Some(2), "12345678901234567890.12"), "12345678901234567890.12");
assert_eq!(shown("real", None, "1.5e-7"), "1.5e-7");
assert_eq!(shown("text", None, "a\nb"), "a\nb");
assert_eq!(shown("variant", None, "{\n \"a\": 1\n}"), "{\n \"a\": 1\n}");
assert_eq!(cell(&column("fixed", Some(0)), serde_json::json!(7)), Some("7".into()));
}
#[test]
fn booleans_and_binary() {
assert_eq!(shown("boolean", None, "1"), "true");
assert_eq!(shown("boolean", None, "true"), "true");
assert_eq!(shown("boolean", None, "0"), "false");
assert_eq!(shown("binary", None, "DEADBEEF"), "0xDEADBEEF");
}
#[test]
fn dates_count_days_from_the_epoch() {
assert_eq!(shown("date", None, "0"), "1970-01-01");
assert_eq!(shown("date", None, "19675"), "2023-11-14");
assert_eq!(shown("date", None, "-1"), "1969-12-31");
assert_eq!(shown("date", None, "11016"), "2000-02-29");
assert_eq!(shown("date", None, "-719162"), "0001-01-01");
assert_eq!(shown("date", None, "2932896"), "9999-12-31");
assert_eq!(shown("date", None, "not a number"), "not a number");
}
#[test]
fn times_and_timestamps_keep_their_scale() {
assert_eq!(shown("time", Some(0), "3723"), "01:02:03");
assert_eq!(shown("time", Some(3), "3723.5"), "01:02:03.500");
assert_eq!(shown("time", Some(9), "86399.999999999"), "23:59:59.999999999");
assert_eq!(shown("timestamp_ntz", Some(0), "1700000000"), "2023-11-14 22:13:20");
assert_eq!(shown("timestamp_ntz", Some(3), "1700000000.123000000"), "2023-11-14 22:13:20.123");
assert_eq!(shown("timestamp_ntz", Some(9), "1700000000.123456789"), "2023-11-14 22:13:20.123456789");
assert_eq!(shown("timestamp_ntz", Some(3), "-0.500"), "1969-12-31 23:59:59.500");
assert_eq!(shown("timestamp_ntz", Some(0), "-86401"), "1969-12-30 23:59:59");
assert_eq!(shown("timestamp_ltz", Some(0), "1700000000"), "2023-11-14 22:13:20 Z");
}
#[test]
fn zoned_timestamps_show_local_time_and_offset() {
assert_eq!(shown("timestamp_tz", Some(3), "1700000000.123 1440"), "2023-11-14 22:13:20.123 +00:00");
assert_eq!(shown("timestamp_tz", Some(0), "1700000000 1560"), "2023-11-15 00:13:20 +02:00");
assert_eq!(shown("timestamp_tz", Some(0), "1700000000 1110"), "2023-11-14 16:43:20 -05:30");
assert_eq!(shown("timestamp_tz", Some(0), "garbage"), "garbage");
}
}
File diff suppressed because it is too large. Load diff
+137
Переглянути файл
@@ -22,6 +22,7 @@ pub enum DbDriver {
Clickhouse,
/// Oracle Database (pure-Rust thin driver, no client library).
Oracle,
Snowflake,
}
impl DbDriver {
@@ -33,6 +34,7 @@ impl DbDriver {
DbDriver::SqlServer => Some(1433),
DbDriver::Clickhouse => Some(8123),
DbDriver::Oracle => Some(1521),
DbDriver::Snowflake => Some(443),
}
}
}
@@ -131,6 +133,37 @@ pub struct DbProperty {
pub value: String,
}
/// How a Snowflake data source signs in. The data source's one saved secret is what the
/// method needs: the private key's passphrase, the password, or the token.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase")]
pub enum DbSnowflakeAuth {
/// RSA key pair: a JWT signed with the private key file.
#[default]
KeyPair,
/// User name and password.
Password,
/// Programmatic access token.
Token,
/// OAuth access token.
OAuth,
/// Single sign-on in the web browser.
Browser,
}
/// Snowflake connection settings beyond the shared ones: the account identifier is
/// [`DataSource::host`], the schema [`DbOptions::default_schema`].
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, Type)]
#[serde(rename_all = "camelCase", default)]
pub struct DbSnowflake {
pub warehouse: String,
pub role: String,
pub auth: DbSnowflakeAuth,
/// PEM private key of [`DbSnowflakeAuth::KeyPair`]; `~/` is the home directory. Only the
/// path is stored: the key is read from the file at each login.
pub private_key_file: String,
}
/// How the SSH tunnel authenticates (shared with Settings › SSH Configurations).
pub use ide_core::remote_settings::SshAuth as DbSshAuth;
@@ -185,6 +218,8 @@ pub struct DataSource {
pub options: DbOptions,
pub introspection: DbIntrospection,
pub properties: Vec<DbProperty>,
/// Snowflake settings (`None`: not a Snowflake data source, or all defaults).
pub snowflake: Option<DbSnowflake>,
/// Stored app-wide and listed in every project. Implied by the file it is read from,
/// so not written to it.
pub global: bool,
@@ -222,6 +257,7 @@ impl Default for DataSource {
options: DbOptions::default(),
introspection: DbIntrospection::default(),
properties: Vec::new(),
snowflake: None,
global: false,
plugin_owner: None,
plugin_source_id: None,
@@ -273,6 +309,7 @@ pub fn scheme_driver(scheme: &str) -> Option<DbDriver> {
"sqlite" | "sqlite3" => Some(DbDriver::Sqlite),
"clickhouse" | "http" | "https" => Some(DbDriver::Clickhouse),
"oracle" | "oracledb" => Some(DbDriver::Oracle),
"snowflake" => Some(DbDriver::Snowflake),
_ => None,
}
}
@@ -375,6 +412,9 @@ impl DataSource {
}
let mut properties = Vec::new();
for (k, v) in u.params {
if s.driver == DbDriver::Snowflake && s.snowflake_url_parameter(&k, &v) {
continue;
}
match k.to_ascii_lowercase().as_str() {
"sslmode" | "ssl-mode" => {
s.ssl = match v.to_ascii_lowercase().replace('-', "_").as_str() {
@@ -405,6 +445,35 @@ impl DataSource {
Ok(s)
}
/// Applies a parameter of a Snowflake JDBC URL that names a connection setting.
fn snowflake_url_parameter(&mut self, name: &str, value: &str) -> bool {
let settings = self.snowflake.get_or_insert_with(DbSnowflake::default);
match name.to_ascii_lowercase().as_str() {
"db" | "database" => self.database = value.to_owned(),
"schema" => self.options.default_schema = value.to_owned(),
"warehouse" => settings.warehouse = value.to_owned(),
"role" => settings.role = value.to_owned(),
"private_key_file" => settings.private_key_file = value.to_owned(),
"authenticator" => {
settings.auth = match value.to_ascii_lowercase().as_str() {
"snowflake_jwt" => DbSnowflakeAuth::KeyPair,
"externalbrowser" => DbSnowflakeAuth::Browser,
"oauth" => DbSnowflakeAuth::OAuth,
"programmatic_access_token" => DbSnowflakeAuth::Token,
"snowflake" => DbSnowflakeAuth::Password,
_ => return false,
}
}
_ => return false,
}
true
}
/// The Snowflake settings (defaults when none were saved).
pub fn snowflake(&self) -> DbSnowflake {
self.snowflake.clone().unwrap_or_default()
}
/// The password written in the URL of a [`DataSource::url_only`] data source.
pub fn url_password(&self) -> Option<String> {
if !self.url_only {
@@ -613,4 +682,72 @@ mod tests {
let error = parse_url("clickhouse://reader:[email protected]:invalid/default").unwrap_err();
assert!(!error.message.contains("secret"));
}
#[test]
fn snowflake_settings_round_trip_and_stay_out_of_other_sources() {
let tmp = tempfile::tempdir().unwrap();
let snowflake = DataSource {
id: "sf".into(),
name: "warehouse".into(),
driver: DbDriver::Snowflake,
host: "myorg-myaccount".into(),
user: "jane".into(),
database: "ANALYTICS".into(),
save_password: true,
snowflake: Some(DbSnowflake {
warehouse: "COMPUTE_WH".into(),
role: "ANALYST".into(),
auth: DbSnowflakeAuth::KeyPair,
private_key_file: "~/.ssh/rsa_key.p8".into(),
}),
..Default::default()
};
let other = DataSource { id: "pg".into(), name: "pg".into(), driver: DbDriver::Postgres, ..Default::default() };
save(tmp.path(), &[snowflake.clone(), other.clone()]).unwrap();
assert_eq!(load(tmp.path()).unwrap(), vec![snowflake, other]);
let text = std::fs::read_to_string(project_dir::path(tmp.path(), FILE)).unwrap();
assert_eq!(text.matches("snowflake]").count(), 1, "{text}");
assert!(text.contains("driver = \"snowflake\""), "{text}");
assert!(text.contains("auth = \"keyPair\""), "{text}");
assert!(text.contains("privateKeyFile = \"~/.ssh/rsa_key.p8\""), "{text}");
assert_eq!(DbDriver::Snowflake.default_port(), Some(443));
assert_eq!(DataSource::default().snowflake(), DbSnowflake::default());
}
#[test]
fn snowflake_jdbc_urls_name_the_session_settings() {
let s = DataSource {
driver: DbDriver::Snowflake,
url_only: true,
url: "jdbc:snowflake://myorg-myaccount.snowflakecomputing.com:443/?warehouse=COMPUTE_WH&db=ANALYTICS\
&schema=PUBLIC&role=ANALYST&user=jane&authenticator=snowflake_jwt\
&private_key_file=%7E%2Fkeys%2Frsa_key.p8&QUERY_TAG=ide"
.into(),
..Default::default()
};
let r = s.resolved().unwrap();
assert_eq!(r.host, "myorg-myaccount.snowflakecomputing.com");
assert_eq!(r.port, Some(443));
assert_eq!(r.user, "jane");
assert_eq!(r.database, "ANALYTICS");
assert_eq!(r.options.default_schema, "PUBLIC");
assert_eq!(
r.snowflake(),
DbSnowflake {
warehouse: "COMPUTE_WH".into(),
role: "ANALYST".into(),
auth: DbSnowflakeAuth::KeyPair,
private_key_file: "~/keys/rsa_key.p8".into(),
}
);
assert_eq!(r.properties, vec![DbProperty { name: "QUERY_TAG".into(), value: "ide".into() }]);
let browser = DataSource {
url: "snowflake://xy12345.eu-central-1.snowflakecomputing.com/DB?authenticator=externalbrowser".into(),
..s
};
let r = browser.resolved().unwrap();
assert_eq!(r.database, "DB");
assert_eq!(r.snowflake().auth, DbSnowflakeAuth::Browser);
assert_eq!(scheme_driver("snowflake"), Some(DbDriver::Snowflake));
}
}
+137 -2
Переглянути файл
@@ -32,6 +32,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let clickhouse = driver == DbDriver::Clickhouse;
// PL/SQL: `;` inside BEGIN…END / DECLARE…END blocks does not terminate a statement.
let plsql = driver == DbDriver::Oracle;
let snowflake = driver == DbDriver::Snowflake;
let chars: Vec<(usize, char)> = text.char_indices().collect();
// UTF-16 offset of each char index (plus the end).
let mut u16_at = Vec::with_capacity(chars.len() + 1);
@@ -57,6 +58,10 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
let mut block = false;
// `CREATE PACKAGE` / `TYPE BODY`: only `/` on its own line (or the end) ends it.
let mut package = false;
// The segment is a Snowflake Scripting block (`DECLARE … BEGIN … END`, `BEGIN … END`).
let mut script = false;
// The segment may carry a Scripting block as its body (`CREATE TASK … AS BEGIN … END`).
let mut defines = false;
let push = |out: &mut Vec<SqlStatement>, s: usize, e: usize, has_code: bool| {
if !has_code {
return;
@@ -91,11 +96,29 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
// Re-read this character as code (it may start BEGIN / DECLARE).
continue;
}
_ if snowflake && !has_code && c.is_alphabetic() => {
if word_at(&chars, i, "DECLARE") {
(script, block, declarations) = (true, true, true);
} else if word_at(&chars, i, "BEGIN") {
// `BEGIN;`, `BEGIN TRANSACTION`… open a transaction; anything else a block.
let after = skip_ws(&chars, i + 5);
let transaction = chars.get(after).is_none_or(|&(_, c)| c == ';')
|| ["TRANSACTION", "WORK", "NAME"].into_iter().any(|w| word_at(&chars, after, w));
(script, block) = (!transaction, !transaction);
} else {
defines = ["CREATE", "WITH", "EXECUTE"].into_iter().any(|w| word_at(&chars, i, w));
}
has_code = true;
// Re-read this character as code (it starts BEGIN / DECLARE).
continue;
}
';' if depth == 0 && !declarations && !package => {
push(&mut out, start, if block { i + 1 } else { i }, has_code);
start = i + 1;
has_code = false;
block = false;
script = false;
defines = false;
}
';' => has_code = true,
'\'' => {
@@ -132,7 +155,16 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
i += 1;
}
'#' if mysql => state = State::LineComment,
'$' if driver == DbDriver::Postgres => {
// The `//` of `file://…` (PUT, GET) or another URL is not a comment.
'/' if snowflake && next == Some('/') && chars[..i].last().is_some_and(|&(_, c)| c == ':') => {
has_code = true;
i += 1;
}
'/' if snowflake && next == Some('/') => {
state = State::LineComment;
i += 1;
}
'$' if driver == DbDriver::Postgres || snowflake => {
// `$tag$ … $tag$` (tag may be empty); `$1` parameters are not quotes.
let rest = &text[byte_at(i)..];
let tag_len = rest[1..].find('$').filter(|&n| {
@@ -151,6 +183,31 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
continue;
}
}
// An unquoted body: `… AS BEGIN … END`, `… AS DECLARE … BEGIN … END`, an alert's
// `THEN BEGIN … END`. A column named `begin` is followed by `,`, `)`, `;` or FROM.
_ if defines && !script && depth == 0 && scripting_body(&chars, i) => {
script = true;
let body = skip_ws(&chars, i + if word_at(&chars, i, "AS") { 2 } else { 4 });
declarations = word_at(&chars, body, "DECLARE");
i = body - 1;
}
_ if script && word_at(&chars, i, "BEGIN") => {
depth += 1;
declarations = false;
}
_ if script && word_at(&chars, i, "CASE") => depth += 1,
_ if script && word_at(&chars, i, "END") => {
// `END IF` / `FOR` / `WHILE` / `LOOP` / `REPEAT` close statements that opened no
// block; `END` and `END CASE` close what BEGIN and CASE opened.
let after = skip_ws(&chars, i + 3);
let closing = ["IF", "FOR", "WHILE", "LOOP", "REPEAT", "CASE"]
.into_iter()
.find(|w| word_at(&chars, after, w));
if matches!(closing, None | Some("CASE")) && depth > 0 {
depth -= 1;
}
i = closing.map_or(i + 3, |w| after + w.len()) - 1;
}
_ if plsql && word_at(&chars, i, "BEGIN") => {
depth += 1;
declarations = false;
@@ -183,7 +240,7 @@ pub fn split(text: &str, driver: DbDriver) -> Vec<SqlStatement> {
_ => {}
},
State::Single => {
if (mysql || clickhouse) && c == '\\' {
if (mysql || clickhouse || snowflake) && c == '\\' {
i += 1;
} else if c == '\'' {
if next == Some('\'') {
@@ -281,6 +338,20 @@ fn word_at(chars: &[(usize, char)], i: usize, word: &str) -> bool {
&& chars.get(i + word.len()).is_none_or(|&(_, c)| !c.is_alphanumeric() && c != '_' && c != '$')
}
/// Whether `AS` or `THEN` at `i` is followed by a Snowflake Scripting block.
fn scripting_body(chars: &[(usize, char)], i: usize) -> bool {
let Some(word) = ["AS", "THEN"].into_iter().find(|w| word_at(chars, i, w)) else { return false };
let body = skip_ws(chars, i + word.len());
if word_at(chars, body, "DECLARE") {
return true;
}
if !word_at(chars, body, "BEGIN") {
return false;
}
let after = skip_ws(chars, body + 5);
chars.get(after).is_some_and(|&(_, c)| !matches!(c, ',' | ')' | ';')) && !word_at(chars, after, "FROM")
}
/// Whether only whitespace follows `i` until the end of its line.
fn ends_line(chars: &[(usize, char)], i: usize) -> bool {
chars[i + 1..].iter().take_while(|&&(_, c)| c != '\n').all(|&(_, c)| c.is_whitespace())
@@ -467,4 +538,68 @@ mod tests {
vec!["create view v as select 1 x from dual", "create type t as object (a number)", "select 4"]
);
}
#[test]
fn snowflake_bodies_comments_and_escapes_keep_statements_intact() {
let procedure = "create procedure p() returns string language javascript as $$ var a = 1; return 'x;y'; $$";
assert_eq!(
texts(
&format!("{procedure};\nselect 2; // done; really\nselect $1, 'it\\'s;here' from @stage;"),
DbDriver::Snowflake
),
vec![procedure, "select 2", "// done; really\nselect $1, 'it\\'s;here' from @stage"]
);
assert_eq!(texts("select 1 / 2; select 3", DbDriver::Snowflake), vec!["select 1 / 2", "select 3"]);
}
#[test]
fn snowflake_unquoted_bodies_and_urls_keep_statements_intact() {
let task = "create task t warehouse = w as\nbegin\n insert into a values (1);\n delete from b;\nend";
assert_eq!(texts(&format!("{task};\nselect 2;"), DbDriver::Snowflake), vec![task, "select 2"]);
let procedure = "create procedure p() returns number language sql as\ndeclare\n n number default 1;\nbegin\n return n;\nend";
assert_eq!(texts(&format!("{procedure};\nselect 2"), DbDriver::Snowflake), vec![procedure, "select 2"]);
let alert =
"create alert a schedule = '1 minute' if (exists (select 1)) then begin insert into h values (1); end";
assert_eq!(texts(&format!("{alert}; select 2"), DbDriver::Snowflake), vec![alert, "select 2"]);
// A column or alias named `begin` opens nothing.
for plain in [
"create view v as select 1 as begin, 2 as n",
"create view v as select a as begin from t",
"with x as (select 1 as begin) select * from x",
] {
assert_eq!(texts(&format!("{plain}; select 2"), DbDriver::Snowflake), vec![plain, "select 2"]);
}
assert_eq!(
texts(
"put file:///tmp/a.csv @s; get @s/a.csv file:///tmp/; select 1 // note; here\n; select 2",
DbDriver::Snowflake
),
vec!["put file:///tmp/a.csv @s", "get @s/a.csv file:///tmp/", "select 1 // note; here", "select 2"]
);
}
#[test]
fn snowflake_scripting_blocks_are_one_statement_and_transactions_are_not() {
let block = "DECLARE\n total NUMBER DEFAULT 0;\n c CURSOR FOR SELECT n FROM t;\nBEGIN\n FOR r IN c DO\n \
IF (r.n > 1) THEN total := total + r.n; END IF;\n END FOR;\n \
LET label := CASE WHEN total > 9 THEN 'big' ELSE 'small' END;\n \
BEGIN\n INSERT INTO log VALUES (:label);\n EXCEPTION WHEN OTHER THEN RETURN 'failed';\n END;\n \
RETURN total;\nEND;";
assert_eq!(texts(&format!("{block}\nselect 2;"), DbDriver::Snowflake), vec![block, "select 2"]);
let plain = "BEGIN\n LET x := 1;\n WHILE (x < 3) DO x := x + 1; END WHILE;\n RETURN x;\nEND;";
assert_eq!(
texts(&format!("select 1;\n{plain}\nselect 3"), DbDriver::Snowflake),
vec!["select 1", plain, "select 3"]
);
for begin in ["begin", "BEGIN TRANSACTION", "begin work", "BEGIN NAME t1"] {
assert_eq!(
texts(&format!("{begin};\ninsert into t values (1);\ncommit;"), DbDriver::Snowflake),
vec![begin, "insert into t values (1)", "commit"]
);
}
assert_eq!(
texts("select case when a then 1 end, b from t; select 2", DbDriver::Snowflake),
vec!["select case when a then 1 end, b from t", "select 2"]
);
}
}
+93 -7
Переглянути файл
@@ -16,7 +16,7 @@ pub fn quote_ident(driver: DbDriver, name: &str) -> String {
pub fn quote_literal(driver: DbDriver, value: &str) -> String {
let mut s = value.replace('\'', "''");
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse) {
if matches!(driver, DbDriver::Mysql | DbDriver::Clickhouse | DbDriver::Snowflake) {
s = s.replace('\\', "\\\\");
}
format!("'{s}'")
@@ -75,12 +75,19 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
DbDriver::Clickhouse => {
"SELECT name FROM system.databases WHERE lower(name) NOT IN ('system', 'information_schema') ORDER BY name"
}
// `information_schema.schemata` lists only owned schemas before PostgreSQL 10 (and on servers
// built on such a core): a read-only account would see none. Any schema it may use is listed.
DbDriver::Postgres if system => {
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
"SELECT nspname FROM pg_catalog.pg_namespace \
WHERE nspname NOT LIKE 'pg\\_toast%' AND nspname NOT LIKE 'pg\\_temp%' \
AND has_schema_privilege(nspname, 'USAGE') ORDER BY 1"
}
DbDriver::Mysql if system => "SELECT schema_name FROM information_schema.schemata ORDER BY 1",
DbDriver::SqlServer if system => "SELECT schema_name FROM information_schema.schemata ORDER BY 1",
DbDriver::Snowflake if system => "SELECT schema_name FROM information_schema.schemata ORDER BY 1",
DbDriver::Snowflake => {
"SELECT schema_name FROM information_schema.schemata WHERE schema_name <> 'INFORMATION_SCHEMA' ORDER BY 1"
}
DbDriver::Oracle if system => "SELECT username FROM all_users ORDER BY username",
DbDriver::Oracle => {
"SELECT username FROM all_users WHERE username NOT IN \
@@ -88,9 +95,10 @@ pub fn schemas(driver: DbDriver, system: bool) -> &'static str {
'OUTLN', 'SYS', 'SYSTEM', 'XDB', 'XS$NULL') ORDER BY username"
}
DbDriver::Postgres => {
"SELECT schema_name FROM information_schema.schemata \
WHERE schema_name NOT IN ('pg_catalog', 'information_schema') \
AND schema_name NOT LIKE 'pg\\_toast%' AND schema_name NOT LIKE 'pg\\_temp%' ORDER BY 1"
"SELECT nspname FROM pg_catalog.pg_namespace \
WHERE nspname NOT IN ('pg_catalog', 'information_schema') \
AND nspname NOT LIKE 'pg\\_toast%' AND nspname NOT LIKE 'pg\\_temp%' \
AND has_schema_privilege(nspname, 'USAGE') ORDER BY 1"
}
DbDriver::Mysql => {
"SELECT schema_name FROM information_schema.schemata \
@@ -117,6 +125,11 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\' ORDER BY name",
quote_ident(driver, schema)
),
DbDriver::Snowflake => format!(
"SELECT table_name, CASE table_type WHEN 'VIEW' THEN 'view' WHEN 'MATERIALIZED VIEW' THEN 'materialized view' \
ELSE 'table' END FROM information_schema.tables WHERE table_schema = {} ORDER BY 1",
quote_literal(driver, schema)
),
DbDriver::Oracle => format!(
"SELECT table_name, 'table' FROM all_tables WHERE owner = {} \
UNION ALL SELECT view_name, 'view' FROM all_views WHERE owner = {} ORDER BY 1",
@@ -142,6 +155,11 @@ pub fn tables(driver: DbDriver, schema: &str) -> String {
}
}
/// A Snowflake column type with its size, from `information_schema.columns`.
const SNOWFLAKE_TYPE: &str = "CASE WHEN data_type = 'NUMBER' THEN 'NUMBER(' || numeric_precision || ',' || numeric_scale || ')' \
WHEN data_type = 'TEXT' THEN 'VARCHAR(' || character_maximum_length || ')' \
WHEN data_type LIKE 'TIME%' THEN data_type || '(' || datetime_precision || ')' ELSE data_type END";
/// Rows: `name, type, nullable (YES/NO or 0/1), default, is_pk (0/1)`.
pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
@@ -182,6 +200,10 @@ pub fn columns(driver: DbDriver, schema: &str, table: &str) -> String {
FROM information_schema.columns c WHERE c.table_schema = {s} AND c.table_name = {t} \
ORDER BY c.ordinal_position"
),
DbDriver::Snowflake => format!(
"SELECT column_name, {SNOWFLAKE_TYPE}, is_nullable, column_default, 0 \
FROM information_schema.columns WHERE table_schema = {s} AND table_name = {t} ORDER BY ordinal_position"
),
DbDriver::Oracle => format!(
"SELECT c.column_name, c.data_type, DECODE(c.nullable, 'Y', 'YES', 'NO'), c.data_default, \
CASE WHEN k.column_name IS NOT NULL THEN 1 ELSE 0 END \
@@ -230,6 +252,7 @@ pub fn indexes(driver: DbDriver, schema: &str, table: &str) -> String {
WHERE OBJECT_SCHEMA_NAME(i.object_id) = {s} AND OBJECT_NAME(i.object_id) = {t} AND i.name IS NOT NULL \
ORDER BY i.name, ic.key_ordinal"
),
DbDriver::Snowflake => "SELECT '', 0, '' WHERE 1 = 0".to_owned(),
DbDriver::Oracle => format!(
"SELECT i.index_name, CASE WHEN i.uniqueness = 'UNIQUE' THEN 1 ELSE 0 END, ic.column_name \
FROM all_indexes i JOIN all_ind_columns ic \
@@ -296,6 +319,15 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
WHERE c.table_schema NOT IN ('sys', 'information_schema') \
ORDER BY c.table_schema, c.table_name, c.ordinal_position"
}
DbDriver::Snowflake => {
"SELECT c.table_schema, c.table_name, CASE t.table_type WHEN 'VIEW' THEN 'view' \
WHEN 'MATERIALIZED VIEW' THEN 'materialized view' ELSE 'table' END, c.column_name, c.data_type, \
c.is_nullable, 0, CASE WHEN c.column_default IS NOT NULL THEN 1 ELSE 0 END \
FROM information_schema.columns c JOIN information_schema.tables t \
ON t.table_schema = c.table_schema AND t.table_name = c.table_name \
WHERE c.table_schema <> 'INFORMATION_SCHEMA' \
ORDER BY c.table_schema, c.table_name, c.ordinal_position LIMIT 50000"
}
DbDriver::Oracle => {
"SELECT c.owner, c.table_name, CASE WHEN v.view_name IS NOT NULL THEN 'view' ELSE 'table' END, \
c.column_name, c.data_type, DECODE(c.nullable, 'Y', 'YES', 'NO'), \
@@ -320,6 +352,7 @@ pub fn completion_columns(driver: DbDriver) -> &'static str {
pub fn foreign_keys(driver: DbDriver) -> &'static str {
match driver {
DbDriver::Clickhouse => "SELECT '', '', '', '', '', '', '' WHERE 0",
DbDriver::Snowflake => "SELECT '', '', '', '', '', '', '' WHERE 1 = 0",
DbDriver::Sqlite => {
"SELECT 'main', m.name, f.id, f.\"from\", 'main', f.\"table\", f.\"to\" \
FROM sqlite_master m JOIN pragma_foreign_key_list(m.name) f \
@@ -370,7 +403,7 @@ pub fn foreign_keys(driver: DbDriver) -> &'static str {
pub fn table_foreign_keys(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => foreign_keys(driver).to_owned(),
DbDriver::Clickhouse | DbDriver::Snowflake => foreign_keys(driver).to_owned(),
DbDriver::Sqlite => format!(
"SELECT {s}, {t}, f.id, f.\"from\", {s}, f.\"table\", f.\"to\" FROM pragma_foreign_key_list({t}, {s}) f ORDER BY f.id, f.seq"
),
@@ -398,6 +431,7 @@ pub fn current_schema(driver: DbDriver) -> Option<&'static str> {
DbDriver::Mysql => Some("SELECT DATABASE()"),
DbDriver::SqlServer => Some("SELECT DB_NAME()"),
DbDriver::Oracle => Some("SELECT SYS_CONTEXT('USERENV', 'CURRENT_SCHEMA') FROM DUAL"),
DbDriver::Snowflake => Some("SELECT CURRENT_SCHEMA()"),
}
}
@@ -410,6 +444,7 @@ pub fn use_schema(driver: DbDriver, schema: &str) -> Option<String> {
DbDriver::Postgres => Some(format!("SET search_path TO {}", quote_ident(driver, schema))),
DbDriver::Mysql | DbDriver::SqlServer => Some(format!("USE {}", quote_ident(driver, schema))),
DbDriver::Oracle => Some(format!("ALTER SESSION SET CURRENT_SCHEMA = {}", quote_ident(driver, schema))),
DbDriver::Snowflake => Some(format!("USE SCHEMA {}", quote_ident(driver, schema))),
}
}
@@ -520,6 +555,9 @@ pub fn edit_statement(driver: DbDriver, schema: &str, table: &str, edit: &DbRowE
DbDriver::Mysql => format!("INSERT INTO {target} () VALUES ()"),
// Oracle has no DEFAULT VALUES; without columns there is nothing to insert.
DbDriver::Oracle => return Err("Oracle needs at least one column value to insert".into()),
DbDriver::Snowflake => {
return Err("Snowflake needs at least one column value to insert".into());
}
_ => format!("INSERT INTO {target} DEFAULT VALUES"),
});
}
@@ -588,6 +626,7 @@ pub fn triggers(driver: DbDriver, schema: &str, table: &str) -> String {
let (s, t) = (quote_literal(driver, schema), quote_literal(driver, table));
match driver {
DbDriver::Clickhouse => "SELECT '', '', '' WHERE 0".to_owned(),
DbDriver::Snowflake => "SELECT '', '', '' WHERE 1 = 0".to_owned(),
DbDriver::Sqlite => format!(
"SELECT name, sql FROM {}.sqlite_master WHERE type = 'trigger' AND tbl_name = {t} ORDER BY name",
quote_ident(driver, schema)
@@ -656,6 +695,12 @@ pub fn routines(driver: DbDriver, schema: &str) -> Option<String> {
"SELECT r.routine_name, lower(r.routine_type), '', r.data_type \
FROM information_schema.routines r WHERE r.routine_schema = {s} ORDER BY 1"
)),
DbDriver::Snowflake => Some(format!(
"SELECT function_name, 'function', argument_signature, data_type FROM information_schema.functions \
WHERE function_schema = {s} UNION ALL \
SELECT procedure_name, 'procedure', argument_signature, data_type FROM information_schema.procedures \
WHERE procedure_schema = {s} ORDER BY 1, 3"
)),
DbDriver::Oracle => Some(format!(
"SELECT object_name, CASE object_type WHEN 'PROCEDURE' THEN 'procedure' ELSE 'function' END, '', NULL \
FROM all_objects WHERE owner = {s} AND object_type IN ('PROCEDURE', 'FUNCTION') \
@@ -697,6 +742,10 @@ pub fn sequences(driver: DbDriver, schema: &str) -> Option<String> {
"SELECT sequence_name, data_type FROM information_schema.sequences WHERE sequence_schema = {} ORDER BY 1",
quote_literal(driver, schema)
)),
DbDriver::Snowflake => Some(format!(
"SELECT sequence_name, data_type FROM information_schema.sequences WHERE sequence_schema = {} ORDER BY 1",
quote_literal(driver, schema)
)),
// Every sequence is numeric; ALL_SEQUENCES has no data_type column before 18c.
DbDriver::Oracle => Some(format!(
"SELECT sequence_name, 'NUMBER' FROM all_sequences WHERE sequence_owner = {} ORDER BY 1",
@@ -930,6 +979,17 @@ mod tests {
assert!(edit_statement(DbDriver::Postgres, "s", "t", &DbRowEdit::Delete { key: vec![] }).is_err());
}
#[test]
fn postgres_lists_the_schemas_the_user_may_use() {
for system in [false, true] {
let sql = schemas(DbDriver::Postgres, system);
assert!(sql.contains("FROM pg_catalog.pg_namespace"), "{sql}");
assert!(sql.contains("has_schema_privilege(nspname, 'USAGE')"), "{sql}");
assert!(sql.contains("NOT LIKE 'pg\\_toast%'"), "{sql}");
assert_eq!(sql.contains("NOT IN ('pg_catalog', 'information_schema')"), !system, "{sql}");
}
}
#[test]
fn sqlserver_quoting_and_schema() {
assert_eq!(quote_ident(DbDriver::SqlServer, "a]b"), "[a]]b]");
@@ -1086,4 +1146,30 @@ mod tests {
"create table \"public\".\"t\"\n(\n \"id\" integer not null,\n \"name\" character varying(40) default 'x'::character varying,\n constraint \"t_pkey\" PRIMARY KEY (id)\n);\n\nCREATE INDEX t_name ON public.t USING btree (name);\n"
);
}
#[test]
fn snowflake_quotes_names_and_reads_its_information_schema() {
let driver = DbDriver::Snowflake;
assert_eq!(quote_ident(driver, "My \"Table\""), "\"My \"\"Table\"\"\"");
assert_eq!(quote_literal(driver, "it's a \\ path"), "'it''s a \\\\ path'");
assert_eq!(qualified(driver, "PUBLIC", "ORDERS"), "\"PUBLIC\".\"ORDERS\"");
assert_eq!(current_schema(driver), Some("SELECT CURRENT_SCHEMA()"));
assert_eq!(use_schema(driver, "Sales"), Some("USE SCHEMA \"Sales\"".into()));
assert!(schemas(driver, true).contains("information_schema.schemata"));
assert!(schemas(driver, false).contains("<> 'INFORMATION_SCHEMA'"));
assert!(tables(driver, "PUBLIC").contains("table_schema = 'PUBLIC'"));
let columns = columns(driver, "PUBLIC", "ORDERS");
assert!(columns.contains("'NUMBER(' || numeric_precision"), "{columns}");
assert!(columns.contains("table_name = 'ORDERS' ORDER BY ordinal_position"), "{columns}");
assert!(completion_columns(driver).contains("LIMIT 50000"));
assert!(routines(driver, "PUBLIC").unwrap().contains("information_schema.procedures"));
assert!(sequences(driver, "PUBLIC").unwrap().contains("information_schema.sequences"));
assert_eq!(
table_page(driver, "PUBLIC", "ORDERS", None, &[], 50, 25),
"SELECT * FROM \"PUBLIC\".\"ORDERS\" LIMIT 26 OFFSET 50"
);
assert!(edit_statement(driver, "PUBLIC", "T", &DbRowEdit::Insert { values: vec![] }).is_err());
assert_eq!(crate::conn::begin_statement(driver), "BEGIN");
assert_eq!(crate::plan::explain_statement(driver, "select 1;"), "EXPLAIN USING TABULAR select 1");
}
}
+4 -1
Переглянути файл
@@ -143,9 +143,12 @@ mod tests {
std::fs::create_dir_all(root.join(".openidea")).unwrap();
let (count, on_change) = counting();
let _watch = SourcesWatch::start(&root, None, on_change).unwrap();
// FSEvents can deliver the creation of `.openidea/` above after the watch started.
std::thread::sleep(Duration::from_secs(1));
let seen = count.load(Ordering::SeqCst);
std::fs::write(root.join(".openidea/settings.toml"), "").unwrap();
std::fs::write(root.join("main.py"), "").unwrap();
std::thread::sleep(SETTLE * 3);
assert_eq!(count.load(Ordering::SeqCst), 0);
assert_eq!(count.load(Ordering::SeqCst), seen);
}
}
+24
Переглянути файл
@@ -0,0 +1,24 @@
Throwaway keys for the Snowflake key-pair tests. They were generated for this repository,
are registered with no account, and protect nothing.
| File | Format |
|---|---|
| `rsa_key.p8` | PKCS#8, unencrypted, RSA 2048 |
| `rsa_key_encrypted.p8` | PKCS#8, PBES2 (PBKDF2 + AES-256-CBC), passphrase `test-passphrase` |
| `rsa_key_scrypt.p8` | PKCS#8, PBES2 (scrypt + AES-256-CBC), passphrase `test-passphrase` |
| `rsa_key_pkcs1.pem` | PKCS#1 (`BEGIN RSA PRIVATE KEY`), unencrypted |
| `rsa_key_legacy_encrypted.pem` | PKCS#1 with `Proc-Type: 4,ENCRYPTED` (rejected) |
| `rsa_key_1024.p8` | RSA 1024 (rejected: too short) |
| `ec_key.p8` | EC P-256 (rejected: not RSA) |
| `rsa_key.pub` | Public key of `rsa_key.p8` |
Fingerprint of `rsa_key.pub` from OpenSSL
(`openssl rsa -pubin -in rsa_key.pub -outform DER | openssl dgst -sha256 -binary | openssl enc -base64`):
`ajjL6Bf6l1FIIyTcJli4i2AWPYDVJGQnSZr+9q9ClgI=`
`commands.json` holds one sample statement for every command of Snowflake's SQL reference
(<https://docs.snowflake.com/en/sql-reference/sql-all>, 786 names). The driver's tests
(`src/snowflake/commands.rs`) and the console's (`src/features/database/sql/snowflakeCommands.test.ts`)
run all of them; the ignored live test runs the listing ones against a real account. `reads`: allowed
on a read-only data source. `rerunnable`: safe to run again for another page of the result. The
samples exercise the client, not Snowflake's grammar, and name no real object.
File diff suppressed because it is too large. Load diff
+5
Переглянути файл
@@ -0,0 +1,5 @@
-----BEGIN PRIVATE KEY-----
MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgwYBQrdPGgoV9c58S
q6Fc5aso2M4wVyXE9WxrpPzcrHahRANCAAR0SjYZ1Pvc5jG2ll+pTSdBiVRccnSB
BirgCQL5f1g4CuEK4YNqTIvXMqJ2bBVx6f2MTbLeCmu8EebiJ99dI/ZM
-----END PRIVATE KEY-----
+28
Переглянути файл
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDPfjbU88uYwtpN
hthRdv7h1cJJ1p2nnb8pKLMi/SdPSfSZLm0eA2Tbjp6HtGqv91enrCPyy++hQOl0
KtfoPGbzDqxnzZPK4BtckmxHiBQifG7A0d3jpq8Y2JjIkkMItnF7kXQob89TOaFj
+pW8/XH6Y+w5cbMYdkf3RXKjZpYRLRBxnzAl4lMhP/boE7reoWuBIwPFVXnX8AVP
p0bjkpR2I1fgUKZxMw/mbRhWVy/9b7Vh79RwPG85rvCia2e7OdDLxNYUcEHocvfb
8phUHtI7ZvnCJcbXJX7ON9/JdU/DN9RHc+az1ZjB4Za1cHoLXlbnUisSf1e3tnaY
uZnGqGLPAgMBAAECggEAQ+8N4sEDSruZH8LjdiCgqK4E04oJygQyvzYNGIqL4UDU
bYauwSHZVq0mKEYgnAY+U65fgrHUfhYzmQ4h7G5CQpfuSHodRlIDg1aJTw41kWju
FgFgRZWgRGE9/IUun16kXxGrV0XsOlJnDcXQWrUcKR+oCtK4EW1AeKvrMaZ3wZ1J
i0xqMlU5bk77kvskBRL4k51V3gOfDoflus2nHpES8yZSAP1uiivbtTOXYB2dRHuD
omB4KlscUdIwT9vTMC7kxECEKDwd5fxJkdIJ/NsBi98tOZS0QrH7y7FrBA3K7gLY
VJSIckKP8rz3kUAbW9vYV2dRwxIMQj2kZJEKa0kA6QKBgQDsNsJPJdt5BcAPYjV0
1LX91ooLCDYux5yUNEno2rmQ/qv5g0d12L94aASklGIIfJV75dXmwa8q4erZU0hs
9/XhyUnjDlBLd6oEztDjDA49J57g2nPmzrOtjugucH7Qe8EAJzz+O0PbtkcQzRmV
8vnYptWHvEKmnjh8KOb5wOMVFQKBgQDg35SYkiys8V3tQApbp6U7Q3fq3vbVu+2A
avBYIE0qgG9fhFrQdihVX1MZ3L4TtXtiikMPO8MOGBjSgN/qbelChhkQV0l+CIf0
nOo1SeoEXp16q1NcZTEohnbaboMjWrkl6Br6SUNvEt7Y63UOi62E+ll50ye8wKgA
yBHrGXOZUwKBgQDoTaIivJRvHNyZRHhWAatJjGgOGXFP6Ddw+UuGmTTOB+eRZy4s
E5GaFJIcWWFf/0Xzp0EJw4lFnxCb0m/z2pgSCSeFCVqmHXUzbUG5SO3n7sMtVymM
39OM6HIm2bOdzVctk2lEjXxa45RYTZzP/zj/fHObHeZeflQGpyMglDW/UQKBgQDE
wRuFOzC3ySCDu6rUcyTRa8D60kdqz1mJKQhOM0URUPshue6zww6bVCKj2E6/c+UX
XR47r/CE3nSpinVg6vZTbH11nMoC6nf6gWkzYw7M7/QeJtVzXF7ctHo9uMApKPIi
xqkEISJNh1dQuQPczD3wwhOT0jtMnczhkiyLZeONXQKBgE4wo4IkUhH2SQRiu8Je
sg18oa1sABaL6bCQ7vqJ2+cZzj5Pz0WxMoyc7ZJlIiKt2joGQwqmRAqVohHpJylm
9RKvymaM0BGVKkAASfwyEf15YvZQx/9IK3uWpRoGrlu7jyu0UlzCgpHXfOu1FKCx
DsVT35WOOVJXuDrNhpYAdg5U
-----END PRIVATE KEY-----
+9
Переглянути файл
@@ -0,0 +1,9 @@
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAz3421PPLmMLaTYbYUXb+
4dXCSdadp52/KSizIv0nT0n0mS5tHgNk246eh7Rqr/dXp6wj8svvoUDpdCrX6Dxm
8w6sZ82TyuAbXJJsR4gUInxuwNHd46avGNiYyJJDCLZxe5F0KG/PUzmhY/qVvP1x
+mPsOXGzGHZH90Vyo2aWES0QcZ8wJeJTIT/26BO63qFrgSMDxVV51/AFT6dG45KU
diNX4FCmcTMP5m0YVlcv/W+1Ye/UcDxvOa7womtnuznQy8TWFHBB6HL32/KYVB7S
O2b5wiXG1yV+zjffyXVPwzfUR3Pms9WYweGWtXB6C15W51IrEn9Xt7Z2mLmZxqhi
zwIDAQAB
-----END PUBLIC KEY-----
+16
Переглянути файл
@@ -0,0 +1,16 @@
-----BEGIN PRIVATE KEY-----
MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBANqRvj6Xi55mKH0x
96UVh/4tMgBb+5TVT5MN39diDr6iP7vFycEnnHDVXaeQJBog+P7NiYC7XY+7yUL3
2J3VxMBzGirm5Sj12uU1C+U4f0Nxx4nVKqonQD8GHJGJxypSZvIculFV9CYSHfoc
JYHR95k1Ce7echs7VnXfjvaOBdaLAgMBAAECgYBYecC6E5xn7VCchTqK7/FR2bew
34syjzqb+3TN+hqf3ovQFdt6ve8c2bIEn1uN3CUO9+KeYroX8dshqbRdB+iRgx8i
EzHAmKLkQj/QF7j1bTLD3zMq9XmUAOaUQwtlzRKSJIOa6EoYlGTcsefZhLhctegE
dOfNbBM6En2el2O3MQJBAPX3BX2hkVU42nRYkmqoqz2DEFrcd7T3WpyxT8U42d4z
rn8iJJAYwpg5SPf23PNrT+0IGGcI2LFWji9ENXM7H5kCQQDjfJahG/PvElaDGGlh
JGjDn5FwR2bs8rII/gxhBXZJSyettkm0PIPnijlcQv3L6lRHi+7LXLFUbV8C6DWn
sg3DAkAGnd2M8qK5Oy1NQfC2Y9Evd24iM5oYySqr2m097fiWbKd+DgoTwkbB15Cp
8v8goTtmMfiOFQPXSW/Lv7b8YZPJAkEA0aC1jbEckGntntCYCxRqFhqtAzJI/oCG
3xJ8hctqvQ+seL9KG/TH+E2/ne19wrMlBulEG9JG9u3yZ9Nj3iyN0QJAM6ROygXj
2ReQpmst38WnAaQvT+0Nd8cWoliGz24Vx1cAmlHI4NrCQsODNuR+gJ4vCbfq8Sa/
qJQfaDmu34JhJw==
-----END PRIVATE KEY-----
+30
Переглянути файл
@@ -0,0 +1,30 @@
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFNTBfBgkqhkiG9w0BBQ0wUjAxBgkqhkiG9w0BBQwwJAQQrUgr104rqBLzsVP2
13KRWQICCAAwDAYIKoZIhvcNAgkFADAdBglghkgBZQMEASoEEKZLvVKRzfkt0lie
Hvjy+vYEggTQ05I/45BV/X9VcWZ0W4OC8JdtYC0nosK3ymyl4iQGo4sOSLkJGpzu
0LhexFRDICNLqHkcs2XQ4Jti8lZyArY3Wql+vl9Z/FVl9s4zJIg8VDXqrPSJ6Ha+
wvVhY59UreqlQeLoFdhCK8O9/QeJow/ZQnVshJCLLQ4jiwYUHpuTrRPInZp/Jt5X
KD7N4oLsXg7NrBDyQu/DxRiPbtOu07NqMFHel4mRXCz0oKhG0pQ0KuqP0arA9RGN
CpT2SuhA3xMl6DMqQhCsGyUVhJhD7FyH/2fxlWHH2WVGWbza3L6l293fpRP2d0Nh
VET2Ia93brNpYsXT1BVAvnXG+yddbCSgTjIh/YU+ciKcaaRsZXSpB3ON2yysBXo1
XRiXbqfzx22ah9p1sT3ed3QRZ5PPOAYLNqEvUz/eNDBBiwOAdSuS4JAYYS0VyJdG
v93pWEf6fuFXdzJsOcmXy86t6JMfgV45zuiUFBD7+coNh8Yn1O+SMSTe3kKjgcaw
j7xMVDlNy6okpDv5mIFTkzhLCPZPNhGTjYBs21edEmB7PyGf9l8d+Mv+ItT9Ualy
5MUJ4ZzOPIT+Umv97DUB8CIvRSuLq34K1puvjoPa1oz6etUkMVxOEG40+h+b3JNo
RiwONkO/0Mvew8uUywnwrdS08I8Y2NeFwQsYf90371xKqSqtwLKS9CLI225WSOXQ
CTeEItkmJRa+RQApLZIvhJZQ0Hreb89W0wjsz4VStsS0laLekfh1UCZnvVZRNxCn
FM0aTTLL5TJAFMiEYDzfrADgbgm56DCMW80ByBpHiqwmysXnAX20IhDcnP6A4iDJ
ukW0qlojrAEh79Y7nCHC8y4CCsEwwUxuxXfSRrFVQeS6K2crGoalnJfXFMhHr8SW
YBD9J4x6tVhKB++rmy3LkX7lHzVLlZbfd8lJouNxTlj9BWwzm3k96v9V6KO/vDLn
Iqh2qENSNAWbRrFoCn1d6XPYt4L9kFRUrKClGVbgMyVpKNX0izVb2TElViRGiA2+
Tq2gXNGuCNcz2Y5EURaJozjCb4udRazXTraZJ4YZgfwdrHzTz+neBSEsiXU7OGrY
R+Q5ijkahA7I0BUUabPEVmxup7iOkJmagjPu8ve75c+VTqs3QHMFjEkYLazxB7a3
N6LtM9fp9X5RtQQBM+ppGjMtQD6Jkp0/npUUCbfPNnH0ZXxiNh6GaYuTY58C9ob8
zBBgfiBH2Olzrd/bZv695Ohxtd97eEqMosixKJLcYsvSqeJlmx8XmcNoWesVmooT
gR4QB1x22UdcZa0ShQLSuDTDEGiU33uJF9VM1CAHWQNEHHfVMUpdCV/m5AkyPm/1
rp4IRG2oJkpId4PHu8aGD+/4T8mnALbSzooButfj9m1u9Mo1156hi+F9Z2jN63aA
Ehnzc0aRAJ5wbnq3RzaB7sfP0+/fh6e97gW+aNeLP5l6wYzDVqwLdIpt8o7s5rvq
FFFqChnzTRUXAkRMEn7WqMU1LeAF9S+rMTcawuolADVd2ntNWMmA6bqrpgq2C/0w
qk3npanYBmHg7KzJMg8jJNIPhRNV0PyMM9RYdwfDiuMW9Gw+pi3DgUOO027Vr7j1
VNlZgdaA9uYT2qETf4/N1tyqKxsrimdUaG9dQe3w2R9SEMXw28doihc=
-----END ENCRYPTED PRIVATE KEY-----
+30
Переглянути файл
@@ -0,0 +1,30 @@
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-256-CBC,A5A125F46CCED791DF3B43E3BFFC72EF
j/qWilDpPFE84GAHLgUM4m+yXGbgjePy/G9GB8jV3doCyDRJhjvqKfJmXmypj5IM
tz2Ab059Gz396FK2N0OEtj4Us9TgPtvW/fTPNoLQp9mcIrRB7+s3TFoxNn9sUxuv
fnEcYWmrrTElin7DbWn1RrilCkf3jiTINysFt2ypUwCqpDKebQ2WRXdUCjIevul4
8hmNoS8YCbr++HkcRqnDTYPrNU1sV4Fl8cXs4BnEBlWULxlA6Y6xBsDur58X/UO3
Up+WLZ8ptCWZMNYlOGNnS3TUzMn6sWWVJaShChvJBaqwTfNoFThzD8bBxf1PXN+7
PkdkDcX5oHt+p0q1EtteFZ55VwWWMDdQTfG7dYAqTm+K4YCVaRh2xm4QTke5Kbol
0B9C3/RaR9zLA4XEf2mq+5yc3PPBcKq+aWX7pGL+9kxMXz2HsiMWb9ir+FNxb2nc
4mG5P/bmersRXhlmUl3bGXx3m5+4nnfBzO88S1MgZo4Z2L4TzJEaF7KjQENicDlA
UVMaVvq9+xUPPtIT8o/VTc5x/drO0HayhMfkz8Yxka2DLsMnLhU9NiDO6nKREEjL
GQvsan0/gIDweXzRFUY5AhPQHAOpjf/BnmTa80/DvW+WxA/B2EMMABbjxGcKK1aP
FyJf0oVvNuC91RPNhCJ2jiGKlLxnWspcEG1hiFZJvQ4Z8u1TRItqIadE0Ar0z54A
qJIr56SoU+xPyfXEt2GWwTPRu9W6l7EKo1yVKvqSdz1UGzrW3WuHI3G/UYvkqAs9
fBQK47sYlrpLYS3qnFbYlrbR0RCHlNLVCs0V1Xb6U+qU3ExQRlo1YvkvH7dkvhqn
Bdqv9iYnlryVP5SJcAeKZ7VqmM/LXq+bF7g/hTcHkg1yISqC1qaYDNi5D7uOVwW6
ALiD5zFyLArnG7girpuT1hc8N+dkh1ZckOGf2nW3I27czY4Xsaz+9keXFvAy5F2K
OGSu5iu/8p7M1bPRdiutcEVrT07FFvVZufASo4brMXptxpozd9o14uMhudneIqtb
qgw+F4nl6LZkTmbdn2suXjG1jNxlv86bKsLuY/IoaOO70YTho2HS1zpQ7SS1Jj63
5v4y6205m9xMJVZywbCwo7FQhxfBT3cmS2KkD9mYSZiFtYkRDLPxneNHKyegnuRy
+rYelwbKCiFoUXg7QMOYfYDfC2UckCblcWlYgtSkL6TRnzl0aKBO8SQiph8LMvtB
+/Om76T4OGlkk4PJsbLjt6uSEgcAwJBjG2ox0FpE75tLIO6ZBWdgn6TnBLFiIniw
apomJZNc3bybpiuH+w9F1gOy/1c+nS6O82559Zn03ez6tX2SHqO7ROX06zLdcdVO
Yg0PJiBaAPNgfKeSb/K4XX4L/bJi9lH6VjUBcQqDqHKSAonmZDLGrWxHyuHbBE1u
wHXigKzIxR0nw8VUPIFDpdgnNlKe9eputx0+7rgrh6sW1rZL3EfuyX/62SLG8Y4L
YYrIx1AcrkKct1yPUXgwUKYn1TItPhK3JSXxmCYbkEuVdZTKKtf/irqoauEIPjm2
ufvhM0Y+PIGGOwAh7EvuDUUemeHd4Q2HnHwnROx5L1rSf2izC1GoxCtP8/UE2OTK
-----END RSA PRIVATE KEY-----
+27
Переглянути файл
@@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAz3421PPLmMLaTYbYUXb+4dXCSdadp52/KSizIv0nT0n0mS5t
HgNk246eh7Rqr/dXp6wj8svvoUDpdCrX6Dxm8w6sZ82TyuAbXJJsR4gUInxuwNHd
46avGNiYyJJDCLZxe5F0KG/PUzmhY/qVvP1x+mPsOXGzGHZH90Vyo2aWES0QcZ8w
JeJTIT/26BO63qFrgSMDxVV51/AFT6dG45KUdiNX4FCmcTMP5m0YVlcv/W+1Ye/U
cDxvOa7womtnuznQy8TWFHBB6HL32/KYVB7SO2b5wiXG1yV+zjffyXVPwzfUR3Pm
s9WYweGWtXB6C15W51IrEn9Xt7Z2mLmZxqhizwIDAQABAoIBAEPvDeLBA0q7mR/C
43YgoKiuBNOKCcoEMr82DRiKi+FA1G2GrsEh2VatJihGIJwGPlOuX4Kx1H4WM5kO
IexuQkKX7kh6HUZSA4NWiU8ONZFo7hYBYEWVoERhPfyFLp9epF8Rq1dF7DpSZw3F
0Fq1HCkfqArSuBFtQHir6zGmd8GdSYtMajJVOW5O+5L7JAUS+JOdVd4Dnw6H5brN
px6REvMmUgD9boor27Uzl2AdnUR7g6JgeCpbHFHSME/b0zAu5MRAhCg8HeX8SZHS
CfzbAYvfLTmUtEKx+8uxawQNyu4C2FSUiHJCj/K895FAG1vb2FdnUcMSDEI9pGSR
CmtJAOkCgYEA7DbCTyXbeQXAD2I1dNS1/daKCwg2LseclDRJ6Nq5kP6r+YNHddi/
eGgEpJRiCHyVe+XV5sGvKuHq2VNIbPf14clJ4w5QS3eqBM7Q4wwOPSee4Npz5s6z
rY7oLnB+0HvBACc8/jtD27ZHEM0ZlfL52KbVh7xCpp44fCjm+cDjFRUCgYEA4N+U
mJIsrPFd7UAKW6elO0N36t721bvtgGrwWCBNKoBvX4Ra0HYoVV9TGdy+E7V7YopD
DzvDDhgY0oDf6m3pQoYZEFdJfgiH9JzqNUnqBF6deqtTXGUxKIZ22m6DI1q5Jega
+klDbxLe2Ot1DouthPpZedMnvMCoAMgR6xlzmVMCgYEA6E2iIryUbxzcmUR4VgGr
SYxoDhlxT+g3cPlLhpk0zgfnkWcuLBORmhSSHFlhX/9F86dBCcOJRZ8Qm9Jv89qY
EgknhQlaph11M21BuUjt5+7DLVcpjN/TjOhyJtmznc1XLZNpRI18WuOUWE2cz/84
/3xzmx3mXn5UBqcjIJQ1v1ECgYEAxMEbhTswt8kgg7uq1HMk0WvA+tJHas9ZiSkI
TjNFEVD7Ibnus8MOm1Qio9hOv3PlF10eO6/whN50qYp1YOr2U2x9dZzKAup3+oFp
M2MOzO/0HibVc1xe3LR6PbjAKSjyIsapBCEiTYdXULkD3Mw98MITk9I7TJ3M4ZIs
i2XjjV0CgYBOMKOCJFIR9kkEYrvCXrINfKGtbAAWi+mwkO76idvnGc4+T89FsTKM
nO2SZSIirdo6BkMKpkQKlaIR6ScpZvUSr8pmjNARlSpAAEn8MhH9eWL2UMf/SCt7
lqUaBq5bu48rtFJcwoKR13zrtRSgsQ7FU9+VjjlSV7g6zYaWAHYOVA==
-----END RSA PRIVATE KEY-----
+30
Переглянути файл
@@ -0,0 +1,30 @@
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkrBgEEAdpHBAswHAQQRIkBIQM391D42515
x8qX7AICQAACAQgCAQEwHQYJYIZIAWUDBAEqBBAV4ik0WbzhBWcA0q2nB8/oBIIE
0KHh9w0Hi98G2eAdpBRCpAC2FryLKvfPcD6puChTFFwSIryfPt9dSrLJHB83c6/+
uYfaQskgdoA1aCx97A+lMAXFemURUBnI59+2eqdkFdMPsWTkRQC0iL0XiRzuwUA7
SbhSopssc21ZtiVvlxpDWaFupOqwu8mb79Lp17Rriirc/dy/e2ODStFzQyCvBJmh
rSbWwFBsZc0zB4Ptp4Ekeah5DEbxKXG5Lw56DetDiLukXlVeVJg+Gw2ZXHlipj4s
/wqbrnbX3VQCWuBJot3F3wYqhGZO7e52Ge+jiojfU1zT4NwMgIAjIv7jIxcxHINj
c+Iuql1Vq3QSM+sU7MYDMxsfgsEEzYgZDtcBMqJTYy0Dq4MixkSgc8zYI+V2axl9
RKkBQAOQQcgCN+hrPgV8JKmeNP8OBsIX3VmXgTBhRt6xpTgW2YtV0Xc4HbTY9YoF
tm1NI6gtcOlBPewJWI6GD8JueqY88lmrG/Qf2g4x7cQz3TCTlwpB6DXHX/blRSVe
kGJVu0CWTRy9yM49xnnZGOrK8UjL61Nd6on8vb9HC+wFsDFh2kJ1/8RvCWLyqPJY
xM8PVmPOwKhigEF2QN/bt1eESm3D1OTeV05TnBDgRPcDwVm/ym80D7tS4Vwj7xgf
gWzVQNoI1wtXMaJalmxSQ5qDdm78T/Mj3EJIDLoBtnkGJGQFkDxpZ4y5jQSHAAc6
7kXnM7SAI4LUvME5LcwJZlHcUnZ7FHqCdJWh+FrMUpsL807Gcd7KMcsx++lQAAVR
B/OUVykJHD4JnZtt3W5yyTA8E8nWbiDHro9n1w6n/fdSTj1AnwWncEx4o0CrSulf
eZ/78eE2ba4cOyOnE1VuMmvqfH98xcXwk1mME59HWbxxuryUgqoJX40dVSay0tTt
Ccfn0oi0WN+P8Qoq/7AueyeBbzN2dTEyDTj/C7dFFeerJT/gV3bfke3DL+J3E4zq
+U5qrI0Z+FXw3cSF66pCY+uWruSllkwGXLjTYWSXJ7oXaANu56zAjhjGgdzZ3o9F
Ub2JuvIuAZ+cYCJnSDPU1r4hUvMoJcfBu7YqkFQXztLU0kvdWR3fmOCapKFHaKXh
y0KVUzi2tkTwj22+hYOTuHTLtleiFmtJERWeCkmn2eyMhSJsbxanlSRddVsMs5mJ
vMpnXh1AlDbKHRqQIxOUhf01Dw1Q4VpQfMuxPcN4GQxaj2N0Kn6OmvxF42ze8OuA
vO1orsV8VRDApFAVmhsfjet0hXo/EnWL+oJ5WadY5hfYX1cXe8BdjiDRyCCDv9ob
lZUHadw2sbr1FkJ3TsxnX0OkOQ+07pDq03EjyScHyNyCForw8F0a867F23TBKunq
q0XuwGG+EgkVEHvy/Ksqrfc60jbDyu3kAusmnLMQv07OHUiiqlBbtLQO31FQH7QE
KIm6BxiZ8A2DxdsbyWO1LZ9+SHttUA3OAAKjFeOO89A7iK+8asRTjIRSt6NtDI6V
rbbWFKrZY9ACfXWoFuCrfIjipPrc8q293rJ8mVxZf34rvEjt0WuV17VDEbozt8XZ
4F2mnYeVF8LX0sdy+OBVfpTGpmn8Ubsxond4OQyzxI3MbT4WOey2BdznmdROEYbJ
RmVzKCBEWVuHptyL6LrWbO3pu+Kdkh/1mK/E+UhFyrjk
-----END ENCRYPTED PRIVATE KEY-----
+48
Переглянути файл
@@ -183,3 +183,51 @@ async fn a_reused_physical_connection_restores_the_configured_database() -> IdeR
assert_eq!(pool.execute("SELECT CONNECTION_ID()", 1, &c).await?.rows, backend);
Ok(())
}
/// Forwards to the test server, adding `CLIENT_DEPRECATE_EOF` to the capabilities its greeting
/// advertises, like proxies that set the flag but leave a 5.x server sending legacy EOF packets.
async fn deprecate_eof_proxy(host: String, port: u16) -> u16 {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let local = listener.local_addr().unwrap().port();
tokio::spawn(async move {
while let Ok((mut client, _)) = listener.accept().await {
let mut server = tokio::net::TcpStream::connect((host.as_str(), port)).await.unwrap();
tokio::spawn(async move {
let mut header = [0u8; 4];
server.read_exact(&mut header).await.unwrap();
let len = u32::from_le_bytes([header[0], header[1], header[2], 0]) as usize;
let mut greeting = vec![0u8; len];
server.read_exact(&mut greeting).await.unwrap();
// protocol version, NUL-terminated server version, connection id, 8 bytes of
// auth data, filler, lower capabilities, charset, status, upper capabilities.
let version_end = 1 + greeting[1..].iter().position(|&b| b == 0).unwrap();
let upper = version_end + 1 + 4 + 8 + 1 + 2 + 1 + 2;
greeting[upper + 1] |= 0x01; // bit 24 of the capability flags
client.write_all(&header).await.unwrap();
client.write_all(&greeting).await.unwrap();
let _ = tokio::io::copy_bidirectional(&mut client, &mut server).await;
});
}
});
local
}
#[tokio::test]
async fn servers_that_advertise_deprecate_eof_but_send_legacy_eof_packets() -> IdeResult<()> {
use ide_db::Pool;
let Some((host, port, user, password)) = server() else { return Ok(()) };
let source = DataSource {
driver: DbDriver::Mysql,
host: "127.0.0.1".into(),
port: Some(deprecate_eof_proxy(host, port).await),
user,
..Default::default()
};
let pool = Pool::connect(&source, Some(&password), None).await?;
assert_eq!(pool.rows("SELECT 1 + 1", 1).await?, vec![vec![Some("2".into())]]);
let c = CancellationToken::new();
let r = pool.execute("SELECT 'a' UNION ALL SELECT 'b'", 10, &c).await?;
assert_eq!(r.rows, vec![vec![Some("a".into())], vec![Some("b".into())]]);
Ok(())
}
+365
Переглянути файл
@@ -0,0 +1,365 @@
//! Against a real Snowflake account, with key-pair authentication:
//!
//! ```sh
//! OPENIDEA_TEST_SNOWFLAKE_ACCOUNT=myorg-myaccount OPENIDEA_TEST_SNOWFLAKE_USER=jane \
//! OPENIDEA_TEST_SNOWFLAKE_KEY_FILE=~/.ssh/rsa_key.p8 \
//! cargo test -p ide-db --test snowflake -- --ignored --nocapture
//! ```
//!
//! Optional: `…_KEY_PASSPHRASE`, `…_WAREHOUSE`, `…_ROLE`, `…_DATABASE`, `…_SCHEMA`.
//! The output holds no credential.
use ide_core::{CancellationToken, error::ErrorKind};
use ide_db::{DataSource, DbDriver, DbOptions, DbSnowflake, DbSnowflakeAuth, Pool};
use std::time::{Duration, Instant};
fn setting(name: &str) -> Option<String> {
std::env::var(format!("OPENIDEA_TEST_SNOWFLAKE_{name}")).ok().filter(|v| !v.trim().is_empty())
}
fn source() -> DataSource {
DataSource {
id: "snowflake".into(),
name: "snowflake".into(),
driver: DbDriver::Snowflake,
host: setting("ACCOUNT").expect("OPENIDEA_TEST_SNOWFLAKE_ACCOUNT"),
user: setting("USER").expect("OPENIDEA_TEST_SNOWFLAKE_USER"),
database: setting("DATABASE").unwrap_or_default(),
options: DbOptions { default_schema: setting("SCHEMA").unwrap_or_default(), ..Default::default() },
snowflake: Some(DbSnowflake {
warehouse: setting("WAREHOUSE").unwrap_or_default(),
role: setting("ROLE").unwrap_or_default(),
auth: DbSnowflakeAuth::KeyPair,
private_key_file: setting("KEY_FILE").expect("OPENIDEA_TEST_SNOWFLAKE_KEY_FILE"),
}),
..Default::default()
}
}
#[tokio::test]
#[ignore = "requires OPENIDEA_TEST_SNOWFLAKE_ACCOUNT, _USER and _KEY_FILE"]
async fn key_pair_session_against_a_real_account() {
let passphrase = setting("KEY_PASSPHRASE");
let started = Instant::now();
let pool = match Pool::connect(&source(), passphrase.as_deref(), None).await {
Ok(pool) => pool,
Err(error) => panic!("login failed ({:?}): {}", error.kind, error.message),
};
println!("[1] logged in with the key pair in {:?}", started.elapsed());
let cancel = CancellationToken::new();
let cell = |rows: &[Vec<Option<String>>], i: usize| rows[0][i].clone().unwrap_or_else(|| "<null>".into());
let version = pool.execute("SELECT 'Snowflake ' || CURRENT_VERSION()", 1, &cancel).await.unwrap();
println!("[2] {}", cell(&version.rows, 0));
let context = pool
.execute("SELECT CURRENT_WAREHOUSE(), CURRENT_DATABASE(), CURRENT_SCHEMA(), CURRENT_ROLE()", 1, &cancel)
.await
.unwrap();
println!(
"[3] context: warehouse={} database={} schema={} role={}",
cell(&context.rows, 0),
cell(&context.rows, 1),
cell(&context.rows, 2),
cell(&context.rows, 3)
);
for (wanted, i) in
[(setting("WAREHOUSE"), 0), (setting("DATABASE"), 1), (setting("SCHEMA"), 2), (setting("ROLE"), 3)]
{
if let Some(wanted) = wanted {
assert!(cell(&context.rows, i).eq_ignore_ascii_case(&wanted), "context {i} is not {wanted}");
}
}
let types = pool
.execute(
"SELECT 12345.67::NUMBER(10,2) AS n, 1.5::FLOAT AS f, 'text' AS t, TRUE AS b, \
'2023-11-14'::DATE AS d, '01:02:03.5'::TIME(3) AS tm, \
'2023-11-14 22:13:20.123'::TIMESTAMP_NTZ(3) AS ntz, \
'2023-11-14 22:13:20 +02:00'::TIMESTAMP_TZ(0) AS tz, \
'2023-11-14 22:13:20'::TIMESTAMP_LTZ(0) AS ltz, \
PARSE_JSON('{\"a\":[1,2]}') AS v, TO_BINARY('DEADBEEF', 'HEX') AS bin, NULL AS nothing",
1,
&cancel,
)
.await
.unwrap();
for (column, value) in types.columns.iter().zip(&types.rows[0]) {
println!("[4] {:8} {:18} {}", column.name, column.type_name, value.as_deref().unwrap_or("<null>"));
}
let shown: Vec<Option<&str>> = types.rows[0].iter().map(|v| v.as_deref()).collect();
assert_eq!(shown[0], Some("12345.67"));
assert_eq!(shown[2], Some("text"));
assert_eq!(shown[3], Some("true"));
assert_eq!(shown[4], Some("2023-11-14"));
assert_eq!(shown[5], Some("01:02:03.500"));
assert_eq!(shown[6], Some("2023-11-14 22:13:20.123"));
assert_eq!(shown[7], Some("2023-11-14 22:13:20 +02:00"));
assert_eq!(shown[10], Some("0xDEADBEEF"));
assert_eq!(shown[11], None);
assert_eq!(types.columns[0].type_name, "NUMBER(10,2)");
pool.execute("SET openidea_probe = 41", 0, &cancel).await.unwrap();
let kept = pool.execute("SELECT $openidea_probe + 1", 1, &cancel).await.unwrap();
assert_eq!(cell(&kept.rows, 0), "42");
println!("[5] session state is kept between statements");
let databases = pool.execute("SHOW TERSE DATABASES", 1000, &cancel).await.unwrap();
println!(
"[6] SHOW TERSE DATABASES: {} databases, columns {:?}",
databases.rows.len(),
databases.columns.iter().map(|c| c.name.as_str()).collect::<Vec<_>>()
);
let missing = pool.execute("SELECT * FROM openidea_no_such_table_x", 1, &cancel).await.unwrap_err();
println!("[7] error for a missing table ({:?}): {}", missing.kind, missing.message.replace('\n', " | "));
assert_eq!(missing.kind, ErrorKind::InvalidInput);
let limited = pool.execute("SELECT SEQ4() FROM TABLE(GENERATOR(ROWCOUNT => 100))", 2, &cancel).await;
match limited {
Ok(limited) => {
assert_eq!(limited.rows.len(), 2);
assert!(limited.has_more);
println!("[8] a row limit keeps 2 of 100 rows and reports more");
let page = pool
.execute_from(
"SELECT SEQ4() AS n, REPEAT('x', 200) FROM TABLE(GENERATOR(ROWCOUNT => 300000)) ORDER BY n",
250_000,
3,
&cancel,
)
.await
.unwrap();
assert_eq!(
page.rows.iter().map(|r| r[0].clone().unwrap()).collect::<Vec<_>>(),
["250000", "250001", "250002"]
);
println!(
"[9] a page deep in a large result: {:?} total, {:?} to first answer",
page.elapsed_ms, page.execution_ms
);
let stop = CancellationToken::new();
let waited = Instant::now();
let statement = pool.execute("CALL SYSTEM$WAIT(120)", 1, &stop);
let cancelling = async {
tokio::time::sleep(Duration::from_secs(2)).await;
stop.cancel();
};
let (outcome, ()) = tokio::join!(statement, cancelling);
let error = outcome.unwrap_err();
assert_eq!(error.kind, ErrorKind::Cancelled, "{}", error.message);
assert!(waited.elapsed() < Duration::from_secs(30));
println!("[10] cancelled a 120 s statement after {:?}: {}", waited.elapsed(), error.message);
let after = pool.execute("SELECT 1", 1, &cancel).await.unwrap();
assert_eq!(cell(&after.rows, 0), "1");
println!("[11] the session works after the cancellation");
}
Err(error) => {
println!("[8] skipped warehouse checks ({:?}): {}", error.kind, error.message.replace('\n', " | "))
}
}
pool.close().await;
assert!(pool.is_closed());
println!("[12] session closed");
}
/// The Database tree's metadata calls. Prints counts and column shapes only, never object names.
#[tokio::test]
#[ignore = "requires OPENIDEA_TEST_SNOWFLAKE_ACCOUNT, _USER and _KEY_FILE"]
async fn explorer_metadata_against_a_real_account() {
let root = tempfile::tempdir().unwrap();
let dir = root.path().to_path_buf();
let ssh = std::sync::Arc::new(ide_ssh::SshManager::new(dir.join("ssh-passwords.json")));
let db = ide_db::DbManager::new(dir.clone(), dir.join("passwords.json"), dir.join("history.json"), ssh);
db.save_sources(vec![source()]).await.unwrap();
db.set_password("snowflake", setting("KEY_PASSPHRASE"), false).await.unwrap();
let databases = db.databases("snowflake").await.unwrap();
println!("[m1] databases: {} (current set: {})", databases.names.len(), !databases.current.is_empty());
assert!(!databases.names.is_empty());
let database = setting("DATABASE")
.map(|d| d.to_uppercase())
.filter(|d| databases.names.contains(d))
.unwrap_or_else(|| databases.names[0].clone());
let target = format!("snowflake:{database}");
let schemas = db.schemas(&target).await.unwrap();
println!("[m2] schemas in the chosen database: {}", schemas.len());
assert!(!schemas.iter().any(|s| s == "INFORMATION_SCHEMA"));
// The first schema that has tables, so the deeper calls have something to read.
let mut found = None;
for schema in schemas.iter().take(25) {
let tables = db.tables(&target, schema).await.unwrap();
if !tables.is_empty() {
found = Some((schema.clone(), tables));
break;
}
}
match found {
None => println!("[m3] no tables in the first 25 schemas: table checks skipped"),
Some((schema, tables)) => {
let views = tables.iter().filter(|t| t.kind != ide_db::DbTableKind::Table).count();
println!("[m3] tables and views in one schema: {} ({} views)", tables.len(), views);
let details = db.table_details(&target, &schema, &tables[0].name).await.unwrap();
let typed = details.columns.iter().filter(|c| !c.data_type.is_empty()).count();
println!(
"[m4] first table: {} columns ({} typed, {} primary key, {} nullable), {} foreign keys; first type: {}",
details.columns.len(),
typed,
details.columns.iter().filter(|c| c.primary_key).count(),
details.columns.iter().filter(|c| c.nullable).count(),
details.foreign_keys.len(),
details.columns.first().map_or("", |c| c.data_type.as_str())
);
assert!(!details.columns.is_empty());
assert_eq!(typed, details.columns.len());
let objects = db.schema_objects(&target, &schema).await.unwrap();
let mut kinds = std::collections::BTreeMap::new();
for other in &objects.others {
*kinds.entry(other.kind.as_str()).or_insert(0) += 1;
}
println!(
"[m5] schema objects: {} routines, {} sequences, others {:?}",
objects.routines.len(),
objects.sequences.len(),
kinds
);
let ddl = db.ddl(&target, &schema, &tables[0].name).await.unwrap();
println!(
"[m6] DDL of the first table: {} characters, starts with {:?}",
ddl.len(),
ddl.split_whitespace().next()
);
assert!(ddl.to_lowercase().starts_with("create"));
let page =
db.table_page(&target, &schema, &tables[0].name, None, &[], 0, 5, &CancellationToken::new()).await;
match page {
Ok(page) => println!(
"[m7] table data page: {} rows, {} columns",
page.result.rows.len(),
page.result.columns.len()
),
Err(error) => {
println!("[m7] table data page failed ({:?}): {}", error.kind, error.message.replace('\n', " | "))
}
}
}
}
let loading = Instant::now();
let completion = db.completion(&target).await.unwrap();
println!("[m8] completion model loaded in {:?}", loading.elapsed());
println!(
"[m8] completion model: {} tables ({} with columns), {} columns, default schema set: {}",
completion.tables.len(),
completion.tables.iter().filter(|t| !t.columns.is_empty()).count(),
completion.tables.iter().map(|t| t.columns.len()).sum::<usize>(),
!completion.default_schema.is_empty()
);
let session = db.session_context(&target).await.unwrap();
println!(
"[m9] session: warehouse set: {}, role set: {}, {} warehouses, {} roles to pick from",
!session.warehouse.is_empty(),
!session.role.is_empty(),
session.warehouses.len(),
session.roles.len()
);
assert!(!session.role.is_empty());
let plan = db.explain(&target, "SELECT 1 AS n", &CancellationToken::new(), None).await;
match plan {
Ok(plan) => println!(
"[m10] explain plan: {} operations, first {:?}",
plan.len(),
plan.first().map(|n| n.operation.clone())
),
Err(error) => println!("[m10] explain failed ({:?}): {}", error.kind, error.message.replace('\n', " | ")),
}
// Introspection as the app runs it: every database through the data source itself.
let listed = db.databases("snowflake").await.unwrap();
let (mut read, mut failed, mut schemas, mut tables) = (0, 0, 0, 0);
let mut first_failure = None;
for database in &listed.names {
match db.database_catalog("snowflake", database).await {
Ok(catalog) => {
read += 1;
schemas += catalog.len();
tables += catalog.iter().map(|s| s.tables.len()).sum::<usize>();
}
Err(error) => {
failed += 1;
// Names stay out of the output: quoted parts of the message are masked.
let mut masked = String::new();
let mut quote = None;
for c in error.message.replace('\n', " | ").chars() {
match quote {
Some(q) if c == q => quote = None,
Some(_) => continue,
None if c == '\'' || c == '"' => quote = Some(c),
None => {}
}
masked.push(c);
}
first_failure.get_or_insert(format!("{:?}: {masked}", error.kind));
}
}
}
println!(
"[m11] introspection: {} databases listed, {read} read, {failed} failed, {schemas} schemas, {tables} tables and views",
listed.names.len()
);
if let Some(failure) = first_failure {
println!("[m11] first failure: {failure}");
}
assert!(read > 0, "no database could be introspected");
// The reference's commands that only list things and name no object, against the real server:
// each is either answered or refused by Snowflake (an edition or privilege it needs), never
// misread by the client. `USE` goes to what the session already has.
let fixture: serde_json::Value =
serde_json::from_str(include_str!("fixtures/snowflake/commands.json")).expect("the fixture parses");
let mut statements: Vec<(String, String)> = fixture["commands"]
.as_array()
.expect("commands")
.iter()
.map(|c| {
(c["command"].as_str().unwrap_or_default().to_owned(), c["sql"].as_str().unwrap_or_default().to_owned())
})
.filter(|(_, sql)| sql.starts_with("SHOW ") && !sql.to_lowercase().contains("my_") && !sql.contains("analyst"))
.collect();
let quoted = |name: &str| format!("\"{}\"", name.replace('"', "\"\""));
statements.push(("USE ROLE".into(), format!("USE ROLE {}", quoted(&session.role))));
if !session.warehouse.is_empty() {
statements.push(("USE WAREHOUSE".into(), format!("USE WAREHOUSE {}", quoted(&session.warehouse))));
}
statements.push(("USE DATABASE".into(), format!("USE DATABASE {}", quoted(&database))));
statements.push(("USE SECONDARY ROLES".into(), "USE SECONDARY ROLES ALL".into()));
let (mut answered, mut refused, mut broken) = (0, Vec::new(), Vec::new());
for (command, sql) in &statements {
match db.execute(&target, sql, 1, &CancellationToken::new()).await {
Ok(_) => answered += 1,
Err(error) if error.kind == ErrorKind::InvalidInput => {
// Snowflake's own refusal: the command and the error code only.
let code = error.message.split(['[', ']']).nth(1).unwrap_or("?").to_owned();
refused.push(format!("{command} [{code}]"));
}
Err(error) => broken.push(format!("{command} ({:?})", error.kind)),
}
}
println!(
"[m12] commands: {} run, {answered} answered, {} refused by Snowflake, {} failed in the client",
statements.len(),
refused.len(),
broken.len()
);
if !refused.is_empty() {
println!("[m12] refused: {}", refused.join(", "));
}
assert!(broken.is_empty(), "the client failed on: {}", broken.join(", "));
assert!(answered > 0);
db.disconnect_source("snowflake").await;
println!("[m13] disconnected");
}
+2 -5
Переглянути файл
@@ -458,16 +458,13 @@ fn canonical_dir(p: &Path) -> PathBuf {
fn is_executable(p: &Path) -> bool {
let Ok(meta) = std::fs::metadata(p) else { return false };
if !meta.is_file() {
return false;
}
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
meta.permissions().mode() & 0o111 != 0
meta.is_file() && meta.permissions().mode() & 0o111 != 0
}
#[cfg(not(unix))]
true
meta.is_file()
}
/// blake3 of a file, hex.
+1 -1
Переглянути файл
@@ -34,7 +34,7 @@ use std::path::{Component, Path, PathBuf};
pub const MANIFEST_FILE: &str = "plugin.toml";
/// The plugin API (engine) version this host implements.
pub const ENGINE_VERSION: &str = "1.6.0";
pub const ENGINE_VERSION: &str = "1.7.0";
/// The 0.1 API, still accepted for contribution-only packages and 0.1 components.
pub const LEGACY_API: &str = "0.1";
/// Marketplace categories, in the order they are shown.
+1
Переглянути файл
@@ -46,6 +46,7 @@ globset.workspace = true
ide-db.workspace = true
ide-ssh.workspace = true
ide-remote.workspace = true
ide-web.workspace = true
ide-jsonrpc.workspace = true
ide-agent.workspace = true
ide-notebook.workspace = true
+175
Переглянути файл
@@ -0,0 +1,175 @@
//! Values coding agents are configured with that may be secrets (MCP servers' environment
//! variables and HTTP headers, agents' environment), kept with the saved passwords in the OS
//! credential store; the files that configure them keep only the names. Without a credential
//! store (a headless Linux host) a value is kept in an owner-only file of the user's data
//! instead, never in a project's files, and the user is told.
use ide_core::event::NotifyLevel;
use ide_core::{IdeEvent, IdeResult, SharedSink};
use ide_ssh::secrets::SecretFile;
use parking_lot::Mutex;
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
pub struct AgentSecrets {
store: SecretFile,
/// Values the credential store refused, by key.
local: PathBuf,
lock: Mutex<()>,
}
/// Where [`AgentSecrets::set`] kept a value.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Kept {
Store,
LocalFile,
}
impl AgentSecrets {
pub fn new(data_dir: &Path) -> Self {
Self::with_store(
SecretFile::new(data_dir.join("agent-secrets.json")),
data_dir.join("agent-secrets.local.json"),
)
}
pub fn with_store(store: SecretFile, local: PathBuf) -> Self {
Self { store, local, lock: Mutex::new(()) }
}
pub fn get(&self, key: &str) -> Option<String> {
self.store.get(key).or_else(|| self.read_local().remove(key))
}
pub fn set(&self, key: &str, value: &str) -> IdeResult<Kept> {
match self.store.set(key, value) {
Ok(()) => {
self.edit_local(|local| local.remove(key).is_some())?;
Ok(Kept::Store)
}
Err(e) => {
tracing::warn!("agent settings: {key} is kept in {}: {}", self.local.display(), e.message);
self.edit_local(|local| local.insert(key.to_owned(), value.to_owned()).as_deref() != Some(value))?;
Ok(Kept::LocalFile)
}
}
}
pub fn delete(&self, key: &str) {
self.store.delete(key);
if let Err(e) = self.edit_local(|local| local.remove(key).is_some()) {
tracing::warn!("agent settings: could not forget {key}: {}", e.message);
}
}
/// The keys that start with `prefix`, sorted.
pub fn keys(&self, prefix: &str) -> Vec<String> {
let keys: BTreeSet<String> = self.store.keys().into_iter().chain(self.read_local().into_keys()).collect();
keys.into_iter().filter(|k| k.starts_with(prefix)).collect()
}
/// Tells the window that values went to the owner-only file rather than the credential store.
pub fn notify_kept_locally(&self, sink: &SharedSink) {
sink.emit(IdeEvent::Notify {
level: NotifyLevel::Warning,
title: "No credential store".into(),
body: Some(format!(
"Values of agent and MCP server settings are kept in {}, readable only by you.",
self.local.display()
)),
});
}
fn read_local(&self) -> BTreeMap<String, String> {
match std::fs::read(&self.local) {
Ok(bytes) => serde_json::from_slice(&bytes).unwrap_or_else(|e| {
tracing::warn!("agent settings: {}: {e}", self.local.display());
BTreeMap::new()
}),
Err(_) => BTreeMap::new(),
}
}
fn edit_local(&self, edit: impl FnOnce(&mut BTreeMap<String, String>) -> bool) -> IdeResult<()> {
let _g = self.lock.lock();
let mut local = self.read_local();
if !edit(&mut local) {
return Ok(());
}
if local.is_empty() {
return match std::fs::remove_file(&self.local) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => Err(e.into()),
_ => Ok(()),
};
}
if let Some(dir) = self.local.parent() {
std::fs::create_dir_all(dir)?;
}
let bytes = serde_json::to_vec_pretty(&local).map_err(|e| ide_core::IdeError::internal(e.to_string()))?;
ide_vfs::write_atomic_private(&self.local, &bytes)?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use ide_ssh::secrets::testing::{files_containing, use_memory_store};
use ide_ssh::secrets::{StoreError, StoreResult, Vault};
use std::sync::Arc;
/// A machine without a credential store (a headless Linux host).
struct NoStore;
impl Vault for NoStore {
fn available(&self) -> bool {
false
}
fn get(&self, _: &str, _: &str) -> StoreResult<String> {
Err(StoreError::NoDefaultStore)
}
fn set(&self, _: &str, _: &str, _: &str) -> StoreResult<()> {
Err(StoreError::NoDefaultStore)
}
fn delete(&self, _: &str, _: &str) -> StoreResult<()> {
Err(StoreError::NoDefaultStore)
}
}
#[test]
fn values_go_to_the_credential_store() {
use_memory_store();
let tmp = tempfile::tempdir().unwrap();
let secrets = AgentSecrets::new(tmp.path());
assert_eq!(secrets.set("mcp/user/a/env/TOKEN", "value-7c1e").unwrap(), Kept::Store);
assert_eq!(files_containing(tmp.path(), "value-7c1e"), Vec::<PathBuf>::new());
assert_eq!(AgentSecrets::new(tmp.path()).get("mcp/user/a/env/TOKEN").as_deref(), Some("value-7c1e"));
assert_eq!(secrets.keys("mcp/user/"), ["mcp/user/a/env/TOKEN"]);
secrets.delete("mcp/user/a/env/TOKEN");
assert_eq!(secrets.get("mcp/user/a/env/TOKEN"), None);
}
#[test]
fn without_a_credential_store_values_stay_in_an_owner_only_file() {
let tmp = tempfile::tempdir().unwrap();
let local = tmp.path().join("agent-secrets.local.json");
let store = SecretFile::with_vault(
tmp.path().join("agent-secrets.json"),
|v| serde_json::from_value(v).ok(),
Arc::new(NoStore),
);
let secrets = AgentSecrets::with_store(store, local.clone());
assert_eq!(secrets.set("mcp/user/a/env/TOKEN", "value-7c1e").unwrap(), Kept::LocalFile);
assert_eq!(files_containing(tmp.path(), "value-7c1e"), std::slice::from_ref(&local));
assert_eq!(secrets.get("mcp/user/a/env/TOKEN").as_deref(), Some("value-7c1e"));
assert_eq!(secrets.keys("mcp/"), ["mcp/user/a/env/TOKEN"]);
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
assert_eq!(std::fs::metadata(&local).unwrap().permissions().mode() & 0o777, 0o600);
}
secrets.delete("mcp/user/a/env/TOKEN");
assert_eq!(secrets.get("mcp/user/a/env/TOKEN"), None);
assert!(!local.exists());
}
}
+3 -3
Переглянути файл
@@ -426,9 +426,9 @@ fn plugins_dir() -> Result<PathBuf, String> {
if let Some(dir) = std::env::var_os("OPENIDEA_PLUGINS_DIR") {
return Ok(PathBuf::from(dir));
}
dirs::data_dir()
.map(|d| d.join("ai.l0ng.openidea").join("plugins"))
.ok_or_else(|| "cannot find the app data directory; set OPENIDEA_PLUGINS_DIR".into())
ide_host::data_dirs::data_dir("ai.l0ng.openidea")
.map(|d| d.join("plugins"))
.map_err(|error| format!("{error}; set OPENIDEA_PLUGINS_DIR"))
}
#[derive(Default, serde::Serialize, serde::Deserialize)]
+66 -4
Переглянути файл
@@ -5,9 +5,52 @@ use std::fs;
use std::io;
use std::path::{Path, PathBuf};
/// Moves the rebuildable local data (tools, language servers, indexes, caches);
/// configuration and roaming data keep following the OS.
/// Moves the rebuildable local data (tools, language servers, indexes, caches).
pub const LOCAL_DATA_ENV: &str = "OPENIDEA_LOCAL_DATA_DIR";
/// Moves the configuration (settings, recent projects, language definitions, window state).
pub const CONFIG_ENV: &str = "OPENIDEA_CONFIG_DIR";
/// Moves the user data (plugins, scratches, Local History, trusted projects).
pub const DATA_ENV: &str = "OPENIDEA_DATA_DIR";
/// Moves the logs.
pub const LOG_ENV: &str = "OPENIDEA_LOG_DIR";
/// `$OPENIDEA_CONFIG_DIR/<identifier>` when set, else the OS configuration directory
/// (Tauri's `app_config_dir`). Like the local data, the identifier keeps release and
/// per-worktree debug builds apart under one root; a relative path is an error.
pub fn config_dir(identifier: &str) -> Result<PathBuf, String> {
resolve_env(CONFIG_ENV, std::env::var_os(CONFIG_ENV), dirs::config_dir(), identifier)
}
/// `$OPENIDEA_DATA_DIR/<identifier>` when set, else the OS data directory (Tauri's `app_data_dir`).
pub fn data_dir(identifier: &str) -> Result<PathBuf, String> {
resolve_env(DATA_ENV, std::env::var_os(DATA_ENV), dirs::data_dir(), identifier)
}
/// `$OPENIDEA_LOG_DIR/<identifier>` when set, else `~/Library/Logs/<identifier>` on macOS and
/// `<local data>/<identifier>/logs` elsewhere (Tauri's `app_log_dir`).
pub fn log_dir(identifier: &str) -> Result<PathBuf, String> {
let os = if cfg!(target_os = "macos") {
dirs::home_dir().map(|home| home.join("Library/Logs").join(identifier))
} else {
dirs::data_local_dir().map(|dir| dir.join(identifier).join("logs"))
};
match std::env::var_os(LOG_ENV).filter(|v| !v.is_empty()) {
Some(var) => resolve_env(LOG_ENV, Some(var), None, identifier),
None => os.ok_or_else(|| "cannot find the log directory".into()),
}
}
/// The directory under `$<env>` when set (absolute, not created here), else under the OS directory.
fn resolve_env(env: &str, var: Option<OsString>, os: Option<PathBuf>, identifier: &str) -> Result<PathBuf, String> {
let Some(root) = var.filter(|v| !v.is_empty()).map(PathBuf::from) else {
return os.map(|dir| dir.join(identifier)).ok_or_else(|| format!("cannot find the directory {env} replaces"));
};
// The working directory depends on how the IDE was launched.
if !root.is_absolute() {
return Err(format!("{env} must be an absolute path: {}", root.display()));
}
Ok(root.join(identifier))
}
/// A custom local-data root chosen in Settings › System Settings › Caches,
/// stored in the configuration directory. The environment variable still wins.
@@ -51,11 +94,11 @@ fn startup_resolve(
/// The file holding the custom local-data root, if the platform exposes its
/// configuration directory. Both startup and Settings use this one path.
pub fn override_path(identifier: &str) -> Option<PathBuf> {
dirs::config_dir().map(|dir| dir.join(identifier).join(OVERRIDE_FILE))
config_dir(identifier).ok().map(|dir| dir.join(OVERRIDE_FILE))
}
fn migration_path(identifier: &str) -> Option<PathBuf> {
dirs::config_dir().map(|dir| dir.join(identifier).join(MIGRATION_FILE))
config_dir(identifier).ok().map(|dir| dir.join(MIGRATION_FILE))
}
/// The custom root stored by Settings, if any (validated when it is used).
@@ -493,6 +536,25 @@ fn relative_wrapper(text: &str, source: &Path) -> String {
mod tests {
use super::*;
#[test]
fn config_data_and_log_dirs_follow_the_os_unless_their_variable_is_set() {
let os = Some(PathBuf::from("/os"));
for env in [CONFIG_ENV, DATA_ENV, LOG_ENV] {
for var in [None, Some(OsString::new())] {
assert_eq!(resolve_env(env, var, os.clone(), "id").unwrap(), PathBuf::from("/os/id"));
}
let root = std::env::temp_dir().join("openidea-moved");
assert_eq!(resolve_env(env, Some(root.clone().into()), os.clone(), "id").unwrap(), root.join("id"));
// Release and per-worktree debug builds stay apart under one root.
assert_ne!(
resolve_env(env, Some(root.clone().into()), None, "dev-a").unwrap(),
resolve_env(env, Some(root.clone().into()), None, "dev-b").unwrap()
);
let error = resolve_env(env, Some("relative/dir".into()), os.clone(), "id").unwrap_err();
assert!(error.contains(env) && error.contains("absolute"), "{error}");
}
}
#[test]
fn local_dir_follows_the_os_unless_the_variable_is_set() {
let os = Some(PathBuf::from("/os-local"));
+14
Переглянути файл
@@ -3,6 +3,7 @@
//! wraps one [`Host`] and adds windows and menus; `openidea-server`
//! runs one on a remote machine and serves it over JSON-RPC ([`rpc`]).
pub mod agent_secrets;
pub mod data_dirs;
pub mod environment;
mod formatter;
@@ -59,7 +60,10 @@ pub struct Host {
plugin_host: RwLock<std::sync::Weak<plugins::PluginHost>>,
pub mcp_reports: mcp::Reports,
pub mcp_requests: mcp::Requests,
/// Commands agents run in the Run tool window (`run_command`).
pub mcp_agent_runs: mcp::AgentRuns,
pub trust: trust::TrustStore,
pub(crate) mcp_approvals: Arc<trust::McpServerApprovals>,
pub config_dir: PathBuf,
/// Persistent user state, including credentials and recovery history.
pub data_dir: PathBuf,
@@ -84,10 +88,15 @@ pub struct Host {
pub remote: Arc<ide_remote::RemoteService>,
/// What child processes get beyond this process's environment (Reload Environment, `remoteEnv`).
pub environment: environment::Environment,
/// The built-in web server (Open In › Browser, the editor's browser bar). On a remote host the
/// client forwards its ports.
pub web: ide_web::WebServer,
/// The HTTP proxy's saved password (see `proxy.rs`).
proxy_password: ide_ssh::secrets::SecretFile,
/// The proxy password when it could not be saved (no credential store).
proxy_session_password: Mutex<Option<String>>,
/// Values of agents' and MCP servers' environment variables and headers (see `agent_secrets.rs`).
pub agent_secrets: Arc<agent_secrets::AgentSecrets>,
pub(crate) sink_for: SinkFactory,
owner: fn(&str) -> &str,
pub runtime: tokio::runtime::Handle,
@@ -134,12 +143,16 @@ impl Host {
mcp_executable: RwLock::default(),
mcp_reports: Default::default(),
mcp_requests: Default::default(),
mcp_agent_runs: Default::default(),
trust: trust::TrustStore::load(&config_dir, &data_dir),
mcp_approvals: Arc::new(trust::McpServerApprovals::load(&data_dir)),
proxy_password: ide_ssh::secrets::SecretFile::new(data_dir.join("proxy-password.json")),
proxy_session_password: Mutex::default(),
agent_secrets: Arc::new(agent_secrets::AgentSecrets::new(&data_dir)),
remote: Arc::new(ide_remote::RemoteService::new(ssh.clone(), data_dir.join("remote"))),
ssh,
environment: environment::Environment::new(project_environment, local_data_dir.clone()),
web: ide_web::WebServer::new(),
global_settings,
dependency_versions: Arc::new(ide_workspace::dependencies::VersionCache::new(
data_dir.join("cache/dependency-versions"),
@@ -259,6 +272,7 @@ impl Host {
fn begin_close(&self, window: &str) -> Option<std::thread::JoinHandle<()>> {
self.mcp_reports.forget_window(window);
self.mcp_requests.forget_window(window);
self.mcp_agent_runs.forget_window(window);
self.external_logs.write().remove(self.owner_window(window));
for request in self.project_requests.write().remove(self.owner_window(window)).unwrap_or_default() {
self.cancel_request(request);
+176
Переглянути файл
@@ -0,0 +1,176 @@
//! `check_files`: fresh diagnostics from the language servers right after an edit, and why a
//! file could not be checked (no server for it, not installed, crashed, misconfigured…).
use super::editors::problem;
use crate::mcp::run::timeout;
use crate::mcp::tool::{Call, Output, Param, ParamType, Tool};
use ide_core::{IdeError, IdeResult, Severity};
use ide_lsp::DocumentCheck;
use serde_json::{Value, json};
use std::path::PathBuf;
use std::time::{Duration, Instant};
const DEFAULT_TIMEOUT_MS: usize = 30_000;
const MAX_FILES: usize = 20;
pub fn check_files() -> Tool {
Tool::new(
"check_files",
&format!(
"Checks files with the language servers now and returns their problems: call it after changing files \
(with any tool, inside or outside the IDE). Each file is synced from disk and saved to its servers, so checks that \
run on save (rust-analyzer's cargo check) run too; then the call waits until every server answered for the current \
text and finished the work it started, up to `timeout` milliseconds (default {DEFAULT_TIMEOUT_MS}) for all files. \
Unlike get_file_problems, this is a fresh analysis, before the inspection profile's filtering.
Per file, `status` says what the problems are worth:
- checked: every server answered; `problems` is complete (empty means no problems).
- incomplete: some server was still analysing when the wait ended, or could not check; call again or raise `timeout`.
- serverUnavailable: no server could check. `servers` gives each one's `status` (missing: not installed; \
misconfigured; crashed; starting: did not finish starting) and `message`; tell the user, since the IDE has to be \
fixed (Settings › Languages & Frameworks › Language Servers), and check another way meanwhile (a build or a compiler).
- noLanguageServer: no language server is configured for this kind of file.
- untrusted: language servers don't run in an untrusted project.
- failed: the file could not be opened (`error`).
`unsavedInEditor` is true when the file has unsaved changes in the IDE: the problems are for the editor's text, not \
the file on disk. Lines and UTF-16 columns are 1-based."
),
|call| async move {
let raw = call.str_list("filePaths")?;
if raw.is_empty() {
return Err(IdeError::invalid("filePaths must name at least one file"));
}
if raw.len() > MAX_FILES {
return Err(IdeError::invalid(format!("at most {MAX_FILES} files at a time")));
}
let errors_only = call.flag("errorsOnly", true)?;
let limit = timeout(&call, DEFAULT_TIMEOUT_MS)?;
let mut paths = Vec::new();
for p in raw {
paths.push(call.resolve_readable(&p).await?);
}
let checks = paths.into_iter().map(|path| check_one(&call, path, limit, errors_only));
let files = futures::future::join_all(checks).await;
Output::json(json!({ "files": files }))
},
)
.param(Param::required("filePaths", ParamType::StringList, "Files to check, relative to the project root"))
.param(Param::optional(
"errorsOnly",
ParamType::Boolean,
"Whether to return only errors (default) or warnings and hints too",
))
.param(Param::optional("timeout", ParamType::Integer, "Milliseconds to wait for the servers at most"))
}
async fn check_one(call: &Call, path: PathBuf, limit: Duration, errors_only: bool) -> Value {
let mut out = json!({ "filePath": call.display(&path) });
if !call.project().is_trusted() {
out["status"] = "untrusted".into();
return out;
}
match check(call, &path, limit).await {
Ok((check, dirty, text)) => {
let servers: Vec<Value> = check
.servers
.iter()
.map(|s| {
let mut v = json!({ "name": s.name, "status": s.status, "checked": s.checked });
if let Some(m) = &s.message {
v["message"] = m.clone().into();
}
if s.incomplete {
v["incomplete"] = true.into();
}
if let Some(e) = &s.error {
v["error"] = e.clone().into();
}
v
})
.collect();
out["status"] = file_status(&check).into();
out["problems"] = check
.diagnostics
.into_iter()
.filter(|d| !errors_only || d.severity == Severity::Error)
.map(|d| problem(d, &text))
.collect::<Vec<_>>()
.into();
out["servers"] = servers.into();
if dirty {
out["unsavedInEditor"] = true.into();
}
}
Err(e) => {
out["status"] = "failed".into();
out["error"] = e.message.into();
}
}
out
}
fn file_status(check: &DocumentCheck) -> &'static str {
if check.servers.is_empty() {
"noLanguageServer"
} else if !check.servers.iter().any(|s| s.checked) {
"serverUnavailable"
} else if check.servers.iter().all(|s| s.checked && !s.incomplete) {
"checked"
} else {
"incomplete"
}
}
/// The check of `path`, whether it has unsaved changes in the IDE, and its text.
async fn check(call: &Call, path: &std::path::Path, limit: Duration) -> IdeResult<(DocumentCheck, bool, String)> {
let project = call.project().clone();
let docs = project.documents.clone();
let p = path.to_owned();
let at = Instant::now();
// Synced now: the file watcher's reload of a change made outside the IDE may not have run.
let (doc, changed, dirty) = call
.host()
.blocking(move || {
let before = docs.by_path(&p).and_then(|d| docs.version(d).ok().map(|v| (d, v)));
docs.on_disk_changed(std::slice::from_ref(&p));
let doc = docs.open(&p)?.id;
let changed = before.is_none_or(|(d, v)| d != doc || docs.version(d).is_ok_and(|now| now != v));
Ok((doc, changed, docs.is_dirty(doc).unwrap_or(false)))
})
.await?;
let result = project.lsp.check_document(doc, changed.then_some(at), limit, &call.cancel).await;
let text = project.documents.text_snapshot(doc).map(|t| t.text).unwrap_or_default();
let docs = project.documents.clone();
let _ = call
.host()
.blocking(move || {
docs.close(doc);
Ok(())
})
.await;
Ok((result?, dirty, text))
}
#[cfg(test)]
mod tests {
use super::*;
use ide_core::event::ServerStatus;
use ide_lsp::CheckedServer;
fn server(status: ServerStatus, checked: bool, incomplete: bool) -> CheckedServer {
CheckedServer { name: "s".into(), status, message: None, checked, incomplete, error: None }
}
#[test]
fn statuses_say_whether_the_problems_are_complete() {
let check = |servers| file_status(&DocumentCheck { diagnostics: Vec::new(), servers });
assert_eq!(check(vec![]), "noLanguageServer");
assert_eq!(check(vec![server(ServerStatus::Missing, false, false)]), "serverUnavailable");
assert_eq!(check(vec![server(ServerStatus::Ready, true, false)]), "checked");
assert_eq!(check(vec![server(ServerStatus::Busy, true, true)]), "incomplete");
assert_eq!(
check(vec![server(ServerStatus::Ready, true, false), server(ServerStatus::Crashed, false, false)]),
"incomplete"
);
}
}
+14 -13
Переглянути файл
@@ -87,7 +87,7 @@ async fn create_or_replace(call: &Call, path: PathBuf, text: Option<String>, ove
return Ok(());
}
let edit = crate::ops::lsp::prepare_workspace_edit(&call.host, call.window(), edit).await?;
crate::ops::lsp::apply_workspace_edit(&call.host, call.window(), edit).await
crate::ops::lsp::apply_agent_workspace_edit(&call.host, call.window(), edit).await
}
/// The open document of `path`, or a transient one (`true`: close it after use).
@@ -112,16 +112,17 @@ fn whole_range(text: &str) -> Range {
/// Whether two paths name the same file (language servers report canonical paths; on a
/// case-insensitive file system `a.txt` and `A.txt` are one file).
fn same_file(a: &Path, b: &Path) -> bool {
if a == b || matches!((dunce::canonicalize(a), dunce::canonicalize(b)), (Ok(x), Ok(y)) if x == y) {
return true;
}
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
if let (Ok(x), Ok(y)) = (std::fs::metadata(a), std::fs::metadata(b)) {
return x.dev() == y.dev() && x.ino() == y.ino();
}
}
a == b || matches!((dunce::canonicalize(a), dunce::canonicalize(b)), (Ok(x), Ok(y)) if x == y) || same_inode(a, b)
}
#[cfg(unix)]
fn same_inode(a: &Path, b: &Path) -> bool {
use std::os::unix::fs::MetadataExt;
matches!((std::fs::metadata(a), std::fs::metadata(b)), (Ok(x), Ok(y)) if x.dev() == y.dev() && x.ino() == y.ino())
}
#[cfg(not(unix))]
fn same_inode(_: &Path, _: &Path) -> bool {
false
}
@@ -316,7 +317,7 @@ async fn apply_op(call: &Call, op: FileOp, notes: &mut Vec<String>) -> IdeResult
preconditions: Vec::new(),
};
let edit = crate::ops::lsp::prepare_workspace_edit(&host, call.window(), edit).await?;
crate::ops::lsp::apply_workspace_edit(&host, call.window(), edit).await
crate::ops::lsp::apply_agent_workspace_edit(&host, call.window(), edit).await
}
FileOp::Update { path: first, move_to, hunks, or } => {
// A plain diff's other names count when its `+++` path does not exist.
@@ -399,7 +400,7 @@ async fn apply_op(call: &Call, op: FileOp, notes: &mut Vec<String>) -> IdeResult
edit.files.push(FileEdit::Rename { from: path.into(), to: to.into() });
}
let edit = crate::ops::lsp::prepare_workspace_edit(&host, call.window(), edit).await?;
crate::ops::lsp::apply_workspace_edit(&host, call.window(), edit).await?;
crate::ops::lsp::apply_agent_workspace_edit(&host, call.window(), edit).await?;
if dirty {
notes.push(format!("{shown}: patched the unsaved editor buffer"));
}
+1 -1
Переглянути файл
@@ -385,7 +385,7 @@ targetIndex, or narrow the call with a contextSnippet that shows the one you mea
_ => None,
});
let (h, w) = (host.clone(), window.to_owned());
let written = crate::ops::lsp::apply_workspace_edit(&h, &w, edit).await;
let written = crate::ops::lsp::apply_agent_workspace_edit(&h, &w, edit).await;
if let Err(e) = written {
let hint = format!("The rename stopped while it wrote. {} Read the files again before you retry.", e.message);
return Ok(RenameResult::failed("write_failed", hint).with_symbol(symbol));
+1 -1
Переглянути файл
@@ -65,7 +65,7 @@ async fn current_file(call: Call) -> IdeResult<Output> {
})
}
fn problem(d: Diagnostic, text: &str) -> Value {
pub(super) fn problem(d: Diagnostic, text: &str) -> Value {
json!({
"severity": d.severity, "description": d.message,
"lineContent": text.lines().nth(d.range.start.line as usize).unwrap_or_default(),
+34 -9
Переглянути файл
@@ -30,9 +30,14 @@ fn read_file() -> Tool {
"Reads a file in the project directory or from any project dependency or other project source root.
Can read text sources inside jar/zip archives; compiled class files are not decompiled.
Returns numbered lines (1-indexed) as text.
By default, reads up to 2000 lines starting from the beginning of the file.
By default, reads up to 2000 lines starting from the beginning of the file; a longer file keeps its first and last
lines with a `…N lines truncated…` marker between them.
With `offset` or `limit`, returns exactly lines `offset` to `offset + limit - 1` (fewer at the end of the file),
followed by `…N more lines; read on with offset=M…` when the file goes on.
The maximum accepted limit is 5000 lines.",
|call| async move {
// An explicit range is read as asked; the default read of a whole file keeps its head and tail.
let window = call.opt_i64("offset")?.is_some() || call.opt_i64("limit")?.is_some();
let offset = positive(&call, "offset", 1)?;
let limit = positive(&call, "limit", DEFAULT_READ_LIMIT)?;
if limit > MAX_READ_LIMIT {
@@ -50,7 +55,7 @@ The maximum accepted limit is 5000 lines.",
)));
}
let text = crate::ops::fs::read_text(&call.host(), call.window(), path).await?;
Ok(Output::text(render_slice(&text, offset as usize, limit as usize)?))
Ok(Output::text(render_slice(&text, offset as usize, limit as usize, window)?))
},
)
.param(Param::required("file_path", ParamType::String, util::FILE_PATH_DESCRIPTION))
@@ -70,10 +75,12 @@ fn positive(call: &Call, name: &str, default: i64) -> IdeResult<i64> {
}
}
/// Lines `start..` of `text` as `L<n>: <line>`; when more than `max_lines` remain, the first
/// and the last halves of the budget with a `…N lines truncated…` marker between them.
/// Lines `start..` of `text` as `L<n>: <line>`, at most `max_lines` of them. A `window` is
/// exactly the lines from `start` on, then how many lines remain; otherwise, when more than
/// `max_lines` remain, the first and the last halves of the budget with a `…N lines truncated…`
/// marker between them.
/// Lines are the editor's: a trailing line break starts an empty last line.
fn render_slice(text: &str, start: usize, max_lines: usize) -> IdeResult<String> {
fn render_slice(text: &str, start: usize, max_lines: usize, window: bool) -> IdeResult<String> {
let lines: Vec<&str> = text.split('\n').map(|l| l.strip_suffix('\r').unwrap_or(l)).collect();
let count = lines.len();
if start > count {
@@ -86,6 +93,15 @@ fn render_slice(text: &str, start: usize, max_lines: usize) -> IdeResult<String>
if available <= max_lines {
return Ok(range(start, count));
}
if window {
let end = start + max_lines - 1;
return Ok(format!(
"{}\n\u{2026}{} more lines; read on with offset={}\u{2026}",
range(start, end),
count - end,
end + 1
));
}
let head = max_lines / 2;
let tail = max_lines - head;
let head_end = start + head - 1;
@@ -245,7 +261,7 @@ mod tests {
use super::*;
fn slice(text: &str, start: usize, max: usize) -> String {
render_slice(text, start, max).unwrap()
render_slice(text, start, max, false).unwrap()
}
#[test]
@@ -254,18 +270,27 @@ mod tests {
assert_eq!(slice("a\r\nb\n", 1, 10), "L1: a\nL2: b\nL3: ");
assert_eq!(slice("", 1, 10), "L1: ");
assert_eq!(slice("a\nb\nc", 2, 10), "L2: b\nL3: c");
assert!(render_slice("", 2, 10).unwrap_err().message.contains("offset exceeds file length"));
assert!(render_slice("a\nb", 3, 10).unwrap_err().message.contains("offset exceeds file length"));
assert!(render_slice("", 2, 10, true).unwrap_err().message.contains("offset exceeds file length"));
assert!(render_slice("a\nb", 3, 10, true).unwrap_err().message.contains("offset exceeds file length"));
}
#[test]
fn keeps_head_and_tail_of_long_slices() {
fn keeps_head_and_tail_of_a_long_default_read() {
let text = (1..=11).map(|n| format!("line {n}")).collect::<Vec<_>>().join("\n");
assert_eq!(slice(&text, 1, 4), "L1: line 1\nL2: line 2\n…7 lines truncated…\nL10: line 10\nL11: line 11");
assert_eq!(slice(&text, 3, 4), "L3: line 3\nL4: line 4\n…5 lines truncated…\nL10: line 10\nL11: line 11");
assert_eq!(slice(&text, 1, 1), "…10 lines truncated…\nL11: line 11");
}
#[test]
fn reads_an_explicit_range_exactly_and_says_where_to_read_on() {
let text = (1..=11).map(|n| format!("line {n}")).collect::<Vec<_>>().join("\n");
let window = |start, max| render_slice(&text, start, max, true).unwrap();
assert_eq!(window(5, 3), "L5: line 5\nL6: line 6\nL7: line 7\n…4 more lines; read on with offset=8…");
assert_eq!(window(1, 1), "L1: line 1\n…10 more lines; read on with offset=2…");
assert_eq!(window(10, 3), "L10: line 10\nL11: line 11", "the end of the file needs no marker");
}
#[test]
fn cuts_long_lines_at_utf16_units() {
let long = "x".repeat(2100);
+4 -1
Переглянути файл
@@ -1,8 +1,9 @@
//! Built-in IDE tools. Transport and discovery live in `ide-mcp`.
mod check;
mod edit;
mod editors;
mod requests;
mod run;
pub(crate) mod run;
mod vcs;
pub use requests::Requests;
mod files;
@@ -16,6 +17,7 @@ use crate::Host;
use ide_core::{CancellationToken, IdeError};
use ide_mcp::{ToolCallError, ToolRegistry};
pub use reports::{FileProblems, FocusedEditor, Reports};
pub use run::agent::{AGENT_PARAM, AgentRun, AgentRuns, NAME_PREFIX, SHELL_PARAM};
use std::path::{Path, PathBuf};
use std::sync::Arc;
@@ -30,6 +32,7 @@ fn tools() -> Vec<tool::Tool> {
let mut tools = files::tools();
tools.extend(search::tools());
tools.extend(editors::tools());
tools.push(check::check_files());
tools.push(symbols::get_symbol_info());
tools.push(symbols::find_references());
tools.extend(edit::tools());
File diff suppressed because it is too large. Load diff
+418
Переглянути файл
@@ -0,0 +1,418 @@
//! `run_command` and the tools that follow its tab up: `wait_for_run`, `read_run_output`,
//! `get_run_status`, `send_run_input`, `stop_run` and `list_runs` (see [`super::agent`]).
use super::agent::{AgentRun, INTERVENTION_ADVICE, TruncateMode, clean_output, tab_name, truncate_lines};
use super::output::RUN_PREVIEW_MAX_LENGTH;
use crate::mcp::run::timeout;
use crate::mcp::tool::{Call, Output, Param, ParamType, Tool};
use ide_core::{IdeError, IdeResult};
use serde_json::{Map, Value};
use std::sync::Arc;
const RUN_TIMEOUT_MS: usize = 10_000;
const WAIT_TIMEOUT_MS: usize = 30_000;
const STOP_TIMEOUT_MS: usize = 10_000;
/// Lines of output returned at most, by default (as JetBrains' `execute_terminal_command`).
const DEFAULT_MAX_LINES: usize = 2000;
/// Bytes of the log read for one result at most.
const READ_CAP: usize = 1 << 20;
/// Lines of output quoted when a name is still running.
const BUSY_TAIL_LINES: usize = 20;
const NAME: &str = "Tab name. OpenIDEA shows the tab as `_agent_<name>`; passing the prefixed name works too.";
const SEQUENCE: &str = "\n\nUsing the same name for every command of one task reuses its tab: a name whose command \
has finished starts the next command in the same tab. A name whose command is still running is refused (with that \
command and its last output); stop it with `stop_run` first or use another name. Nothing is stopped implicitly.";
const INTERVENTIONS: &str = "\n\nThe user sees every tab and can type into it, rerun or stop its command, or close \
it. Results then carry `interventions` (for example `User input: …` or `The user reran this command at …`) and \
`advice`: unless the user asked otherwise, stop what you are doing, tell the user you received the intervention and \
wait for their confirmation before you continue.";
const OUTPUT: &str = "\n\nThe command runs on pipes, not a terminal, so programs print plain lines (no progress \
bars). `output` is that text cleaned of escape sequences, at most `maxLinesCount` lines (default 2000): beyond it, \
`truncateMode` START (default) keeps the end, END the start, MIDDLE both ends, NONE everything, with a \
`<<<...content truncated...>>>` line where lines were left out and `truncatedLines` saying how many. `outputStart` \
and `nextOffset` are positions in the tab's raw output log; pass `nextOffset` as `offset` to read on.";
const MAX_LINES: &str = "Lines of output to return at most (default 2000)";
const TRUNCATE_MODE: &str = "Which lines to leave out beyond maxLinesCount: START (default, keeps the end), END \
(keeps the start), MIDDLE (keeps both ends) or NONE";
fn find(call: &Call) -> IdeResult<Arc<AgentRun>> {
let name = tab_name(&call.str("name")?)?;
call.host()
.mcp_agent_runs
.get(call.root(), &name)
.ok_or_else(|| IdeError::not_found(format!("no agent tab named `{name}` (see list_runs)")))
}
/// `status` of `run`, with interventions not told yet.
fn result(run: &AgentRun, extra: Map<String, Value>) -> IdeResult<Output> {
let mut out = run.status();
out.extend(extra);
let interventions = run.take_interventions();
if !interventions.is_empty() {
out.insert("interventions".into(), interventions.into());
out.insert("advice".into(), INTERVENTION_ADVICE.into());
}
Output::json(Value::Object(out))
}
/// The output from `from` as the agent reads it: cleaned, then cut to `maxLinesCount` lines
/// as `truncateMode` says.
fn output_view(call: &Call, run: &AgentRun, from: u64) -> IdeResult<Map<String, Value>> {
let max_lines = match call.opt_i64("maxLinesCount")? {
Some(n) if n < 1 => return Err(IdeError::invalid("maxLinesCount must be > 0")),
Some(n) => n as usize,
None => DEFAULT_MAX_LINES,
};
let mode = match call.opt_str("truncateMode")? {
Some(name) => TruncateMode::parse(&name)
.ok_or_else(|| IdeError::invalid("truncateMode must be START, MIDDLE, END or NONE"))?,
None => TruncateMode::Start,
};
let len = run.log_len();
let from = from.min(len);
// A long output is read from the end that stays.
let start = if mode == TruncateMode::Start && len - from > READ_CAP as u64 { len - READ_CAP as u64 } else { from };
let (raw, start, end) = run.read(start, READ_CAP)?;
let (text, truncated) = truncate_lines(&clean_output(&raw), max_lines, mode);
let mut out = Map::new();
out.insert("output".into(), text.into());
if truncated > 0 {
out.insert("truncatedLines".into(), truncated.into());
}
out.insert("outputStart".into(), start.into());
out.insert("nextOffset".into(), end.into());
Ok(out)
}
fn deadline(call: &Call, default: usize) -> IdeResult<tokio::time::Instant> {
Ok(tokio::time::Instant::now() + timeout(call, default)?)
}
fn pattern(call: &Call) -> IdeResult<Option<regex_lite::Regex>> {
call.opt_str("pattern")?
.filter(|p| !p.is_empty())
.map(|p| regex_lite::Regex::new(&p).map_err(|e| IdeError::invalid(format!("invalid pattern: {e}"))))
.transpose()
}
fn offset(call: &Call, name: &str) -> IdeResult<Option<u64>> {
match call.opt_i64(name)? {
Some(n) if n < 0 => Err(IdeError::invalid(format!("{name} must not be negative"))),
n => Ok(n.map(|n| n as u64)),
}
}
fn busy(run: &AgentRun) -> IdeError {
let tail = run.tail(run.process_offset(), BUSY_TAIL_LINES, RUN_PREVIEW_MAX_LENGTH).map(|t| t.0).unwrap_or_default();
IdeError::invalid(format!(
"`{}` is still running `{}`. Stop it with stop_run first, or use another name.\nLast output:\n{}",
run.name,
run.command(),
clean_output(&tail).trim_end()
))
}
pub fn run_command() -> Tool {
Tool::new(
"run_command",
&format!(
"Runs a shell command line in a new or reused tab of OpenIDEA's Run tool window, with the shell of \
Settings › Tools › Terminal. The user is asked first (unless the IDE runs tools without \
confirmation). Waits up to `timeout` milliseconds (default {RUN_TIMEOUT_MS}) for the command to finish; a command \
still running then keeps running, and `wait_for_run`, `read_run_output`, `get_run_status`, `send_run_input` and \
`stop_run` follow it up by name. Returns the tab's status (`state`: starting, running, stopping, succeeded, failed, \
crashed, stoppedByAgent, stoppedByUser or startFailed; `exitCode`; `crash` with the signal or exception of a crashed \
process) and its output so far.{OUTPUT}{SEQUENCE}{INTERVENTIONS}"
),
|call| async move {
let name = tab_name(&call.str("name")?)?;
let command = call.str("command")?;
if command.trim().is_empty() {
return Err(IdeError::invalid("command must not be empty"));
}
let cwd = match call.opt_str("workingDirectory")?.filter(|d| !d.trim().is_empty()) {
Some(dir) => {
let path = call.resolve_inside(&dir)?;
if !path.is_dir() {
return Err(IdeError::not_found(format!("workingDirectory {} is not a directory", dir)));
}
path
}
None => call.root().to_path_buf(),
};
let env = call.args.get("envs").filter(|v| !v.is_null()).map(|_| call.str_map("envs")).transpose()?;
let deadline = deadline(&call, RUN_TIMEOUT_MS)?;
let host = call.host();
let runs = &host.mcp_agent_runs;
if let Some(run) = runs.get(call.root(), &name).filter(|r| r.running()) {
return Err(busy(&run));
}
let shown_cwd = call.display(&cwd);
let mut detail = format!(
"{command}\n\nTab: {name}\nWorking directory: {}",
if shown_cwd.is_empty() { ".".to_owned() } else { shown_cwd.clone() }
);
for (key, value) in env.iter().flatten() {
detail.push_str(&format!("\n{key}={value}"));
}
call.confirm(&name, Some(detail)).await?;
let run = runs.prepare(call.root(), call.window(), &name, &command, &shown_cwd).map_err(|r| busy(&r))?;
let request = ide_core::event::McpUiRequest::AgentRun {
name: name.clone(),
command: command.clone(),
cwd: cwd.into(),
env: env.unwrap_or_default().into_iter().collect(),
};
if let Err(e) = call.ask(request).await {
let error = run.start_error();
runs.abandon(&run);
return Err(match error {
Some(error) => IdeError::invalid(format!("`{name}` did not start: {error}")),
None => e,
});
}
if let Some(error) = run.start_error() {
return Err(IdeError::invalid(format!("`{name}` did not start: {error}")));
}
let from = run.process_offset();
let end = run.wait(deadline, None, from, &call.cancel).await?;
let mut extra = output_view(&call, &run, from)?;
extra.insert("waitEnded".into(), end.as_str().into());
result(&run, extra)
},
)
.write()
.param(Param::required("name", ParamType::String, NAME))
.param(Param::required(
"command",
ParamType::String,
"Shell command line, e.g. `npm test -- --watch=false` or `cargo build && ./target/debug/app`",
))
.param(Param::optional(
"workingDirectory",
ParamType::String,
"Directory inside the project to run in, relative to the project root; default: the project root",
))
.param(Param::optional("envs", ParamType::StringMap, "Environment variables added for this command"))
.param(Param::optional(
"timeout",
ParamType::Integer,
"Milliseconds to wait for the command to finish before returning (0 returns once it started); the command \
keeps running after it",
))
.param(Param::optional("maxLinesCount", ParamType::Integer, MAX_LINES))
.param(Param::optional("truncateMode", ParamType::String, TRUNCATE_MODE))
}
pub fn wait_for_run() -> Tool {
Tool::new(
"wait_for_run",
&format!(
"Waits until the command of an agent tab finishes, its output matches `pattern`, the user intervenes, or \
`timeout` milliseconds pass (default {WAIT_TIMEOUT_MS}). Cancelling the wait leaves the command running. Returns \
`waitEnded` (exited, matched, userIntervened or timedOut), the tab's status and the output since `offset`. Use \
`pattern` to wait for a server to be ready, e.g. `Listening on|Started .* in`.{OUTPUT}{INTERVENTIONS}"
),
|call| async move {
let run = find(&call)?;
let deadline = deadline(&call, WAIT_TIMEOUT_MS)?;
let pattern = pattern(&call)?;
let from = offset(&call, "offset")?.unwrap_or_else(|| run.process_offset());
let end = run.wait(deadline, pattern.as_ref(), from, &call.cancel).await?;
let mut extra = output_view(&call, &run, from)?;
extra.insert("waitEnded".into(), end.as_str().into());
result(&run, extra)
},
)
.param(Param::required("name", ParamType::String, NAME))
.param(Param::optional("timeout", ParamType::Integer, "Milliseconds to wait at most"))
.param(Param::optional(
"pattern",
ParamType::String,
"Regular expression to wait for in the output after `offset` (as the command wrote it)",
))
.param(Param::optional(
"offset",
ParamType::Integer,
"Where in the tab's output log to start: a `nextOffset` from an earlier result. Default: where the current \
command's output starts",
))
.param(Param::optional("maxLinesCount", ParamType::Integer, MAX_LINES))
.param(Param::optional("truncateMode", ParamType::String, TRUNCATE_MODE))
}
pub fn read_run_output() -> Tool {
Tool::new(
"read_run_output",
&format!(
"Reads the output log of an agent tab, which goes on across the commands and reruns of the tab: from \
`offset` (a `nextOffset` from an earlier result), or from its start. Also returns the tab's status.\
{OUTPUT}{INTERVENTIONS}"
),
|call| async move {
let run = find(&call)?;
let from = offset(&call, "offset")?.unwrap_or(0);
let extra = output_view(&call, &run, from)?;
result(&run, extra)
},
)
.param(Param::required("name", ParamType::String, NAME))
.param(Param::optional("offset", ParamType::Integer, "Where in the log to read from"))
.param(Param::optional("maxLinesCount", ParamType::Integer, MAX_LINES))
.param(Param::optional("truncateMode", ParamType::String, TRUNCATE_MODE))
}
pub fn get_run_status() -> Tool {
Tool::new(
"get_run_status",
&format!(
"Returns the status of an agent tab: its command, `state` (starting, running, stopping, succeeded, \
failed, crashed, stoppedByAgent, stoppedByUser or startFailed), `startedBy` (agent, or user when the user reran \
it), times, `exitCode`, `crash`, `outputLength`, `lastOutputAt` (a running command silent for long may be waiting \
for input) and `tabClosedAt` when the user closed the tab.{INTERVENTIONS}"
),
|call| async move {
let run = find(&call)?;
result(&run, Map::new())
},
)
.param(Param::required("name", ParamType::String, NAME))
}
pub fn list_runs() -> Tool {
Tool::new(
"list_runs",
"Lists the agent tabs of the project (those started by `run_command`) with their status.",
|call| async move {
let runs: Vec<Value> =
call.host().mcp_agent_runs.list(call.root()).iter().map(|r| Value::Object(r.status())).collect();
Output::json(serde_json::json!({ "runs": runs }))
},
)
}
/// What a key does to a command on pipes.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum Key {
/// Writes this text to its input.
Text(&'static str),
/// Ctrl+C: stops it, as `stop_run` does.
Interrupt,
/// Ctrl+D: ends its input.
EndOfInput,
}
fn key(name: &str) -> IdeResult<Key> {
Ok(match name.trim().to_ascii_lowercase().replace(' ', "").as_str() {
"enter" | "return" => Key::Text("\n"),
"tab" => Key::Text("\t"),
"ctrl+c" => Key::Interrupt,
"ctrl+d" => Key::EndOfInput,
"escape" | "esc" | "backspace" | "ctrl+z" | "ctrl+\\" | "ctrl+l" | "up" | "down" | "left" | "right" => {
return Err(IdeError::invalid(format!(
"`{name}` is a terminal key, and the command runs on pipes, not a terminal: send text and Enter, \
Ctrl+C to stop it or Ctrl+D to end its input"
)));
}
_ => return Err(IdeError::invalid(format!("unknown key `{name}`"))),
})
}
pub fn send_run_input() -> Tool {
Tool::new(
"send_run_input",
"Writes to the input of the running command of an agent tab, which runs on pipes: `text` as it is, then \
`keys` in order: Enter (a line break), Tab, Ctrl+C (stops the command, as `stop_run` does) and Ctrl+D (ends its \
input). Terminal keys (arrows, Escape, Backspace, Ctrl+Z…) are refused. To answer a prompt, pass the answer as \
`text` and `keys: [\"Enter\"]`. The user sees the command's output in the tab. Returns the tab's status.",
|call| async move {
let run = find(&call)?;
let text = call.opt_str("text")?.unwrap_or_default();
let keys = call.str_list("keys")?.iter().map(|k| key(k)).collect::<IdeResult<Vec<_>>>()?;
if text.is_empty() && keys.is_empty() {
return Err(IdeError::invalid("pass `text`, `keys` or both"));
}
let id = run.live_process().ok_or_else(|| IdeError::invalid(format!("`{}` is not running", run.name)))?;
call.project().require_trust()?;
let host = call.host();
let mut data = text;
for key in keys {
match key {
Key::Text(text) => data.push_str(text),
Key::EndOfInput | Key::Interrupt => {
if !data.is_empty() {
host.processes.write_stdin(id, &std::mem::take(&mut data))?;
}
if key == Key::EndOfInput {
crate::ops::run::close_stdin(&host, id)?;
} else {
run.agent_stopping();
crate::ops::run::stop(&host, &run.window(), id)?;
break;
}
}
}
}
if !data.is_empty() {
host.processes.write_stdin(id, &data)?;
}
result(&run, Map::new())
},
)
.write()
.param(Param::required("name", ParamType::String, NAME))
.param(Param::optional("text", ParamType::String, "Text to type"))
.param(Param::optional("keys", ParamType::StringList, "Keys to press after `text`"))
}
pub fn stop_run() -> Tool {
Tool::new(
"stop_run",
&format!(
"Stops the running command of an agent tab, the way the Run tool window's Stop does: it asks the \
process tree to exit and, with `force`, kills it. Waits up to `timeout` milliseconds (default {STOP_TIMEOUT_MS}) \
for it to end and returns the tab's status (`stoppedByAgent` once it ended). A process that ignores the request \
keeps `stopping`: call again with `force`. The tab stays open with its output."
),
|call| async move {
let run = find(&call)?;
let id = run.live_process().ok_or_else(|| IdeError::invalid(format!("`{}` is not running", run.name)))?;
let force = call.flag("force", false)?;
let deadline = deadline(&call, STOP_TIMEOUT_MS)?;
let host = call.host();
run.agent_stopping();
crate::ops::run::stop(&host, &run.window(), id)?;
if force {
// A second Stop kills the tree.
crate::ops::run::stop(&host, &run.window(), id)?;
}
let end = run.wait(deadline, None, run.log_len(), &call.cancel).await?;
let mut extra = Map::new();
extra.insert("waitEnded".into(), end.as_str().into());
result(&run, extra)
},
)
.write()
.param(Param::required("name", ParamType::String, NAME))
.param(Param::optional("force", ParamType::Boolean, "Kill the process tree instead of asking it to exit"))
.param(Param::optional("timeout", ParamType::Integer, "Milliseconds to wait for the command to end"))
}
pub fn tools() -> Vec<Tool> {
vec![run_command(), wait_for_run(), read_run_output(), get_run_status(), send_run_input(), stop_run(), list_runs()]
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn keys_are_what_pipes_can_take() {
assert_eq!(key(" enter ").unwrap(), Key::Text("\n"));
assert_eq!(key("Ctrl+C").unwrap(), Key::Interrupt);
assert_eq!(key("ctrl+d").unwrap(), Key::EndOfInput);
assert!(key("Up").unwrap_err().message.contains("runs on pipes"));
assert!(key("F13").unwrap_err().message.contains("unknown key"));
}
}
+2 -3
Переглянути файл
@@ -15,7 +15,6 @@ use std::sync::Arc;
use std::time::Instant;
const DEFAULT_TIMEOUT_MS: usize = 60_000;
const MAX_TIMEOUT_MS: usize = 3_600_000;
const MAX_PROBLEMS: usize = 300;
/// Console lines attached to a failure whose messages were not parsed.
const CONSOLE_LINES: usize = 200;
@@ -84,13 +83,13 @@ You have to use this tool after performing edits to validate if the edits are va
In OpenIDEA the project's build system builds it (Maven, Gradle, Cargo, Go or TypeScript); with `filesToRebuild`, the \
modules containing those files. Asks the user first for every call. Problems have the project-relative \
`file` and 1-based `line` / `column`; at most 300 problems are listed, and `isSuccess` is false whenever one of them \
is an ERROR. On timeout (at most 1 hour; 0 times out at once) the build is stopped.",
is an ERROR. On timeout (0 times out at once) the build is stopped.",
|call| async move {
let files: Vec<String> =
call.str_list("filesToRebuild")?.into_iter().filter(|f| !f.trim().is_empty()).collect();
let rebuild = call.flag("rebuild", false)? && files.is_empty();
let kind = if rebuild { BuildKind::Rebuild } else { BuildKind::Build };
let limit = timeout(&call, DEFAULT_TIMEOUT_MS, MAX_TIMEOUT_MS)?;
let limit = timeout(&call, DEFAULT_TIMEOUT_MS)?;
let mut targets: Vec<Option<PathBuf>> = Vec::new();
let mut dirs: Vec<PathBuf> = Vec::new();
for f in &files {
+1 -3
Переглянути файл
@@ -19,8 +19,6 @@ use std::sync::Arc;
use std::time::{Duration, Instant};
const DEFAULT_TIMEOUT_MS: usize = 10_000;
/// No limit in JetBrains; the process keeps running after the timeout anyway.
const MAX_TIMEOUT_MS: usize = 1_800_000;
/// Output of a failed before-launch process kept for the error message.
const MAX_PROCESS_OUTPUT_LENGTH: usize = 20_000;
/// A before-launch build gets this long.
@@ -541,7 +539,7 @@ tool window; output files are private and deleted after a day.",
)?;
let wait = call.flag("waitForExit", true)?;
// As JetBrains: `timeout` is ignored without waiting, and 0 returns without waiting.
let limit = if wait { Some(timeout(&call, DEFAULT_TIMEOUT_MS, MAX_TIMEOUT_MS)?) } else { None };
let limit = if wait { Some(timeout(&call, DEFAULT_TIMEOUT_MS)?) } else { None };
let envs = call.args.get("envs").filter(|v| !v.is_null()).map(|_| call.str_map("envs")).transpose()?;
let configs = load_configs(&call).await?;
let mut config = match &target {
+14 -5
Переглянути файл
@@ -1,5 +1,7 @@
//! Run configurations and builds; every call asks the user first.
//! Run configurations, builds and agent commands; every start asks the user first.
pub(crate) mod agent;
mod agent_tools;
mod build;
mod configs;
mod output;
@@ -11,15 +13,22 @@ use ide_core::IdeResult;
use std::time::Duration;
pub fn tools() -> Vec<Tool> {
vec![configs::get_run_configurations(), configs::execute_run_configuration(), build::build_project()]
let mut tools =
vec![configs::get_run_configurations(), configs::execute_run_configuration(), build::build_project()];
tools.extend(agent_tools::tools());
tools
}
/// The `timeout` argument: `default` when absent, at most `max` milliseconds. As in JetBrains
/// Longest wait: far enough to be no limit, near enough not to overflow a deadline.
const FOREVER_MS: u64 = 365 * 24 * 60 * 60 * 1000;
/// The `timeout` argument in milliseconds, `default` when absent: the agent decides how long
/// to wait (long waits are answered on an event stream, see `ide_mcp`). As in JetBrains
/// (`withTimeoutOrNull`), zero or a negative value times out at once.
fn timeout(call: &Call, default: usize, max: usize) -> IdeResult<Duration> {
pub(crate) fn timeout(call: &Call, default: usize) -> IdeResult<Duration> {
Ok(match call.opt_i64("timeout")? {
None => Duration::from_millis(default as u64),
Some(n) if n <= 0 => Duration::ZERO,
Some(n) => Duration::from_millis((n as u64).min(max as u64)),
Some(n) => Duration::from_millis((n as u64).min(FOREVER_MS)),
})
}
+42 -2
Переглянути файл
@@ -434,10 +434,23 @@ fn cwd_of(root: &Path, c: &RunConfig) -> String {
cwd.trim_end_matches('/').to_owned()
}
/// Configs whose working directory most specifically contains `rel` (ties in their original order).
/// The directory a config covers: a Cargo workspace member's own directory (its `cargo test -p`
/// runs from the workspace root), else its working directory.
fn scope_of(root: &Path, c: &RunConfig) -> String {
match c.extras.module.as_deref() {
Some(module) if c.kind == "cargo" => {
let module = module.replace('\\', "/");
let module = module.strip_prefix("./").unwrap_or(&module);
if module == "." { String::new() } else { module.trim_end_matches('/').to_owned() }
}
_ => cwd_of(root, c),
}
}
/// Configs whose scope ([`scope_of`]) most specifically contains `rel` (ties in their original order).
fn all_closest<'a>(root: &Path, configs: &[&'a RunConfig], rel: &str) -> Vec<&'a RunConfig> {
let score = |c: &RunConfig| -> Option<usize> {
let cwd = cwd_of(root, c);
let cwd = scope_of(root, c);
if cwd.is_empty() { Some(0) } else { (rel == cwd || rel.starts_with(&format!("{cwd}/"))).then_some(cwd.len()) }
};
let scored: Vec<(&RunConfig, usize)> = configs.iter().filter_map(|c| score(c).map(|s| (*c, s))).collect();
@@ -922,6 +935,33 @@ mod tests {
assert_eq!(c.args, ["test", "--", "math::tests::"]);
}
#[test]
fn a_workspace_member_runs_its_own_tests() {
let root = Path::new("/p");
let member = |pkg: &str, dir: &str| {
let mut c = blank(
format!("cargo:test:{pkg}"),
"test",
"cargo",
"cargo".into(),
vec!["test".into(), "-p".into(), pkg.into()],
);
c.is_test = true;
c.extras.module = Some(dir.into());
c
};
let mut workspace = blank("cargo:test".into(), "test", "cargo", "cargo".into(), vec!["test".into()]);
workspace.is_test = true;
let configs = vec![member("ide-agent", "crates/ide-agent"), member("ide-mcp", "crates/ide-mcp"), workspace];
let src = "#[cfg(test)]\nmod tests {\n #[test]\n fn adds() {}\n}\n";
let rel = "crates/ide-mcp/src/stdio.rs";
let ts = detect(src, rel);
let c = target_config(root, &ts[1], rel, &configs, src).unwrap();
assert_eq!(c.args, ["test", "-p", "ide-mcp", "--", "stdio::tests::adds", "--exact"]);
let c = target_config(root, &ts[1], "src/lib.rs", &configs, src).unwrap();
assert!(!c.args.iter().any(|a| a == "-p"), "outside every member: the workspace's tests, {:?}", c.args);
}
#[test]
fn cargo_test_args_get_one_separator() {
let root = Path::new("/p");
+131 -2
Переглянути файл
@@ -38,6 +38,14 @@ const LINE_WINDOW: usize = 500;
const PATHS_DESCRIPTION: &str = "Optional list of project-relative glob patterns to filter results. Supports '!' \
excludes. Trailing '/' expands to '**'. Patterns without '/' are treated as '**/pattern'. Empty strings are ignored.";
const LIMIT_DESCRIPTION: &str = "Maximum number of results to return";
const CONTEXT_DESCRIPTION: &str = "Lines around each match to return besides the matched line (`lineText`): \"+N\" the \
N lines after it (`contextAfter`), \"-N\" the N lines before it (`contextBefore`), \"N\" (or the number N) N lines on \
each side. At most 50. Default: the matched line only.";
/// A matched line longer than this (UTF-16 units) is cut to the match and [`SNIPPET_AROUND`] units
/// on each side of it; a longer context line to its start.
const MAX_LINE_TEXT: usize = 300;
const SNIPPET_AROUND: usize = 100;
const MAX_CONTEXT_LINES: u32 = 50;
pub fn tools() -> Vec<Tool> {
vec![search_text(), search_regex(), search_symbol(), search_file()]
@@ -49,6 +57,8 @@ fn search_text() -> Tool {
"Searches for a text substring within project files.
Use this tool for fast text search with match coordinates.
Results include match coordinates when available (1-based line/column, end exclusive).
Each result has the matched line (`lineText`; a long line cut to the match and 100 characters on each side, marked
with …), and with `context` the lines around it.
Paths are glob patterns relative to the project root.
Examples: [\"src/**\", \"!**/test/**\"], [\"**/*.kt\"], [\"foo/\"].",
@@ -57,6 +67,7 @@ Examples: [\"src/**\", \"!**/test/**\"], [\"**/*.kt\"], [\"foo/\"].",
.param(Param::required("q", ParamType::String, "Text to search for"))
.param(Param::optional("paths", ParamType::StringList, PATHS_DESCRIPTION))
.param(Param::optional("limit", ParamType::Integer, LIMIT_DESCRIPTION))
.param(Param::optional("context", ParamType::String, CONTEXT_DESCRIPTION))
.output(result_schema())
}
@@ -66,6 +77,8 @@ fn search_regex() -> Tool {
"Searches for regex matches within project files.
Use this tool when you need regex search with match coordinates.
Results include match coordinates when available (1-based line/column, end exclusive).
Each result has the matched line (`lineText`; a long line cut to the match and 100 characters on each side, marked
with …), and with `context` the lines around it.
The pattern uses Rust regex syntax and is case-sensitive; prefix `(?i)` to ignore case.
Paths are glob patterns relative to the project root.
@@ -75,6 +88,7 @@ Examples: [\"src/**\", \"!**/test/**\"], [\"**/*.kt\"], [\"foo/\"].",
.param(Param::required("q", ParamType::String, "Regex pattern to search for"))
.param(Param::optional("paths", ParamType::StringList, PATHS_DESCRIPTION))
.param(Param::optional("limit", ParamType::Integer, LIMIT_DESCRIPTION))
.param(Param::optional("context", ParamType::String, CONTEXT_DESCRIPTION))
.output(result_schema())
}
@@ -84,6 +98,7 @@ fn search_symbol() -> Tool {
"Searches for symbols (classes, methods, fields).
Use this tool for semantic lookup by identifier fragments.
Results include match coordinates when available (1-based line/column, end exclusive).
Project results have their line (`lineText`), and with `context` the lines around it.
Paths are glob patterns relative to the project root.
By default this searches project symbols only.
@@ -99,6 +114,7 @@ If you don't find a suitable result, try again with include_external=true to sea
with include_external=true.",
))
.param(Param::optional("limit", ParamType::Integer, LIMIT_DESCRIPTION))
.param(Param::optional("context", ParamType::String, CONTEXT_DESCRIPTION))
.output(result_schema())
}
@@ -145,6 +161,13 @@ struct SearchItem {
name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
container: Option<String>,
/// The text of `start_line` (cut when long).
#[serde(skip_serializing_if = "Option::is_none")]
line_text: Option<String>,
#[serde(skip_serializing_if = "Vec::is_empty")]
context_before: Vec<String>,
#[serde(skip_serializing_if = "Vec::is_empty")]
context_after: Vec<String>,
}
impl SearchItem {
@@ -182,7 +205,9 @@ fn result_schema() -> Value {
"type": "object",
"properties": {
"filePath": string, "startLine": int, "startColumn": int, "endLine": int, "endColumn": int,
"name": string, "container": string,
"name": string, "container": string, "lineText": string,
"contextBefore": { "type": "array", "items": string },
"contextAfter": { "type": "array", "items": string },
},
"required": ["filePath"],
} },
@@ -207,6 +232,87 @@ fn query(call: &Call, empty: &str) -> IdeResult<String> {
Ok(q)
}
// --- matched lines -------------------------------------------------------------
/// The `context` argument as (lines before, lines after).
fn context(call: &Call) -> IdeResult<(u32, u32)> {
let invalid = || IdeError::invalid("context must be \"+N\", \"-N\" or N (lines after, before, or on each side)");
let (sign, n) = match call.args.get("context") {
None | Some(Value::Null) => return Ok((0, 0)),
Some(Value::Number(n)) => (None, n.as_u64().ok_or_else(invalid)?),
Some(Value::String(s)) => {
let s = s.trim();
match s.strip_prefix('+').map(|r| ('+', r)).or_else(|| s.strip_prefix('-').map(|r| ('-', r))) {
Some((sign, rest)) => (Some(sign), rest.trim().parse().map_err(|_| invalid())?),
None => (None, s.parse().map_err(|_| invalid())?),
}
}
Some(_) => return Err(invalid()),
};
let n = (n.min(MAX_CONTEXT_LINES as u64)) as u32;
Ok(match sign {
Some('+') => (0, n),
Some(_) => (n, 0),
None => (n, n),
})
}
/// `line`, or when longer than [`MAX_LINE_TEXT`] UTF-16 units, the part from [`SNIPPET_AROUND`]
/// units before 0-based UTF-16 column `start` to as many after `end`, with `…` where it was cut.
fn snippet(line: &str, start: usize, end: usize) -> String {
let units: usize = line.chars().map(char::len_utf16).sum();
if units <= MAX_LINE_TEXT {
return line.to_owned();
}
let from = start.saturating_sub(SNIPPET_AROUND);
let to = end.max(start).saturating_add(SNIPPET_AROUND).min(units);
let (mut out, mut at) = (String::new(), 0);
for c in line.chars() {
if at >= from && at < to {
out.push(c);
}
at += c.len_utf16();
}
let before = if from > 0 { "\u{2026}" } else { "" };
let after = if to < units { "\u{2026}" } else { "" };
format!("{before}{out}{after}")
}
/// A context line: its first [`MAX_LINE_TEXT`] UTF-16 units.
fn context_line(line: &str) -> String {
snippet(line, 0, MAX_LINE_TEXT.saturating_sub(SNIPPET_AROUND))
}
/// Fills `line_text` and the context of `items`, whose files are `absolute(item)` (none: left
/// as they are); each file is read once, with unsaved editor text.
async fn fill_lines(
call: &Call,
items: &mut [SearchItem],
absolute: impl Fn(&SearchItem) -> Option<PathBuf>,
(before, after): (u32, u32),
) {
let host = call.host();
let mut current: Option<(PathBuf, Option<Vec<String>>)> = None;
for item in items {
let (Some(path), Some(line)) = (absolute(item), item.start_line) else { continue };
if current.as_ref().is_none_or(|(p, _)| p != &path) {
let text = crate::ops::fs::read_text(&host, call.window(), path.clone()).await.ok();
let lines = text.map(|t| t.split('\n').map(|l| l.strip_suffix('\r').unwrap_or(l).to_owned()).collect());
current = Some((path, lines));
}
let Some((_, Some(lines))) = &current else { continue };
let index = line as usize - 1;
let Some(text) = lines.get(index) else { continue };
let start = item.start_column.unwrap_or(1) as usize - 1;
let end = if item.end_line == Some(line) { item.end_column.map_or(start, |c| c as usize - 1) } else { start };
item.line_text = Some(snippet(text, start, end));
let first = index.saturating_sub(before as usize);
item.context_before = lines[first..index].iter().map(|l| context_line(l)).collect();
let last = (index + after as usize).min(lines.len() - 1);
item.context_after = lines[index + 1..=last].iter().map(|l| context_line(l)).collect();
}
}
// --- path globs ----------------------------------------------------------------
/// The `paths` argument: project-relative globs, `!` excludes.
@@ -416,6 +522,7 @@ struct TextHit {
async fn text_search(call: Call, is_regex: bool) -> IdeResult<Output> {
let pattern = query(&call, if is_regex { "Regex pattern is empty" } else { "Search text is empty" })?;
let limit = limit(&call, DEFAULT_LIMIT)?;
let around = context(&call)?;
let scope = PathScope::new(call.root(), &call.str_list("paths")?)?;
// Globs Find in Files cannot take are applied afterwards, so the engine must not
// stop at `limit` matches the filter may drop.
@@ -494,7 +601,7 @@ async fn text_search(call: Call, is_regex: bool) -> IdeResult<Output> {
let more = hits.len() > limit || summary.limit_hit || timed_out;
hits.truncate(limit);
extend_spanning(&call, &query, &mut hits, spanning).await;
let items = hits
let mut items: Vec<SearchItem> = hits
.into_iter()
.map(|h| SearchItem {
file_path: h.file_path,
@@ -505,6 +612,9 @@ async fn text_search(call: Call, is_regex: bool) -> IdeResult<Output> {
..Default::default()
})
.collect();
let absolute: std::collections::HashMap<&str, &Path> =
files.iter().map(|(path, file)| (path.as_str(), file.path.0.as_path())).collect();
fill_lines(&call, &mut items, |item| absolute.get(item.file_path.as_str()).map(|p| p.to_path_buf()), around).await;
let partial_result_reason = timed_out.then_some(SEARCH_TIMEOUT_PARTIAL_RESULT_REASON);
Output::json(SearchResult { items, more, partial_result_reason })
}
@@ -726,6 +836,7 @@ async fn symbol_search(call: Call) -> IdeResult<Output> {
let started = Instant::now();
let q = query(&call, "Search query is empty")?;
let limit = limit(&call, SYMBOL_SEARCH_DEFAULT_LIMIT)?;
let around = context(&call)?;
let scope = PathScope::new(call.root(), &call.str_list("paths")?)?;
let external = call.flag("include_external", false)?;
let host = call.host();
@@ -775,6 +886,13 @@ async fn symbol_search(call: Call) -> IdeResult<Output> {
}
let more = items.len() > limit || timed_out;
items.truncate(limit);
// Project files (relative paths, or absolute ones of content roots); library URIs are left out.
let root = call.root().to_path_buf();
let absolute = |item: &SearchItem| {
let path = Path::new(&item.file_path);
(!item.file_path.contains("://")).then(|| if path.is_absolute() { path.to_path_buf() } else { root.join(path) })
};
fill_lines(&call, &mut items, absolute, around).await;
let indexing = indexing_before || project.index.status().indexing;
let partial_result_reason = if indexing {
Some(INDEXING_PARTIAL_RESULT_REASON)
@@ -869,6 +987,17 @@ mod tests {
assert_eq!(byte_at("ab", 5), None);
}
#[test]
fn long_lines_are_cut_around_the_match() {
assert_eq!(snippet("short line", 0, 5), "short line");
let line = format!("{}needle{}", "a".repeat(400), "b".repeat(400));
let cut = snippet(&line, 400, 406);
assert_eq!(cut, format!("\u{2026}{}needle{}\u{2026}", "a".repeat(100), "b".repeat(100)));
let start = snippet(&line, 0, 0);
assert!(start.starts_with('a') && start.ends_with('\u{2026}') && start.chars().count() == 101);
assert_eq!(context_line(&"x".repeat(1000)).chars().count(), MAX_LINE_TEXT + 1);
}
#[test]
fn path_scopes() {
let (_dir, root) = root();
+109 -17
Переглянути файл
@@ -5,9 +5,9 @@ use crate::Host;
use crate::rpc::Dispatcher;
use ide_agent::registry::{AgentRegistry, RegistryPaths};
use ide_agent::{
AcpImportOutcome, AgentAcceptOutcome, AgentChange, AgentCommand, AgentEntry, AgentEvent, AgentProbe, AgentPrompt,
AgentProvider, AgentQueued, AgentTask, CustomAgentInput, ExternalSession, McpScope, McpServer, McpServerInput,
NewAgentTask,
AcpImportOutcome, AgentAcceptOutcome, AgentChange, AgentChanges, AgentChangesView, AgentCommand, AgentEntry,
AgentEvent, AgentProbe, AgentPrompt, AgentProvider, AgentQueued, AgentTask, CustomAgentInput, ExternalSession,
McpScope, McpServer, McpServerInput, NewAgentTask,
};
use ide_core::error::ErrorKind;
use ide_core::{IdeError, IdeEvent, IdeResult, IpcPath, RequestId, SharedSink, TaskId};
@@ -304,16 +304,28 @@ pub async fn changes(host: &Host, window: &str, task: String) -> IdeResult<Vec<A
host.project(window)?.agents.changes(&task).await
}
/// The task's uncommitted changes and the commits made since it started.
pub async fn grouped_changes(host: &Host, window: &str, task: String) -> IdeResult<AgentChanges> {
host.require_trust(window)?;
host.project(window)?.agents.grouped_changes(&task).await
}
/// Content of `path` when the task started (left side of its diff).
pub async fn base_text(host: &Host, window: &str, task: String, path: IpcPath) -> IdeResult<Option<String>> {
host.require_trust(window)?;
host.project(window)?.agents.base_text(&task, path.into_path_buf()).await
}
/// Rollback: the task's changed files among `paths` (all without) back as they were when it started.
pub async fn rollback_changes(host: &Host, window: &str, task: String, paths: Option<Vec<String>>) -> IdeResult<()> {
/// Rollback: the task's changed files among `paths` (all without) back as `view` compares them.
pub async fn rollback_changes(
host: &Host,
window: &str,
task: String,
paths: Option<Vec<String>>,
view: Option<AgentChangesView>,
) -> IdeResult<()> {
host.require_trust(window)?;
host.project(window)?.agents.rollback_changes(&task, paths).await
host.project(window)?.agents.rollback_changes(&task, paths, view.unwrap_or_default()).await
}
/// Accept: copies a worktree task's changed files among `paths` (all without) into the
@@ -369,7 +381,7 @@ fn persist_mcp_map(
Ok(())
}
fn update_mcp_maps<F>(host: &Host, window: &str, f: F) -> IdeResult<Vec<McpServer>>
fn update_mcp_maps<F>(host: &Host, window: &str, approve: Option<&str>, f: F) -> IdeResult<Vec<McpServer>>
where
F: FnOnce(&mut McpServerMap, &mut McpServerMap, Option<&std::path::Path>) -> Result<(), String>,
{
@@ -378,43 +390,83 @@ where
let prev_workspace = workspace.clone();
let project_root = project.as_ref().map(|p| p.root.as_path());
f(&mut user, &mut workspace, project_root).map_err(IdeError::invalid)?;
if let (Some(name), Some(p)) = (approve, &project)
&& let Some(server) = workspace.get(name)
{
host.mcp_approvals.approve(&p.root, name, server)?;
}
if workspace != prev_workspace {
persist_mcp_map(host, project.as_ref(), McpScope::Workspace, &workspace)?;
}
if user != prev_user {
persist_mcp_map(host, project.as_ref(), McpScope::User, &user)?;
}
Ok(ide_agent::mcp::servers_from_settings(&user, &workspace))
Ok(listed(host, project.as_ref(), &user, workspace))
}
/// Workspace servers the user hasn't turned on in Settings are listed as off: agents don't start them.
fn listed(
host: &Host,
project: Option<&std::sync::Arc<crate::Project>>,
user: &McpServerMap,
workspace: McpServerMap,
) -> Vec<McpServer> {
let workspace = match project {
Some(p) => host.mcp_approvals.apply(&p.root, workspace),
None => workspace,
};
ide_agent::mcp::servers_from_settings(user, &workspace)
}
/// The MCP servers agent sessions of the project at `root` get: the user's, and the project's the
/// user turned on.
pub(crate) fn session_mcp_servers(
settings: &ide_workspace::SettingsStore,
approvals: &crate::trust::McpServerApprovals,
root: &std::path::Path,
) -> Vec<McpServer> {
use ide_workspace::settings::SettingsScope;
let user = settings.layer(SettingsScope::Global).agents.mcp_servers;
let workspace = approvals.apply(root, settings.layer(SettingsScope::Project).agents.mcp_servers);
ide_agent::mcp::servers_from_settings(&user, &workspace)
}
/// Lists configured MCP servers across User (`settings.toml`) and Workspace (`.openidea/settings.toml`) scopes.
pub fn mcp_list(host: &Host, window: &str) -> IdeResult<Vec<McpServer>> {
let (user, workspace, _) = mcp_maps(host, window);
Ok(ide_agent::mcp::servers_from_settings(&user, &workspace))
let (user, workspace, project) = mcp_maps(host, window);
Ok(listed(host, project.as_ref(), &user, workspace))
}
/// Creates or updates a User or Workspace MCP server and returns the updated list.
/// Creates or updates a User or Workspace MCP server and returns the updated list. A Workspace
/// server saved here is one the user turned on.
pub fn mcp_save(host: &Host, window: &str, input: McpServerInput) -> IdeResult<Vec<McpServer>> {
update_mcp_maps(host, window, |user, workspace, project_root| {
let approve = (input.scope == McpScope::Workspace).then(|| input.name.trim().to_owned());
update_mcp_maps(host, window, approve.as_deref(), |user, workspace, project_root| {
ide_agent::mcp::apply_save(user, workspace, project_root.is_some(), input)
})
}
/// Enables or disables an MCP server and returns the updated list.
pub fn mcp_set_enabled(host: &Host, window: &str, id: String, enabled: bool) -> IdeResult<Vec<McpServer>> {
update_mcp_maps(host, window, |user, workspace, _| {
let approve = ide_agent::mcp::parse_server_id(&id)
.filter(|(scope, _)| enabled && *scope == McpScope::Workspace)
.map(|(_, name)| name.to_owned());
update_mcp_maps(host, window, approve.as_deref(), |user, workspace, _| {
ide_agent::mcp::apply_set_enabled(user, workspace, &id, enabled).map(|_| ())
})
}
/// Removes a User or Workspace MCP server and returns the updated list.
pub fn mcp_remove(host: &Host, window: &str, id: String) -> IdeResult<Vec<McpServer>> {
update_mcp_maps(host, window, |user, workspace, _| ide_agent::mcp::apply_remove(user, workspace, &id).map(|_| ()))
update_mcp_maps(host, window, None, |user, workspace, _| {
ide_agent::mcp::apply_remove(user, workspace, &id).map(|_| ())
})
}
/// Copies MCP servers from `.mcp.json` and `.vscode/mcp.json` into the workspace settings once.
/// Copies MCP servers from `.mcp.json` and `.vscode/mcp.json` into the workspace settings once;
/// they stay off until the user turns them on.
pub fn mcp_import(host: &Host, window: &str) -> IdeResult<Vec<McpServer>> {
update_mcp_maps(host, window, |_, workspace, project_root| {
update_mcp_maps(host, window, None, |_, workspace, project_root| {
let root = project_root.ok_or_else(|| "Open a project to import workspace MCP servers".to_string())?;
ide_agent::mcp::apply_import(workspace, root);
Ok(())
@@ -476,9 +528,10 @@ pub(crate) fn register(d: &mut Dispatcher) {
|c| async move { delete(&c.host, &c.window, c.arg("task")?, c.arg("removeWorktree")?).await },
);
d.add("agent_changes", |c| async move { changes(&c.host, &c.window, c.arg("task")?).await });
d.add("agent_grouped_changes", |c| async move { grouped_changes(&c.host, &c.window, c.arg("task")?).await });
d.add("agent_base_text", |c| async move { base_text(&c.host, &c.window, c.arg("task")?, c.arg("path")?).await });
d.add("agent_rollback_changes", |c| async move {
rollback_changes(&c.host, &c.window, c.arg("task")?, c.arg("paths")?).await
rollback_changes(&c.host, &c.window, c.arg("task")?, c.arg("paths")?, c.arg("view")?).await
});
d.add("agent_accept_changes", |c| async move {
accept_changes(&c.host, &c.window, c.arg("task")?, c.arg("paths")?).await
@@ -537,4 +590,43 @@ mod tests {
assert!(validate_project_root(&root, Some(&rejected)).is_err(), "{rejected:?}");
}
}
#[test]
fn sessions_get_the_users_servers_and_the_projects_turned_on_ones() {
let tmp = tempfile::tempdir().unwrap();
let (config, root) = (tmp.path().join("config"), tmp.path().join("project"));
std::fs::create_dir_all(&config).unwrap();
std::fs::create_dir_all(root.join(".openidea")).unwrap();
std::fs::write(config.join("settings.toml"), "[agents.mcpServers.shared]\ncommand = \"user-cmd\"\n").unwrap();
std::fs::write(
root.join(".openidea/settings.toml"),
"[agents.mcpServers.shared]\ncommand = \"project-cmd\"\n\n[agents.mcpServers.extra]\ncommand = \"extra\"\n",
)
.unwrap();
let settings = ide_workspace::SettingsStore::load(&config, Some(&root));
let approvals = crate::trust::McpServerApprovals::load(&tmp.path().join("data"));
let on = |servers: Vec<McpServer>| -> Vec<(McpScope, String, String)> {
servers
.into_iter()
.filter(|s| s.enabled)
.map(|s| (s.scope, s.name, s.command.unwrap_or_default()))
.collect()
};
assert_eq!(
on(session_mcp_servers(&settings, &approvals, &root)),
[(McpScope::User, "shared".to_owned(), "user-cmd".to_owned())]
);
let extra = settings.layer(ide_workspace::settings::SettingsScope::Project).agents.mcp_servers["extra"].clone();
approvals.approve(&root, "extra", &extra).unwrap();
assert!(on(session_mcp_servers(&settings, &approvals, &root)).contains(&(
McpScope::Workspace,
"extra".to_owned(),
"extra".to_owned()
)));
// Remembered across restarts, and only for this project.
let reloaded = crate::trust::McpServerApprovals::load(&tmp.path().join("data"));
assert!(reloaded.approved(&root, "extra", &extra));
assert!(!reloaded.approved(&tmp.path().join("other"), "extra", &extra));
}
}
+255 -4
Переглянути файл
@@ -2,16 +2,18 @@
//! Build › Build Project (compiling with the project's build system).
mod workspace;
use super::dependencies::{MavenCentralArtifacts, MavenCentralGroups, MavenCentralVersions};
use crate::ops::Channel;
use crate::rpc::Dispatcher;
use crate::{Host, Project, apply_workspace_edit};
use ide_build::compile::{self, BuildEvent, BuildKind, BuildResult};
use ide_build::maven::PomGoto;
use ide_build::maven::completion::{PomCompletion, PomCompletionOption};
use ide_build::pom_edit::{ArtifactKey, Exclusion};
use ide_build::{BuildModel, BuildTool, SyncOptions, SyncProgress};
use ide_core::settings::Settings;
use ide_core::{FileEdit, IdeError, IdeEvent, IdeResult, Location, Pos, Range, RequestId, WorkspaceEdit};
use ide_core::{DocId, FileEdit, IdeError, IdeEvent, IdeResult, Location, Pos, Range, RequestId, WorkspaceEdit};
use std::path::Path;
fn join_err(e: tokio::task::JoinError) -> IdeError {
@@ -311,10 +313,11 @@ pub async fn build_sync(
if tool == BuildTool::Gradle {
// The sync may report project folders outside the project (`includeFlat`): index and list them.
project.refresh_index_scope();
project.sink.emit(ide_core::IdeEvent::ProjectRootsChanged { root: project.root.clone().into() });
let p = project.clone();
tokio::task::spawn_blocking(move || p.refresh_repositories());
}
// New source folders and dependency versions: the Project view and External Libraries reload.
project.sink.emit(ide_core::IdeEvent::ProjectRootsChanged { root: project.root.clone().into() });
if tool == BuildTool::Maven
&& model.maven.is_some()
&& let Some(bridge) = &project.kotlin_bridge
@@ -453,7 +456,8 @@ pub async fn maven_dependency_declaration(
/// absolute for a parent POM opened from the local repository): the definition of the
/// `${property}` at the caret, or the `<dependencyManagement>` / `<pluginManagement>`
/// entry providing the dependency's / plugin's version, along the parent chain (into
/// the local repository when a parent is not in the project). `None` when nothing
/// the local repository when a parent is not in the project), else the artifact's own
/// POM (a project module's, or the local repository's). `None` when nothing
/// resolvable is under the caret.
pub async fn maven_goto_declaration(host: &Host, window: &str, pom: String, pos: Pos) -> IdeResult<Option<PomGoto>> {
let project = host.project(window)?;
@@ -464,12 +468,107 @@ pub async fn maven_goto_declaration(host: &Host, window: &str, pom: String, pos:
let settings = project.settings.effective();
let repo = ide_sdk::apply::maven_local_repository(&project.root, &settings)
.or_else(ide_build::maven::default_local_repo);
Ok(ide_build::maven::goto_declaration(&text, &path, pos, repo))
let modules = || {
let model = load(&project.root, &project.build_cache_dir(), &settings).maven.unwrap_or_default();
model
.projects
.into_iter()
.map(|p| ide_build::maven::ModulePom {
group_id: p.group_id,
artifact_id: p.artifact_id,
pom: project.root.join(&p.pom),
})
.collect()
};
Ok(ide_build::maven::goto_declaration_in(&text, &path, pos, repo, modules))
})
.await
.map_err(join_err)?
}
/// How long completion waits for Maven Central before answering from the local repository.
const CENTRAL_WAIT: std::time::Duration = std::time::Duration::from_secs(1);
/// Completion of the `<dependency>` coordinate the caret (`offset`, UTF-16) sits
/// in the value of in the POM `doc`: the coordinates installed in the local
/// repository (Settings › Maven's, else `~/.m2/repository`) — group ids,
/// the chosen group's artifacts, the chosen artifact's versions — topped up
/// with Maven Central's (the search endpoint's popular coordinates, the
/// published version metadata), which the shared version cache keeps for the
/// session. `None` when the caret is not in a `<dependency>` coordinate element.
pub async fn maven_dependency_completion(
host: &Host,
window: &str,
doc: DocId,
offset: u32,
request: RequestId,
) -> IdeResult<Option<PomCompletion>> {
use ide_build::maven::completion::{PomCompletionKind, local_options};
let project = host.project(window)?;
let cache = host.dependency_versions.clone();
let cancel = host.begin_project_read_request(window, request);
let result = host
.blocking(move || {
let snap = project.documents.snapshot(doc)?;
if snap.large || snap.read_only || !snap.path.file_name().is_some_and(|name| name == "pom.xml") {
return Ok(None);
}
let Some(target) = ide_build::maven::completion::completion_target(&snap.text, offset) else {
return Ok(None);
};
let settings = project.settings.effective();
let repo = ide_sdk::apply::maven_local_repository(&project.root, &settings)
.or_else(ide_build::maven::default_local_repo);
let mut options = repo.map_or_else(Vec::new, |repo| local_options(&repo, &target));
// Central's answers are cached for the session (the shared version cache).
// The local repository answers at once: Central gets `CENTRAL_WAIT`, then the
// lookup finishes in the background and fills the cache for the next request.
// Offline, behind a proxy or rate-limited, a failed fetch never fails completion.
let remote = {
let (cache, target) = (cache.clone(), target.clone());
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let detached = ide_core::CancellationToken::new();
let found = match target.kind {
PomCompletionKind::GroupId if !target.typed.is_empty() => {
cache.versions(&MavenCentralGroups, &target.typed, &detached).unwrap_or_default()
}
PomCompletionKind::ArtifactId => match &target.group {
Some(group) => cache
.versions(&MavenCentralArtifacts { group: group.clone() }, &target.typed, &detached)
.unwrap_or_default(),
None => Vec::new(),
},
PomCompletionKind::Version => match (&target.group, &target.artifact) {
(Some(group), Some(artifact)) => cache
.versions(&MavenCentralVersions, &format!("{group}:{artifact}"), &detached)
.unwrap_or_default(),
_ => Vec::new(),
},
_ => Vec::new(), // A group id from nothing: the search endpoint has no such query.
};
let _ = tx.send(found);
});
rx.recv_timeout(CENTRAL_WAIT).unwrap_or_default()
};
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
let known: std::collections::HashSet<String> = options.iter().map(|o| o.label.clone()).collect();
options.extend(
remote
.into_iter()
.filter(|label| !known.contains(label))
.map(|label| PomCompletionOption { label, detail: None }),
);
options.truncate(ide_build::maven::completion::MAX_OPTIONS);
Ok(Some(PomCompletion { from: target.from, to: target.to, kind: target.kind, options }))
})
.await;
host.end_request(request);
result
}
/// Adds an `<exclusion>` of `exclusion` to the declaration of the direct dependency
/// `dependency` of the Maven module `pom`, through the open document when there is one
/// (saved). Returns the declaration; nothing changes when it already excludes it.
@@ -711,6 +810,9 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("maven_goto_declaration", |c| async move {
maven_goto_declaration(&c.host, &c.window, c.arg("pom")?, c.arg("pos")?).await
});
d.add("maven_dependency_completion", |c| async move {
maven_dependency_completion(&c.host, &c.window, c.arg("doc")?, c.arg("offset")?, c.arg("request")?).await
});
d.add("maven_exclude_dependency", |c| async move {
maven_exclude_dependency(&c.host, &c.window, c.arg("pom")?, c.arg("dependency")?, c.arg("exclusion")?).await
});
@@ -883,6 +985,38 @@ mod build_tool_settings_tests {
}
}
/// Changing Project JDK updates the next Maven build without reopening the project.
#[test]
fn changing_project_jdk_updates_the_next_maven_build() {
let tmp = tempfile::tempdir().unwrap();
let root = dunce::canonicalize(tmp.path()).unwrap();
std::fs::write(
root.join("pom.xml"),
r#"<project><modelVersion>4.0.0</modelVersion><groupId>test</groupId><artifactId>app</artifactId><version>1</version></project>"#,
)
.unwrap();
for (name, version) in [("jdk8", "1.8"), ("jdk21", "21")] {
let home = root.join(name);
std::fs::create_dir_all(home.join("bin")).unwrap();
std::fs::write(home.join("bin").join(if cfg!(windows) { "java.exe" } else { "java" }), "").unwrap();
std::fs::write(home.join("release"), format!("JAVA_VERSION=\"{version}\"\n")).unwrap();
}
let mut settings = Settings::default();
settings.languages.java.maven.runner_jre.clear();
for name in ["jdk8", "jdk21"] {
settings.languages.java.jdk = name.into();
let expected = root.join(name);
let model = load(&root, &root.join("cache"), &settings);
assert_eq!(model.maven.as_ref().unwrap().projects[0].env["JAVA_HOME"], expected.to_string_lossy());
let plan = compile::plan(&root, &model, BuildKind::Build, None, None).unwrap();
assert!(!plan.steps.is_empty());
for step in plan.steps {
assert_eq!(step.env["JAVA_HOME"], expected.to_string_lossy());
}
}
}
#[test]
fn model_for_sync_tasks_and_compile_uses_gradle_settings() {
let tmp = tempfile::tempdir().unwrap();
@@ -909,3 +1043,120 @@ mod build_tool_settings_tests {
assert_eq!(model.env["GRADLE_USER_HOME"], root.join("user-home").to_string_lossy());
}
}
#[cfg(test)]
mod maven_completion_tests {
use super::*;
use crate::ops::dependencies::hashed_key;
use ide_build::maven::completion::PomCompletionKind;
use ide_core::CancellationToken;
use ide_workspace::dependencies::DependencyVersionSource;
use std::sync::Arc;
/// Cache answers the test plants before completing, so the op's Central
/// lookups hit the session cache and the network is never reached.
struct FakeSource(&'static str, Vec<&'static str>);
impl DependencyVersionSource for FakeSource {
fn id(&self) -> &'static str {
self.0
}
fn cache_key(&self, package: &str) -> Option<String> {
Some(hashed_key(self.0, package))
}
fn fetch(&self, _: &str, _: &CancellationToken) -> IdeResult<Vec<String>> {
Ok(self.1.iter().map(|s| (*s).to_owned()).collect())
}
}
const GROUPS: &str = "maven-central-groups";
const ARTIFACTS: &str = "maven-central-artifacts";
const VERSIONS: &str = "maven-central-versions";
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn completes_dependency_coordinates_from_the_local_repository_then_central() {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let (root, repo) = (base.join("project"), base.join("repository"));
std::fs::create_dir_all(&root).unwrap();
// Settings › Maven's local repository points at the fixture, so the
// test machine's own ~/.m2 is never read.
std::fs::create_dir_all(base.join("config")).unwrap();
std::fs::write(
base.join("config/settings.toml"),
format!("[languages.java.maven]\nlocalRepository = \"{}\"\n", repo.to_string_lossy().replace('\\', "/")),
)
.unwrap();
for file in [
"org/apache/maven/maven-core/4.0.0-rc-5/maven-core-4.0.0-rc-5.jar",
"org/example/lib/2.0/lib-2.0.jar",
"org/lib/1.0/lib-1.0.jar",
] {
let path = repo.join(file);
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
std::fs::write(&path, b"").unwrap();
}
let text = "<project>\n <dependencies>\n <dependency>\n <groupId>org</groupId>\n <artifactId>lib</artifactId>\n <version>1.0</version>\n </dependency>\n <dependency>\n <groupId>org.example</groupId>\n <artifactId></artifactId>\n </dependency>\n <dependency>\n <groupId>org.example</groupId>\n <artifactId>lib</artifactId>\n <version></version>\n </dependency>\n </dependencies>\n</project>\n";
let path = root.join("pom.xml");
std::fs::write(&path, text).unwrap();
let host = Host::new(crate::HostConfig {
config_dir: base.join("config"),
data_dir: base.join("data"),
local_data_dir: base.join("local"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(|_| {})),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
let project = host.open_project("main", &root).unwrap();
let info = project.documents.open(&path).unwrap();
let token = CancellationToken::new();
host.dependency_versions.versions(&FakeSource(GROUPS, vec!["org.zzz"]), "org", &token).unwrap();
host.dependency_versions.versions(&FakeSource(ARTIFACTS, vec!["api"]), "org.example\u{1}", &token).unwrap();
host.dependency_versions.versions(&FakeSource(VERSIONS, vec!["9.9"]), "org.example:lib", &token).unwrap();
let at = |needle: &str| (text.find(needle).unwrap() + needle.len()) as u32;
let next = std::sync::atomic::AtomicU32::new(500);
let complete = |offset: u32| {
maven_dependency_completion(
&host,
"main",
info.id,
offset,
RequestId(next.fetch_add(1, std::sync::atomic::Ordering::Relaxed)),
)
};
fn labels(c: &PomCompletion) -> Vec<&str> {
c.options.iter().map(|o| o.label.as_str()).collect()
}
let group =
complete(text.find("<groupId>org").unwrap() as u32 + "<groupId>org".len() as u32).await.unwrap().unwrap();
assert_eq!(group.kind, PomCompletionKind::GroupId);
// The installed groups first, Central's cached suggestions behind them.
assert_eq!(labels(&group), ["org", "org.apache.maven", "org.example", "org.zzz"]);
// Nothing outside a dependency's coordinates.
let outside = complete(at("<dependencies>")).await;
assert!(matches!(outside, Ok(None)), "{outside:?}");
let empty_artifact = text.find("<artifactId></artifactId>").unwrap() as u32 + "<artifactId>".len() as u32;
let artifact = complete(empty_artifact).await;
let artifact = match artifact {
Ok(Some(a)) => a,
other => panic!("artifact completion: {other:?}"),
};
assert_eq!(artifact.kind, PomCompletionKind::ArtifactId);
let installed = artifact.options.first().unwrap();
assert_eq!((installed.label.as_str(), installed.detail.as_deref()), ("lib", Some("2.0")));
assert_eq!(labels(&artifact), ["lib", "api"]);
let empty_version = text.find("<version></version>").unwrap() as u32 + "<version>".len() as u32;
let version = complete(empty_version).await.unwrap().unwrap();
assert_eq!(version.kind, PomCompletionKind::Version);
assert_eq!(labels(&version), ["2.0", "9.9"]);
host.close_project("main");
}
}
+45
Переглянути файл
@@ -71,6 +71,22 @@ pub(crate) fn register(d: &mut Dispatcher) {
let id = c.arg::<String>("id")?;
databases(&c.host, &c.window, &id).await
});
d.add("db_database_catalog", |c| async move {
let (id, database) = (c.arg::<String>("id")?, c.arg::<String>("database")?);
database_catalog(&c.host, &c.window, &id, &database).await
});
d.add("db_saved_catalog", |c| async move {
let id = c.arg::<String>("id")?;
saved_catalog(&c.host, &c.window, &id).await
});
d.add("db_retain_catalog", |c| async move {
let (id, databases) = (c.arg::<String>("id")?, c.arg::<Vec<String>>("databases")?);
retain_catalog(&c.host, &c.window, &id, databases).await
});
d.add("db_session_context", |c| async move {
let id = c.arg::<String>("id")?;
session_context(&c.host, &c.window, &id).await
});
d.add("db_schemas", |c| async move {
let id = c.arg::<String>("id")?;
schemas(&c.host, &c.window, &id).await
@@ -246,6 +262,35 @@ pub async fn databases(host: &Host, window: &str, id: &str) -> IdeResult<Databas
db.databases(id).await
}
/// The schemas of `database` of Snowflake data source `id` with their tables and views.
pub async fn database_catalog(
host: &Host,
window: &str,
id: &str,
database: &str,
) -> IdeResult<Vec<ide_db::CatalogSchema>> {
let db = host.project(window)?.db.clone();
db.database_catalog(id, database).await
}
/// What earlier introspection of data source `id` saved; the data source is not connected to.
pub async fn saved_catalog(host: &Host, window: &str, id: &str) -> IdeResult<Option<ide_db::SourceCatalog>> {
let db = host.project(window)?.db.clone();
db.saved_catalog(id).await
}
/// Drops the saved databases of data source `id` that are not in `databases` any more.
pub async fn retain_catalog(host: &Host, window: &str, id: &str, databases: Vec<String>) -> IdeResult<()> {
let db = host.project(window)?.db.clone();
db.retain_catalog(id, databases).await
}
/// The warehouse and role of the Snowflake session of data source `id`, and what it may switch to.
pub async fn session_context(host: &Host, window: &str, id: &str) -> IdeResult<ide_db::SessionContext> {
let db = host.project(window)?.db.clone();
db.session_context(id).await
}
/// The schemas of data source `id`.
pub async fn schemas(host: &Host, window: &str, id: &str) -> IdeResult<Vec<String>> {
let db = host.project(window)?.db.clone();
+74 -1
Переглянути файл
@@ -1,7 +1,7 @@
//! Repository version lookups for manifest inspections and completion.
use crate::{Host, rpc::Dispatcher};
use ide_core::{CancellationToken, DocId, IdeError, IdeResult, RequestId};
use ide_workspace::dependencies::{DependencyVersionSource, DependencyVersions, cargo, go, npm};
use ide_workspace::dependencies::{DependencyVersionSource, DependencyVersions, cargo, go, maven, npm};
use std::collections::BTreeMap;
use std::hash::{DefaultHasher, Hash, Hasher};
use std::path::{Path, PathBuf};
@@ -25,6 +25,79 @@ impl DependencyVersionSource for CratesIo {
}
}
/// A cache key for a `package` that is not a filesystem-safe name (a Maven
/// coordinate with dots, a query): hashed under the source's id.
pub(crate) fn hashed_key(id: &str, package: &str) -> String {
let mut hash = DefaultHasher::new();
package.hash(&mut hash);
format!("{id}-{:016x}", hash.finish())
}
/// Maven Central's group-id suggestions: the search endpoint's groups matching
/// a typed prefix, most recently updated first.
pub(crate) struct MavenCentralGroups;
impl DependencyVersionSource for MavenCentralGroups {
fn id(&self) -> &'static str {
"maven-central-groups"
}
fn cache_key(&self, prefix: &str) -> Option<String> {
Some(hashed_key(self.id(), prefix))
}
fn fetch(&self, prefix: &str, cancel: &CancellationToken) -> IdeResult<Vec<String>> {
let url = maven::groups_url(prefix).ok_or_else(|| IdeError::internal("invalid group prefix"))?;
let search_path = std::env::var_os("PATH").unwrap_or_default();
let text =
ide_tools::Http::new(&search_path)?.https_only().get_text_limited(&url, Some(1024 * 1024), cancel)?;
maven::parse_search_groups(&text).ok_or_else(|| IdeError::internal("invalid search response"))
}
}
/// The artifacts of one group matching a typed prefix, from the same endpoint.
pub(crate) struct MavenCentralArtifacts {
pub group: String,
}
impl DependencyVersionSource for MavenCentralArtifacts {
fn id(&self) -> &'static str {
"maven-central-artifacts"
}
fn cache_key(&self, prefix: &str) -> Option<String> {
Some(hashed_key(self.id(), &format!("{}\u{1}{prefix}", self.group)))
}
fn fetch(&self, prefix: &str, cancel: &CancellationToken) -> IdeResult<Vec<String>> {
let url =
maven::artifacts_url(&self.group, prefix).ok_or_else(|| IdeError::internal("invalid artifact prefix"))?;
let search_path = std::env::var_os("PATH").unwrap_or_default();
let text =
ide_tools::Http::new(&search_path)?.https_only().get_text_limited(&url, Some(1024 * 1024), cancel)?;
maven::parse_search_artifacts(&text).ok_or_else(|| IdeError::internal("invalid search response"))
}
}
/// Every version published of one `group:artifact` on Central, newest first.
pub(crate) struct MavenCentralVersions;
impl DependencyVersionSource for MavenCentralVersions {
fn id(&self) -> &'static str {
"maven-central-versions"
}
fn cache_key(&self, coordinates: &str) -> Option<String> {
Some(hashed_key(self.id(), coordinates))
}
fn fetch(&self, coordinates: &str, cancel: &CancellationToken) -> IdeResult<Vec<String>> {
let (group, artifact) = coordinates.split_once(':').ok_or_else(|| IdeError::internal("invalid coordinates"))?;
let url = maven::metadata_url(group, artifact).ok_or_else(|| IdeError::internal("invalid coordinates"))?;
let search_path = std::env::var_os("PATH").unwrap_or_default();
let text =
ide_tools::Http::new(&search_path)?.https_only().get_text_limited(&url, Some(2 * 1024 * 1024), cancel)?;
Ok(maven::parse_metadata_versions(&text))
}
}
struct NpmRegistry;
impl DependencyVersionSource for NpmRegistry {
fn id(&self) -> &'static str {
+53 -10
Переглянути файл
@@ -114,14 +114,28 @@ pub fn create_file(host: &Host, window: &str, path: PathBuf) -> IdeResult<()> {
if empty && !bom.is_empty() {
std::fs::write(&path, bom)?;
}
// A checkout backend opens the file for add; Git's hook is a no-op.
if let Err(e) = p.with_backend_of(&path, |r| r.files_added(std::slice::from_ref(&path))) {
tracing::debug!("vcs files_added for {}: {e}", path.display());
}
Ok(())
}
Err(_) => ide_vfs::create_file(&path),
}
}
pub async fn create_file_bytes(host: &Host, path: PathBuf, data: Vec<u8>) -> IdeResult<()> {
host.blocking(move || ide_vfs::create_file_bytes(&path, &data)).await
pub async fn create_file_bytes(host: &Host, window: &str, path: PathBuf, data: Vec<u8>) -> IdeResult<()> {
let project = host.project(window).ok();
host.blocking(move || {
ide_vfs::create_file_bytes(&path, &data)?;
if let Some(p) = project
&& let Err(e) = p.with_backend_of(&path, |r| r.files_added(std::slice::from_ref(&path)))
{
tracing::debug!("vcs files_added for {}: {e}", path.display());
}
Ok(())
})
.await
}
/// Toggle Read-only Attribute: sets the writable bit (and the open document's
@@ -135,25 +149,54 @@ pub fn set_writable(host: &Host, window: &str, path: PathBuf, writable: bool) ->
/// Moves to the trash; Local History keeps the contents so deleted files can be restored.
pub async fn delete(host: &Host, window: &str, path: PathBuf) -> IdeResult<()> {
let history = host.project(window).ok().map(|p| p.history.clone());
let project = host.project(window).ok();
let history = project.as_ref().map(|p| p.history.clone());
host.blocking(move || {
let paths = ide_workspace::file_ops::deletion_paths(&path)?;
if let Some(h) = history {
for p in paths {
h.record_deletion(&p, DELETE_SNAPSHOT_LIMIT);
for p in &paths {
h.record_deletion(p, DELETE_SNAPSHOT_LIMIT);
}
}
ide_workspace::file_ops::delete(&path)
ide_workspace::file_ops::delete(&path)?;
// A checkout backend opens the paths for delete; Git's hook is a no-op.
if let Some(p) = project {
let _ = p.with_backend_of(&path, |r| r.files_deleted(&paths));
}
Ok(())
})
.await
}
pub async fn copy(host: &Host, from: PathBuf, to: PathBuf) -> IdeResult<()> {
host.blocking(move || ide_workspace::file_ops::copy(&from, &to)).await
pub async fn copy(host: &Host, window: &str, from: PathBuf, to: PathBuf) -> IdeResult<()> {
let project = host.project(window).ok();
host.blocking(move || {
ide_workspace::file_ops::copy(&from, &to)?;
// A copy is an add at the destination; Git's hook is a no-op.
if let Some(p) = project
&& let Err(e) = p.with_backend_of(&to, |r| r.files_added(std::slice::from_ref(&to)))
{
tracing::debug!("vcs files_added for {}: {e}", to.display());
}
Ok(())
})
.await
}
pub async fn rename(host: &Host, from: PathBuf, to: PathBuf) -> IdeResult<()> {
host.blocking(move || ide_workspace::file_ops::rename(&from, &to)).await
pub async fn rename(host: &Host, window: &str, from: PathBuf, to: PathBuf) -> IdeResult<()> {
let project = host.project(window).ok();
host.blocking(move || {
ide_workspace::file_ops::rename(&from, &to)?;
// A checkout backend records the rename so history follows the file;
// Git's hook is a no-op.
if let Some(p) = project
&& let Err(e) = p.with_backend_of(&to, |r| r.file_moved(&from, &to))
{
tracing::debug!("vcs file_moved {} → {}: {e}", from.display(), to.display());
}
Ok(())
})
.await
}
/// Reads a text file without opening it as a document; prefers the unsaved buffer.
+166 -30
Переглянути файл
@@ -48,7 +48,7 @@ async fn resolve_launch(
let build = policy != BuildPolicy::NoBuild;
let full = policy == BuildPolicy::Rebuild;
let build_kind = if full { compile::BuildKind::Rebuild } else { compile::BuildKind::Build };
let (model, external, delegated, prefer_maven) = {
let (model, external, mut delegated, prefer_maven, is_maven) = {
let project = project.clone();
let settings = settings.clone();
let module = module.clone();
@@ -68,30 +68,47 @@ async fn resolve_launch(
prefer_maven,
);
let external = non_java || delegated;
(model, external, delegated, prefer_maven)
(model, external, delegated, prefer_maven, is_maven)
})
.await
.map_err(|e| IdeError::internal(e.to_string()))?
};
let external_build = if delegated { Some(project.lsp.java_external_build(&module, cancel).await?) } else { None };
let mut external_build =
if delegated { Some(project.lsp.java_external_build(&module, cancel).await?) } else { None };
let mut info = project.lsp.java_launch(&module, test, build && !delegated, full, delegated, cancel).await?;
if info.errors.iter().any(|e| processor_recreated_a_file(&e.message)) {
if build && !delegated && is_maven {
// JDT runs an annotation processor again over sources it generated in the same build
// (MapStruct over MapStruct-Plus's mappers) and fails on the file it already wrote;
// javac doesn't, so Maven builds the module.
tracing::info!("Annotation processing failed in JDT; building with Maven");
delegated = true;
external_build = Some(project.lsp.java_external_build(&module, cancel).await?);
info = project.lsp.java_launch(&module, test, false, false, true, cancel).await?;
}
if delegated {
// Maven runs the annotation processors itself; JDT's failure to rerun one is no compile error.
drop_processor_recreate_errors(&mut info);
}
}
let lsp_ms = started.elapsed().as_millis();
let data_dir = host.local_data_dir.clone();
let cancel = cancel.clone();
tokio::task::spawn_blocking(move || {
let _external_build = external_build;
let local_repo = ide_sdk::apply::maven_local_repository(&project.root, &settings);
let built = if delegated {
let policy = delegated_policy(policy, prefer_maven, info.error_count);
let rebuild_on_jdt_errors = rebuild_on_jdt_errors(policy, prefer_maven, info.error_count);
let outputs = project.java_launch_snapshots.lock().build(
&project.root,
&model,
&module,
cache::BuildOptions { test, policy },
ide_sdk::apply::maven_local_repository(&project.root, &settings),
cache::BuildOptions { test, policy, rebuild_on_jdt_errors },
local_repo.clone(),
&cancel,
)?;
let copies = use_snapshot(&mut info, &outputs.originals, &outputs.snapshot);
apply_external_build_result(&mut info, outputs.failure, policy == BuildPolicy::Rebuild);
apply_external_build_result(&mut info, outputs.failure, outputs.authoritative);
copies
} else if external {
let built = compile::build_for_launch(&project.root, &model, &module, test, build_kind, &cancel)?;
@@ -107,10 +124,22 @@ async fn resolve_launch(
Vec::new()
};
let mut class_paths: Vec<String> = built.iter().map(|p| p.to_string_lossy().into_owned()).collect();
for p in std::mem::take(&mut info.class_paths) {
if !class_paths.iter().any(|c| same_entry(c, &p, cfg!(windows))) {
class_paths.push(p);
}
if delegated {
// m2e runs inside jdtls on the server JDK and can resolve a different
// Maven profile/classifier than the module build. Ask the same Maven
// process that built the module for its runtime/test dependencies.
let maven_cp = compile::module_classpath(
&project.root,
&model,
&module,
test,
local_repo,
&project.build_cache_dir(),
&cancel,
);
merge_maven_launch_classpath(&mut class_paths, &mut info, maven_cp)?;
} else {
append_unique_class_paths(&mut class_paths, std::mem::take(&mut info.class_paths));
}
info.class_paths = class_paths;
apply_library_classpath(&mut info, &project.root, &settings.project_structure.libraries, test);
@@ -175,21 +204,18 @@ async fn resolve_launch(
/// With Settings › Maven › Delegate IDE build/run actions to Maven, javac's verdict replaces
/// JDT's. Classes JDT wrote for sources with errors look up to date to Maven's incremental
/// compiler, so a launch JDT reports errors for recompiles everything.
fn delegated_policy(policy: BuildPolicy, prefer_maven: bool, jdt_errors: u32) -> BuildPolicy {
if prefer_maven && jdt_errors > 0 && matches!(policy, BuildPolicy::Automatic | BuildPolicy::Make) {
BuildPolicy::Rebuild
} else {
policy
}
fn rebuild_on_jdt_errors(policy: BuildPolicy, prefer_maven: bool, jdt_errors: u32) -> bool {
prefer_maven && jdt_errors > 0 && matches!(policy, BuildPolicy::Automatic | BuildPolicy::Make)
}
fn apply_external_build_result(info: &mut JavaLaunchInfo, failure: Option<String>, clean: bool) {
fn apply_external_build_result(info: &mut JavaLaunchInfo, failure: Option<String>, authoritative: bool) {
info.build = Some(if let Some(failure) = failure {
info.hints.push(failure);
JavaBuildStatus::Failed
} else if clean {
// A clean external build recompiled every source; JDT's diagnostics still describe
// the previous output (and reject sources javac accepts, e.g. a package mismatch).
} else if authoritative {
// A clean rebuild or a fingerprint-verified snapshot is authoritative. JDT diagnostics
// may describe an older output or its own classpath, so they must not mark this launch
// as failed after Maven has produced usable classes.
info.errors.clear();
info.error_count = 0;
JavaBuildStatus::Succeeded
@@ -270,6 +296,44 @@ fn same_entry(a: &str, b: &str, windows: bool) -> bool {
a == b || entry_key(a, windows) == entry_key(b, windows)
}
/// Merge the Maven launch classpath without turning a recoverable probe failure into a
/// failed launch. Cancellation still propagates; ordinary failures keep jdtls' classpath
/// and leave the reason in the run console.
fn merge_maven_launch_classpath(
class_paths: &mut Vec<String>,
info: &mut JavaLaunchInfo,
classpath: IdeResult<String>,
) -> IdeResult<()> {
match classpath {
Ok(classpath) => {
append_unique_class_paths(
class_paths,
std::env::split_paths(&classpath).map(|path| path.to_string_lossy().into_owned()),
);
Ok(())
}
Err(error) if error.kind == ErrorKind::Cancelled => Err(error),
Err(error) => {
info.hints.push(format!(
"Maven launch classpath resolution failed; using the Java language server classpath instead:\n{}",
error.message
));
append_unique_class_paths(class_paths, std::mem::take(&mut info.class_paths));
Ok(())
}
}
}
/// Append entries without changing their precedence; `same_entry` also catches
/// Windows case-insensitive duplicates that would otherwise lengthen the command.
fn append_unique_class_paths(class_paths: &mut Vec<String>, entries: impl IntoIterator<Item = String>) {
for entry in entries {
if !class_paths.iter().any(|existing| same_entry(existing, &entry, cfg!(windows))) {
class_paths.push(entry);
}
}
}
fn use_snapshot(info: &mut JavaLaunchInfo, built: &[PathBuf], snapshot: &std::path::Path) -> Vec<PathBuf> {
use_snapshot_on(info, built, snapshot, cfg!(windows))
}
@@ -338,20 +402,92 @@ pub(crate) fn register(d: &mut Dispatcher) {
});
}
/// An annotation processor's error for writing a file this build already wrote (JDT's
/// `IdeFilerImpl`: "Source file already created", javac's Filer: "Attempt to recreate a file").
fn processor_recreated_a_file(message: &str) -> bool {
message.contains("FilerException")
&& (message.contains("Source file already created") || message.contains("Attempt to recreate a file"))
}
fn drop_processor_recreate_errors(info: &mut JavaLaunchInfo) {
let before = info.errors.len();
info.errors.retain(|e| !processor_recreated_a_file(&e.message));
info.error_count = info.error_count.saturating_sub((before - info.errors.len()) as u32);
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn recognises_a_processor_recreating_its_output() {
assert!(processor_recreated_a_file(
"Internal error in the mapping processor: java.lang.RuntimeException: \
javax.annotation.processing.FilerException: Source file already created: /m/target/generated-sources/annotations/a/AMapper__4Impl.java"
));
assert!(processor_recreated_a_file(
"javax.annotation.processing.FilerException: Attempt to recreate a file for type a.B"
));
assert!(!processor_recreated_a_file("Source file already created"));
assert!(!processor_recreated_a_file("cannot find symbol"));
}
#[test]
fn a_delegated_launch_ignores_jdt_processor_recreate_errors() {
let error = |message: &str| ide_lsp::jdtls::JavaCompileError {
path: "src/main/java/a/A.java".into(),
line: 1,
column: 1,
message: message.into(),
};
let mut info = JavaLaunchInfo {
error_count: 2,
errors: vec![
error(
"javax.annotation.processing.FilerException: Source file already created: /m/AMapper__1Impl.java",
),
error("cannot find symbol"),
],
..Default::default()
};
drop_processor_recreate_errors(&mut info);
assert_eq!(info.error_count, 1);
assert_eq!(info.errors.len(), 1);
assert_eq!(info.errors[0].message, "cannot find symbol");
}
#[test]
fn delegated_launches_with_jdt_errors_recompile_with_maven() {
assert_eq!(delegated_policy(BuildPolicy::Make, true, 1), BuildPolicy::Rebuild);
assert_eq!(delegated_policy(BuildPolicy::Automatic, true, 2), BuildPolicy::Rebuild);
assert_eq!(delegated_policy(BuildPolicy::Make, true, 0), BuildPolicy::Make);
assert_eq!(delegated_policy(BuildPolicy::Make, false, 1), BuildPolicy::Make);
assert_eq!(
delegated_policy(BuildPolicy::MakeAndContinueOnErrors, true, 1),
BuildPolicy::MakeAndContinueOnErrors
);
assert!(rebuild_on_jdt_errors(BuildPolicy::Make, true, 1));
assert!(rebuild_on_jdt_errors(BuildPolicy::Automatic, true, 2));
assert!(!rebuild_on_jdt_errors(BuildPolicy::Make, true, 0));
assert!(!rebuild_on_jdt_errors(BuildPolicy::Make, false, 1));
assert!(!rebuild_on_jdt_errors(BuildPolicy::MakeAndContinueOnErrors, true, 1));
}
#[test]
fn maven_classpath_probe_failure_falls_back_to_jdtls_classpaths() {
let maven = std::env::join_paths(["maven.jar", "snapshot.jar"]).unwrap().to_string_lossy().into_owned();
let mut class_paths = vec!["snapshot.jar".to_owned()];
let mut info =
JavaLaunchInfo { class_paths: vec!["snapshot.jar".into(), "jdtls.jar".into()], ..Default::default() };
merge_maven_launch_classpath(&mut class_paths, &mut info, Ok(maven)).unwrap();
assert_eq!(class_paths, ["snapshot.jar", "maven.jar"]);
merge_maven_launch_classpath(&mut class_paths, &mut info, Err(IdeError::internal("offline dependency")))
.unwrap();
assert_eq!(class_paths, ["snapshot.jar", "maven.jar", "jdtls.jar"]);
assert!(info.hints[0].contains("offline dependency"), "{:?}", info.hints);
}
#[test]
fn cancelled_maven_classpath_probe_propagates_without_fallback() {
let mut class_paths = vec!["snapshot.jar".to_owned()];
let mut info = JavaLaunchInfo { class_paths: vec!["jdtls.jar".into()], ..Default::default() };
let error = merge_maven_launch_classpath(&mut class_paths, &mut info, Err(IdeError::cancelled())).unwrap_err();
assert_eq!(error.kind, ErrorKind::Cancelled);
assert_eq!(class_paths, ["snapshot.jar"]);
assert!(info.hints.is_empty());
}
#[test]
@@ -395,7 +531,7 @@ mod tests {
}
#[test]
fn clean_external_build_clears_stale_jdt_errors() {
fn authoritative_external_result_clears_stale_jdt_errors() {
let mut info = JavaLaunchInfo {
error_count: 7,
errors: vec![ide_lsp::jdtls::JavaCompileError {
+71 -4
Переглянути файл
@@ -14,6 +14,10 @@ pub(super) struct Outputs {
pub originals: Vec<PathBuf>,
#[serde(skip)]
pub failure: Option<String>,
/// Whether this result replaces jdtls diagnostics with a clean Maven build or a
/// verified snapshot. It is runtime-only because reuse is determined per launch.
#[serde(skip)]
pub authoritative: bool,
}
impl Outputs {
@@ -42,6 +46,9 @@ pub(super) fn has_record(root: &Path, cache: &Path, module: &Path) -> bool {
pub(super) struct BuildOptions {
pub test: bool,
pub policy: BuildPolicy,
/// A JDT error makes a cache miss rebuild, but it must not prevent an unchanged
/// snapshot from being reused on the next launch.
pub rebuild_on_jdt_errors: bool,
}
/// Owned by Project: running JVMs may still load superseded outputs until it drops.
@@ -75,7 +82,7 @@ impl Session {
cancel: &CancellationToken,
) -> IdeResult<Outputs> {
self.initialize();
let BuildOptions { test, policy } = options;
let BuildOptions { test, policy, rebuild_on_jdt_errors } = options;
let cache = &self.cache;
let record =
record_path(root, cache, module, test).ok_or_else(|| IdeError::internal("launch outside project"))?;
@@ -115,8 +122,14 @@ impl Session {
&& outputs.reusable(&before, &snapshots)
{
tracing::info!(module = %module.display(), test, "Reusing Maven launch snapshot");
return Ok(previous.unwrap());
let mut outputs = previous.unwrap();
outputs.authoritative = true;
return Ok(outputs);
}
// JDT can report errors from its own classpath while the last Maven snapshot is
// still valid. Reuse that snapshot first; only a cache miss needs the expensive
// clean rebuild that replaces JDT's error-stub classes.
let policy = if rebuild_on_jdt_errors { BuildPolicy::Rebuild } else { policy };
// Retain a non-reusable record even on failure, so removing the final mixed
// source still delegates to Maven to clean its old classes on the next launch.
fs::create_dir_all(record.parent().unwrap())?;
@@ -156,7 +169,13 @@ impl Session {
if cancel.is_cancelled() {
return Err(IdeError::cancelled());
}
let mut outputs = Outputs { fingerprint: None, snapshot, originals, failure: built.failure };
let mut outputs = Outputs {
fingerprint: None,
snapshot,
originals,
failure: built.failure,
authoritative: policy == BuildPolicy::Rebuild,
};
if outputs.failure.is_none() && before.is_some() && before == after {
outputs.fingerprint = before;
fs::create_dir_all(record.parent().unwrap())?;
@@ -334,6 +353,48 @@ mod tests {
);
}
#[test]
fn jdt_errors_reuse_an_unchanged_snapshot_and_rebuild_only_on_a_cache_miss() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
let model = fixture(root);
let cancel = CancellationToken::new();
let mut session = Session::new(root.join("cache"));
let first = session.build(root, &model, root, BuildOptions::default(), None, &cancel).unwrap();
assert!(!first.authoritative);
let reused = session
.build(
root,
&model,
root,
BuildOptions { rebuild_on_jdt_errors: true, ..Default::default() },
None,
&cancel,
)
.unwrap();
assert_eq!(reused.snapshot, first.snapshot);
assert!(reused.authoritative);
assert_eq!(fs::read_to_string(root.join("calls")).unwrap().lines().count(), 1);
fs::write(root.join("src/main/kotlin/Main.kt"), "updated").unwrap();
let rebuilt = session
.build(
root,
&model,
root,
BuildOptions { rebuild_on_jdt_errors: true, ..Default::default() },
None,
&cancel,
)
.unwrap();
assert_ne!(rebuilt.snapshot, first.snapshot);
assert!(rebuilt.authoritative);
let calls = fs::read_to_string(root.join("calls")).unwrap();
assert_eq!(calls.lines().count(), 2);
assert!(calls.lines().last().unwrap().contains("clean"));
}
#[test]
fn successful_snapshots_reuse_across_close_and_rebuild_for_changes_missing_outputs_or_failure() {
let dir = tempfile::tempdir().unwrap();
@@ -483,7 +544,13 @@ mod tests {
fs::write(snapshots.join("launch-file"), "keep").unwrap();
let record = cache.join("java-launch/records/deep/module/test.json");
fs::create_dir_all(record.parent().unwrap()).unwrap();
let outputs = Outputs { fingerprint: None, snapshot: retained.clone(), originals: Vec::new(), failure: None };
let outputs = Outputs {
fingerprint: None,
snapshot: retained.clone(),
originals: Vec::new(),
failure: None,
authoritative: false,
};
fs::write(record, serde_json::to_vec(&outputs).unwrap()).unwrap();
let mut session = Session::new(cache);
session.initialize();
+27 -12
Переглянути файл
@@ -1483,23 +1483,38 @@ pub async fn lsp_restart_named(host: &Host, name: &str) -> IdeResult<()> {
pub async fn apply_workspace_edit(host: &Host, window: &str, edit: WorkspaceEdit) -> IdeResult<()> {
let project = host.project(window)?;
blocking(move || {
// Multi-file refactorings leave a Local History point to go back to.
let paths: Vec<&std::path::Path> = edit
.files
.iter()
.filter_map(|f| match f {
ide_core::FileEdit::Edit { path, .. } => Some(&**path),
_ => None,
})
.collect();
if edit.files.len() > 1 {
project.record_before_refactoring(&paths, "Apply changes");
}
record_before_multi_file_edit(&project, &edit);
apply_edit(&project.documents, &edit)
})
.await
}
/// [`apply_workspace_edit`] for the MCP server's tools: undoable the same way, but the UI does
/// not announce it as a refactoring.
pub async fn apply_agent_workspace_edit(host: &Host, window: &str, edit: WorkspaceEdit) -> IdeResult<()> {
let project = host.project(window)?;
blocking(move || {
record_before_multi_file_edit(&project, &edit);
project.documents.apply_agent_workspace_edit(&edit)
})
.await
}
/// Multi-file edits leave a Local History point to go back to.
fn record_before_multi_file_edit(project: &crate::Project, edit: &WorkspaceEdit) {
let paths: Vec<&std::path::Path> = edit
.files
.iter()
.filter_map(|f| match f {
ide_core::FileEdit::Edit { path, .. } => Some(&**path),
_ => None,
})
.collect();
if edit.files.len() > 1 {
project.record_before_refactoring(&paths, "Apply changes");
}
}
pub async fn prepare_workspace_edit(host: &Host, window: &str, mut edit: WorkspaceEdit) -> IdeResult<WorkspaceEdit> {
let project = host.project(window)?;
blocking(move || {
+1
Переглянути файл
@@ -35,6 +35,7 @@ pub mod syntax;
pub mod term;
pub mod tools;
pub mod vcs;
pub mod web;
use crate::Host;
use ide_core::{IdeError, IdeResult};
+12 -1
Переглянути файл
@@ -164,10 +164,21 @@ pub fn set_settings(
// A LightEdit window has no project layer: its values are global.
let project_scope = matches!(scope, SettingsScope::Project) && current.as_ref().is_some_and(|p| !p.light);
let is_proxy = key == "proxy" || key.starts_with("proxy.");
let Some(value) = host.take_proxy_password(key, value)? else {
let Some(mut value) = host.take_proxy_password(key, value)? else {
host.apply_proxy();
return Ok(SettingsApplied { settings: settings(host, window), project_scope, roots_changed: None });
};
// One canonical spelling per linked POM, including a project outside the opened root.
if project_scope
&& key == "languages.java.maven.linkedProjects"
&& let Some(project) = current.as_ref()
&& let Some(list) = value.as_array()
{
let poms: Vec<String> = list.iter().filter_map(|item| item.as_str().map(str::to_owned)).collect();
if poms.len() == list.len() {
value = serde_json::json!(ide_build::maven::normalize_linked_poms(&project.root, &poms));
}
}
let settings = match &current {
Some(p) if project_scope => p.settings.set(scope, key, value)?,
Some(p) => {
+68 -9
Переглянути файл
@@ -3,6 +3,7 @@ use crate::Host;
use ide_core::remote_settings::RunTargetKind;
use ide_core::{IdeError, IdeResult, RunId};
use ide_run::{OutputChunk, RunConfig, RunInfo, StartOptions};
use std::path::Path;
use std::time::Duration;
/// Detected configs followed by user-defined ones (user ids win on conflict).
@@ -138,11 +139,24 @@ pub fn save_configs(
ide_run::save_user_configs_with_removals(&project.root, &configs, removed_detected_ids)
}
/// The project `config` runs in (a workspace member, by its working directory) and its settings.
/// The project `config` runs in (a workspace member, by its working directory) and its
/// settings, with the primary project as the inherited fallback: a member with no Python
/// of its own runs on the primary project's interpreter (see `apply_to_run_configs_in`).
fn settings_for(
host: &Host,
project: &crate::project::Project,
config: &mut RunConfig,
) -> IdeResult<(std::path::PathBuf, ide_core::settings::Settings, Option<std::path::PathBuf>)> {
let (root, settings) = settings_only(host, project, config)?;
let fallback = (root != project.root).then_some(project.root.clone());
Ok((root, settings, fallback))
}
/// [`settings_for`] without the fallback, for callers that only rebase or read settings.
fn settings_only(
host: &Host,
project: &crate::project::Project,
config: &mut RunConfig,
) -> IdeResult<(std::path::PathBuf, ide_core::settings::Settings)> {
let workspace = ide_workspace::workspace::Workspace::load(&project.root)?;
let owner = ide_run::scope::owner(&project.root, config);
@@ -180,8 +194,8 @@ pub fn without_derived(host: &Host, window: &str, configs: Vec<RunConfig>) -> Id
if c.kind == "gradle" {
return Ok(c);
}
let (root, settings) = settings_for(host, &project, &mut c)?;
Ok(ide_sdk::apply::without_derived(&c, &root, &settings))
let (root, settings, fallback) = settings_for(host, &project, &mut c)?;
Ok(ide_sdk::apply::without_derived_in(&c, &root, fallback.as_deref(), &settings))
})
.collect()
}
@@ -194,8 +208,13 @@ pub fn apply_settings(host: &Host, window: &str, configs: Vec<RunConfig>) -> Ide
.into_iter()
.map(|mut c| {
if c.kind != "gradle" {
let (root, settings) = settings_for(host, &project, &mut c)?;
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut c), &root, &settings);
let (root, settings, fallback) = settings_for(host, &project, &mut c)?;
ide_sdk::apply::apply_to_run_configs_in(
std::slice::from_mut(&mut c),
&root,
fallback.as_deref(),
&settings,
);
}
Ok(c)
})
@@ -215,7 +234,19 @@ pub async fn start_request(
) -> IdeResult<RunInfo> {
let cancel = request.map(|id| host.begin_request(id)).unwrap_or_default();
let operation = config.extras.params.get("plugin.operation").cloned();
let result = start_inner(host, window, config, on_output, &cancel).await;
let mut config = config;
let agent = match host.project(window) {
Ok(project) => host.mcp_agent_runs.launch(&project.root, window, &mut config),
Err(_) => None,
};
let on_output = match &agent {
Some(agent) => agent.wrap_output(on_output),
None => on_output,
};
let result = start_inner(host, window, config, on_output, agent.as_ref(), &cancel).await;
if let Some(agent) = &agent {
agent.started(&result);
}
if let Some(id) = request {
host.end_request(id);
}
@@ -230,6 +261,7 @@ async fn start_inner(
window: &str,
mut config: RunConfig,
on_output: Emit<OutputChunk>,
agent: Option<&crate::mcp::run::agent::Launch>,
cancel: &ide_core::CancellationToken,
) -> IdeResult<RunInfo> {
if cancel.is_cancelled() {
@@ -263,6 +295,9 @@ async fn start_inner(
if let Some(plugins) = host.plugin_host() {
sink = plugins.run_operation_sink(window, &mut config, sink)?;
}
if let Some(agent) = agent {
sink = agent.wrap_sink(sink);
}
if config.command.is_empty() {
return Err(IdeError::invalid("run configuration has no command"));
}
@@ -329,7 +364,7 @@ async fn start_inner(
return host.remote.run(&settings, &project.root, &config, sink.clone(), Box::new(move |c| out(c))).await;
}
let mut config = config;
let (root, settings) = settings_for(host, &project, &mut config)?;
let (root, settings, fallback) = settings_for(host, &project, &mut config)?;
let root = &root;
// Configurations made in the UI name bare tools (`godot`) that may live outside PATH.
// Maven re-applies too: a config listed before a settings change (daemon toggle,
@@ -338,17 +373,38 @@ async fn start_inner(
if config.kind == "maven" {
ide_sdk::apply::prepare_maven_run(&mut config, root, &settings);
} else if matches!(config.kind.as_str(), "gradle" | "tomcat" | "junit") {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut config), root, &settings);
ide_sdk::apply::apply_to_run_configs_in(
std::slice::from_mut(&mut config),
root,
fallback.as_deref(),
&settings,
);
} else {
ide_sdk::apply::resolve_run_command(&mut config, root, &settings);
}
if config.kind == "go" {
ide_sdk::apply::apply_to_run_configs(std::slice::from_mut(&mut config), root, &settings);
ide_sdk::apply::apply_to_run_configs_in(
std::slice::from_mut(&mut config),
root,
fallback.as_deref(),
&settings,
);
ide_sdk::go::apply_to_run(&mut config, &settings.languages.go.build_tags);
}
if config.command == ide_sdk::godot::COMMAND {
return Err(IdeError::unavailable(ide_sdk::godot::NOT_FOUND));
}
// A Python configuration pointing at a missing interpreter fails in the depth of
// process spawn otherwise; name the cause and the fix like PyCharm's "Invalid SDK".
if matches!(config.kind.as_str(), "python" | "pytest" | "django-test")
&& !Path::new(&config.command).is_file()
&& Path::new(&config.command).components().count() > 1
{
return Err(IdeError::invalid(format!(
"The configured Python interpreter does not exist: {}\nSelect a working interpreter in Settings › Languages & Frameworks › Python Interpreter.",
config.command
)));
}
if config.kind == "tomcat" {
if host.processes.list().iter().any(|r| r.running && r.config_id == config.id) {
return Err(IdeError::invalid("This Tomcat configuration is already running"));
@@ -410,6 +466,7 @@ pub fn stop(host: &Host, window: &str, id: RunId) -> IdeResult<()> {
if let Some(plugins) = host.plugin_host() {
plugins.run_operation_stopped(id);
}
host.mcp_agent_runs.stopping(id);
let settings = match host.project(window) {
Ok(p) => p.settings.effective(),
Err(_) => host.global_settings.effective(),
@@ -421,7 +478,9 @@ pub fn stop(host: &Host, window: &str, id: RunId) -> IdeResult<()> {
host.processes.stop_with_grace(id, grace)
}
/// Input the user typed into a run (the agent's goes to the process directly).
pub fn stdin(host: &Host, id: RunId, data: &str) -> IdeResult<()> {
host.mcp_agent_runs.user_input(id, data);
if host.remote.runs.contains(id) {
return host.remote.runs.write_stdin(id, data);
}
+118 -1
Переглянути файл
@@ -7,6 +7,8 @@ use ide_core::{IdeError, IdeResult, RequestId};
use ide_sdk::python::PythonPackage;
use ide_sdk::{Sdk, SdkKind};
use std::path::PathBuf;
use std::sync::Mutex;
use std::sync::atomic::{AtomicUsize, Ordering};
fn join_err(e: impl std::fmt::Display) -> IdeError {
IdeError::internal(e.to_string())
@@ -20,6 +22,14 @@ pub struct LocatedTools {
pub compile_dir: ide_core::IpcPath,
}
/// An interpreter another project uses: the project root and the SDK itself.
#[derive(Clone, Debug, serde::Serialize, specta::Type)]
#[serde(rename_all = "camelCase")]
pub struct ProjectInterpreter {
pub project: ide_core::IpcPath,
pub sdk: Sdk,
}
/// Installed interpreters / toolchains of `kind`; project-local ones first.
pub async fn sdk_detect(host: &Host, window: &str, kind: SdkKind) -> IdeResult<Vec<Sdk>> {
host.require_trust(window)?;
@@ -45,6 +55,53 @@ pub async fn sdk_detect(host: &Host, window: &str, kind: SdkKind) -> IdeResult<V
.map_err(join_err)
}
/// The Python interpreters of the projects opened recently (newest first), each with the
/// project it belongs to. The open project is skipped: [`sdk_detect`] lists its own.
pub async fn sdk_python_project_interpreters(host: &Host, window: &str) -> IdeResult<Vec<ProjectInterpreter>> {
host.require_trust(window)?;
let current = host.project(window).ok().map(|p| p.root.clone());
let config = host.config_dir.clone();
// Probing runs the project's interpreter (and its settings may name any program): only
// projects trusted in their own right, not ones last opened in Restricted Mode.
let roots: Vec<PathBuf> = ide_workspace::RecentProjects::load(&config)
.list()
.into_iter()
.map(|p| p.path.into_path_buf())
.filter(|root| current.as_deref() != Some(root.as_path()))
.filter(|root| host.trust.window_trusted(root).unwrap_or(false))
.collect();
tokio::task::spawn_blocking(move || {
// Each project resolves its configured interpreter, else its own virtualenv. The
// probes run `python`, so keep them off the async runtime and bound how many run at
// once: the recent list can hold ~30 projects, and each probe starts a process.
const PROBES: usize = 4;
let results = Mutex::new(Vec::with_capacity(roots.len()));
let next = AtomicUsize::new(0);
std::thread::scope(|scope| {
for _ in 0..roots.len().min(PROBES) {
scope.spawn(|| {
loop {
let index = next.fetch_add(1, Ordering::Relaxed);
let Some(root) = roots.get(index) else { break };
let settings = ide_workspace::SettingsStore::load(&config, Some(root)).effective();
let Some(python) = ide_sdk::apply::python_interpreter(root, &settings) else { continue };
let Some(sdk) = ide_sdk::python_interpreter_sdk(&python, Some(root)) else { continue };
results
.lock()
.unwrap()
.push((index, ProjectInterpreter { project: root.as_path().into(), sdk }));
}
});
}
});
let mut found = results.into_inner().unwrap();
found.sort_by_key(|(index, _)| *index);
found.into_iter().map(|(_, interpreter)| interpreter).collect()
})
.await
.map_err(join_err)
}
/// An existing Python executable or virtualenv folder as an SDK; `None` if Python cannot run.
pub async fn sdk_python_interpreter(host: &Host, window: &str, path: PathBuf) -> IdeResult<Option<Sdk>> {
host.require_trust(window)?;
@@ -69,7 +126,14 @@ pub fn sdk_python_effective(host: &Host, window: &str, near: Option<PathBuf>) ->
}
None => (project.root.clone(), project.settings.effective()),
};
Ok(ide_sdk::apply::python_interpreter_near(&root, near.as_deref(), &settings).map(Into::into))
// A member with no Python of its own falls back to the primary project's
// (the same chain run configurations use), so every answer names one interpreter.
let interpreter = ide_sdk::apply::python_interpreter_near(&root, near.as_deref(), &settings).or_else(|| {
(root != project.root)
.then(|| ide_sdk::apply::python_interpreter(&project.root, &project.settings.effective()))
.flatten()
});
Ok(interpreter.map(Into::into))
}
/// The PHP interpreter chosen in Settings › Languages & Frameworks › PHP; `None`: the `php` on `PATH`.
@@ -303,6 +367,10 @@ pub async fn sdk_python_pip(
/// Commands of this module the remote server answers.
pub(crate) fn register(d: &mut Dispatcher) {
d.add("sdk_detect", |c| async move { sdk_detect(&c.host, &c.window, c.arg("kind")?).await });
d.add(
"sdk_python_project_interpreters",
|c| async move { sdk_python_project_interpreters(&c.host, &c.window).await },
);
d.add(
"sdk_python_interpreter",
|c| async move { sdk_python_interpreter(&c.host, &c.window, c.arg("path")?).await },
@@ -418,4 +486,53 @@ mod tests {
assert!(sdk_python_interpreter(&host, "main", root.join("README.md")).await.unwrap().is_none());
host.close_project("main");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn another_projects_interpreter_is_listed_with_its_project() {
let temp = tempfile::tempdir().unwrap();
let base = dunce::canonicalize(temp.path()).unwrap();
let root = base.join("project");
let other = base.join("shop");
std::fs::create_dir_all(other.join(".openidea")).unwrap();
std::fs::create_dir_all(&root).unwrap();
let python = fake_venv(&other, ".venv");
std::fs::write(
other.join(".openidea/settings.toml"),
"[languages.python]\ninterpreter = \".venv/bin/python\"\n",
)
.unwrap();
// The projects the user opened before this one.
let config = base.join("config");
std::fs::create_dir_all(&config).unwrap();
let recent = serde_json::json!([
{ "path": other.to_str().unwrap(), "name": "shop", "openedAtMs": 2.0, "branch": null },
{ "path": root.to_str().unwrap(), "name": "project", "openedAtMs": 1.0, "branch": null },
]);
std::fs::write(config.join("recent.json"), serde_json::to_vec(&recent).unwrap()).unwrap();
let host = Host::new(HostConfig {
config_dir: config,
data_dir: base.join("data"),
local_data_dir: base.join("local"),
languages: Arc::new(ide_language::LanguageRegistry::load(None).unwrap()),
syntax: Arc::default(),
sink_for: Box::new(|_| Arc::new(|_| {})),
owner: |w| w,
runtime: tokio::runtime::Handle::current(),
project_environment: false,
});
host.trust.decide(&root, true, false).unwrap();
host.open_project("main", &root).unwrap();
// A project last opened in Restricted Mode: its interpreter is never run.
host.trust.decide(&other, false, false).unwrap();
assert!(sdk_python_project_interpreters(&host, "main").await.unwrap().is_empty());
host.trust.decide(&other, true, false).unwrap();
let list = sdk_python_project_interpreters(&host, "main").await.unwrap();
assert_eq!(list.len(), 1, "{list:?}");
assert_eq!(&*list[0].project, other.as_path());
assert_eq!(&*list[0].sdk.path, python.as_path());
assert_eq!(list[0].sdk.name, "Python 3.12 (shop)");
host.close_project("main");
}
}
+85
Переглянути файл
@@ -48,6 +48,91 @@ pub fn spawn(
Ok(info)
}
/// Tools › Start SSH Session: a login shell on SSH configuration `id`, opened by the
/// built-in client with the configuration's saved password, key or agent.
pub async fn spawn_ssh(
host: &Host,
window: &str,
id: &str,
cols: u16,
rows: u16,
on_output: Emit<String>,
) -> IdeResult<TerminalInfo> {
let config = host.ssh.config(id)?;
if config.host.trim().is_empty() {
return Err(ide_core::IdeError::invalid(format!("SSH configuration {} has no host", config.name)));
}
let size = (if cols == 0 { 80 } else { cols }, if rows == 0 { 24 } else { rows });
let session = host.ssh.session(id).await?;
let (feed, output) = ide_terminal::feed();
let out = feed.clone();
let process = session
.spawn(
"",
Some(size),
Box::new(move |chunk| match chunk {
ide_ssh::Output::Stdout(data) | ide_ssh::Output::Stderr(data) => out.output(data),
}),
)
.await?;
let exit = process.exit_status();
let (input, mut inputs) = tokio::sync::mpsc::unbounded_channel();
host.runtime.spawn(async move {
let p = process;
while let Some(op) = inputs.recv().await {
let result = match op {
SshInput::Data(data) => p.write(&data).await,
SshInput::Resize(cols, rows) => p.resize(cols, rows).await,
SshInput::Close => {
p.close().await;
break;
}
};
if let Err(e) = result {
tracing::debug!("ssh terminal: {e}");
}
}
});
host.runtime.spawn(async move {
let code = exit.await;
feed.exited(code.and_then(|c| i32::try_from(c).ok()));
});
let title = if config.name.trim().is_empty() { config.label() } else { config.name.clone() };
host.terminals.attach(
title,
size,
Box::new(SshTerminal(input)),
output,
host.owner_window(window).to_owned(),
host.window_sink(window),
Box::new(move |chunk| on_output(chunk)),
)
}
enum SshInput {
Data(Vec<u8>),
Resize(u16, u16),
Close,
}
/// Input of an SSH shell, sent in order by one task.
struct SshTerminal(tokio::sync::mpsc::UnboundedSender<SshInput>);
impl ide_terminal::TerminalChannel for SshTerminal {
fn write(&self, data: &[u8]) -> IdeResult<()> {
self.0.send(SshInput::Data(data.to_vec())).map_err(|_| ide_core::IdeError::invalid("the SSH session has ended"))
}
fn resize(&self, cols: u16, rows: u16) -> IdeResult<()> {
let _ = self.0.send(SshInput::Resize(cols, rows));
Ok(())
}
fn close(&self) {
let _ = self.0.send(SshInput::Close);
}
}
pub fn write(host: &Host, id: TerminalId, data: &str) -> IdeResult<()> {
host.terminals.write(id, data)
}
+50 -15
Переглянути файл
@@ -138,6 +138,11 @@ impl LogAccess {
.ok_or_else(|| ide_core::IdeError::unavailable("external Git log is not open"))?
.with_log_repo_at(at, f)
}
/// `with_log_repo_at` through the backend-agnostic Changes-view surface.
fn with_log_backend_at<T>(&self, at: Option<&Path>, f: impl FnOnce(&dyn Backend) -> IdeResult<T>) -> IdeResult<T> {
self.with_log_repo_at(at, |r| f(r as &dyn Backend))
}
}
/// A repository of the project (IDEA's VCS root).
@@ -280,6 +285,16 @@ fn per_repo(
Ok(())
}
/// `per_repo` through the backend-agnostic Changes-view surface.
fn per_backend(
project: &Project,
paths: &[PathBuf],
recorded: Option<&str>,
f: impl Fn(&dyn Backend, &[PathBuf]) -> IdeResult<()>,
) -> IdeResult<()> {
per_repo(project, paths, recorded, |r, paths| f(r as &dyn Backend, paths))
}
/// A shelf is one patch for one owning repository. Validate the whole selection before reverting files.
fn shelf_repository(project: &Project, paths: &[PathBuf]) -> IdeResult<PathBuf> {
let repos = project.repos();
@@ -348,7 +363,7 @@ pub async fn vcs_line_hunks(host: &Host, window: &str, path: PathBuf) -> IdeResu
let project = host.project(window)?;
blocking(move || {
let text = project.text_of(&path)?;
project.with_repo_of(&path, |r| r.line_hunks(&path, &text, false, project.vcs_encoding(&path, None)?))
project.with_backend_of(&path, |r| r.line_hunks(&path, &text, false, project.vcs_encoding(&path, None)?))
})
.await
}
@@ -360,7 +375,7 @@ pub async fn vcs_file_at(host: &Host, window: &str, path: PathBuf, rev: Option<S
});
blocking(move || {
if let Some(project) = project {
return project.with_log_repo_of(&path, |r| {
return project.with_log_backend_of(&path, |r| {
r.file_at(&path, rev.as_deref(), project.vcs_encoding(&path, rev.as_deref())?)
});
}
@@ -373,6 +388,25 @@ pub async fn vcs_file_at(host: &Host, window: &str, path: PathBuf, rev: Option<S
.await
}
/// The file's git mode at `rev` (`100644`, `100755`, `120000`…); `"WT"` reads the
/// worktree's. `None` when the path is absent on that side. The diff viewer shows
/// "File mode changed" when the texts compare equal but the modes differ.
pub async fn vcs_file_mode(host: &Host, window: &str, path: PathBuf, rev: Option<String>) -> IdeResult<Option<String>> {
let project = host.project(window).ok();
let external = host.external_logs.read().get(host.owner_window(window)).and_then(|rs| {
rs.iter().filter(|r| path.starts_with(r.root())).max_by_key(|r| r.root().components().count()).cloned()
});
blocking(move || {
if let Some(project) = project {
return project.with_log_backend_of(&path, |r| r.file_mode_at(&path, rev.as_deref()));
}
external
.ok_or_else(|| ide_core::IdeError::unavailable("external Git log is not open"))?
.file_mode_at(&path, rev.as_deref())
})
.await
}
pub async fn vcs_blame(
host: &Host,
window: &str,
@@ -384,7 +418,7 @@ pub async fn vcs_blame(
// An explicit text wins (merge view annotates its side panes, not the worktree).
blocking(move || {
let text = text.or_else(|| project.documents.text_of(&path));
project.with_repo_of(&path, |r| {
project.with_backend_of(&path, |r| {
r.blame(&path, text.as_deref(), revision.as_deref(), project.vcs_encoding(&path, revision.as_deref())?)
})
})
@@ -405,7 +439,7 @@ pub async fn vcs_block_history(
let cancel = host.begin_request(request);
let result = blocking(move || {
let text = text.or_else(|| project.documents.text_of(&path));
project.with_repo_of(&path, |r| {
project.with_backend_of(&path, |r| {
r.block_history(&path, text.as_deref(), lines, project.vcs_encoding(&path, None)?, &cancel)
})
})
@@ -422,7 +456,7 @@ pub async fn vcs_log(host: &Host, window: &str, repo: Option<PathBuf>, query: Lo
let result = blocking(move || {
project.with_log_repo_at(repo.as_deref(), |r| {
r.enable_log_index(&project.cache);
r.log_cancellable(&query, &cancel)
(r as &dyn Backend).log(&query, &cancel)
})
})
.await;
@@ -521,7 +555,7 @@ pub async fn vcs_commit_files(
commit: String,
) -> IdeResult<Vec<ChangedFile>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.commit_files(&commit))).await
blocking(move || project.with_log_backend_at(repo.as_deref(), |r| r.commit_files(&commit))).await
}
/// Combined changes of several commits, `commits` oldest first.
@@ -532,7 +566,7 @@ pub async fn vcs_commits_changes(
commits: Vec<String>,
) -> IdeResult<Vec<RevisionChange>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.commits_changes(&commits))).await
blocking(move || project.with_log_backend_at(repo.as_deref(), |r| r.commits_changes(&commits))).await
}
pub async fn vcs_branches(host: &Host, window: &str, repo: Option<PathBuf>) -> IdeResult<Vec<Branch>> {
@@ -548,7 +582,7 @@ pub async fn vcs_stashes(host: &Host, window: &str, repo: Option<PathBuf>) -> Id
pub async fn vcs_stage(host: &Host, window: &str, paths: Vec<PathBuf>) -> IdeResult<()> {
host.require_trust(window)?;
let project = host.project(window)?;
blocking(move || per_repo(&project, &paths, None, |r, paths| r.stage(paths))).await
blocking(move || per_backend(&project, &paths, None, |r, paths| r.stage(paths))).await
}
/// Project-host operation (also registered with the remote dispatcher).
@@ -583,7 +617,7 @@ pub async fn vcs_add_to_ignore(
pub async fn vcs_unstage(host: &Host, window: &str, paths: Vec<PathBuf>) -> IdeResult<()> {
host.require_trust(window)?;
let project = host.project(window)?;
blocking(move || per_repo(&project, &paths, None, |r, paths| r.unstage(paths))).await
blocking(move || per_backend(&project, &paths, None, |r, paths| r.unstage(paths))).await
}
pub async fn vcs_stage_patch(host: &Host, window: &str, patch: String) -> IdeResult<()> {
@@ -595,7 +629,7 @@ pub async fn vcs_stage_patch(host: &Host, window: &str, patch: String) -> IdeRes
pub async fn vcs_discard(host: &Host, window: &str, paths: Vec<PathBuf>) -> IdeResult<()> {
host.require_trust(window)?;
let project = host.project(window)?;
blocking(move || per_repo(&project, &paths, Some("Rollback"), |r, paths| r.discard(paths))).await
blocking(move || per_backend(&project, &paths, Some("Rollback"), |r, paths| r.discard(paths))).await
}
pub async fn vcs_commit(
@@ -609,8 +643,8 @@ pub async fn vcs_commit(
// Commit hooks may run for a long time.
let project = host.project(window)?;
blocking(move || {
project.with_repo_recorded(repo.as_deref(), None, |r| {
r.commit_with(&message, &options, &|path| project.vcs_encoding(path, None))
project.with_backend_recorded(repo.as_deref(), None, |r| {
r.commit(&message, &options, &|path| project.vcs_encoding(path, None))
})
})
.await
@@ -867,7 +901,7 @@ pub async fn vcs_conflict_versions(host: &Host, window: &str, path: PathBuf) ->
pub async fn vcs_diff_text(host: &Host, window: &str, path: PathBuf, staged: bool) -> IdeResult<String> {
let project = host.project(window)?;
blocking(move || project.with_repo_of(&path, |r| r.diff_text(&path, staged, project.vcs_encoding(&path, None)?)))
blocking(move || project.with_backend_of(&path, |r| r.diff_text(&path, staged, project.vcs_encoding(&path, None)?)))
.await
}
@@ -1182,7 +1216,7 @@ pub async fn vcs_diff_with_worktree(
rev: String,
) -> IdeResult<Vec<ChangedFile>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.diff_with_worktree(&rev))).await
blocking(move || project.with_log_backend_at(repo.as_deref(), |r| r.diff_with_worktree(&rev))).await
}
/// Files that differ between two revisions (the "N files updated" of Update Project).
@@ -1194,7 +1228,7 @@ pub async fn vcs_diff_revisions(
to: String,
) -> IdeResult<Vec<ChangedFile>> {
let project = LogAccess::new(host, window, repo.as_deref());
blocking(move || project.with_log_repo_at(repo.as_deref(), |r| r.diff_revisions(&from, &to))).await
blocking(move || project.with_log_backend_at(repo.as_deref(), |r| r.diff_revisions(&from, &to))).await
}
/// Added and deleted lines between two revisions. `paths` limits the diff; empty diffs every file.
@@ -1392,6 +1426,7 @@ pub(crate) fn register(d: &mut Dispatcher) {
d.add("vcs_branch_info", |c| async move { vcs_branch_info(&c.host, &c.window, c.arg("repo")?).await });
d.add("vcs_line_hunks", |c| async move { vcs_line_hunks(&c.host, &c.window, c.arg("path")?).await });
d.add("vcs_file_at", |c| async move { vcs_file_at(&c.host, &c.window, c.arg("path")?, c.arg("rev")?).await });
d.add("vcs_file_mode", |c| async move { vcs_file_mode(&c.host, &c.window, c.arg("path")?, c.arg("rev")?).await });
d.add("vcs_blame", |c| async move {
vcs_blame(&c.host, &c.window, c.arg("path")?, c.arg("text")?, c.arg("revision")?).await
});
+11
Переглянути файл
@@ -0,0 +1,11 @@
//! The built-in web server's URLs.
use crate::Host;
use ide_core::IdeResult;
use std::path::Path;
/// The URL serving `file`, under the window's project when it holds the file.
pub async fn url(host: &Host, window: &str, file: &Path) -> IdeResult<String> {
let root = host.project(window).ok().map(|p| p.root.clone());
host.web.url_for(root.as_deref(), file).await
}
Loaded 100 of 835 files, more files were not shown because too many files have changed in this diff. Show more