Release v1.1: live results, duplicate verification, and a Settings tab.
Search results now track the disk as it changes (live.rs), duplicate groups can be confirmed byte-for-byte before deletion (verify.rs), the Options window becomes a Settings tab with configurable/sortable result columns, and a first-start tutorial explains the basics.
This commit is contained in:
parent
03d0b6a50f
commit
9760503a3f
76 changed files with 9666 additions and 1414 deletions
|
|
@ -63,12 +63,6 @@ jobs:
|
||||||
HOME: /root
|
HOME: /root
|
||||||
# The highest libc6 version the .deb is allowed to require.
|
# The highest libc6 version the .deb is allowed to require.
|
||||||
MAX_GLIBC: '2.35'
|
MAX_GLIBC: '2.35'
|
||||||
# full_index.rs asserts a heavy indexing root cannot stall a light one, and
|
|
||||||
# measures that as wall-clock stall. The 100 ms default is calibrated on a
|
|
||||||
# developer machine; this runner measured 188 ms for the same correct
|
|
||||||
# behaviour. 600 ms keeps the check meaningful - the regression it exists to
|
|
||||||
# catch is ~6x the healthy figure, so it would land near 1.2 s here.
|
|
||||||
QSB_STALL_BUDGET_MS: '600'
|
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
|
|
||||||
4
Cargo.lock
generated
4
Cargo.lock
generated
|
|
@ -3188,7 +3188,7 @@ dependencies = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "quicksearch-core"
|
name = "quicksearch-core"
|
||||||
version = "1.0.6"
|
version = "1.1.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"argon2",
|
"argon2",
|
||||||
"cfb",
|
"cfb",
|
||||||
|
|
@ -3222,7 +3222,7 @@ dependencies = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "quicksearch-gui"
|
name = "quicksearch-gui"
|
||||||
version = "1.0.6"
|
version = "1.1.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"ashpd",
|
"ashpd",
|
||||||
"chrono",
|
"chrono",
|
||||||
|
|
|
||||||
|
|
@ -25,7 +25,7 @@ members = [
|
||||||
# into a killed process instead of one skipped file.
|
# into a killed process instead of one skipped file.
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
version = "1.0.6"
|
version = "1.1.0"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
license = "GPL-3.0-or-later"
|
license = "GPL-3.0-or-later"
|
||||||
authors = ["Jeremy <jeremy@karsttech.com>"]
|
authors = ["Jeremy <jeremy@karsttech.com>"]
|
||||||
|
|
|
||||||
158
README.md
158
README.md
|
|
@ -187,7 +187,9 @@ The install is per-machine and asks for elevation. Into
|
||||||
| `uninstall.exe` | written by the installer; Add/Remove Programs runs it |
|
| `uninstall.exe` | written by the installer; Add/Remove Programs runs it |
|
||||||
|
|
||||||
The components page offers a Start menu shortcut (on) and a desktop shortcut
|
The components page offers a Start menu shortcut (on) and a desktop shortcut
|
||||||
(off); both are created for all users. No `config.toml` is installed, for the
|
(off); both are created for all users. The final page lists what was installed
|
||||||
|
and where — the install itself takes about a second, which without saying so
|
||||||
|
reads as a failure — and offers to start QuickSearch, ticked. No `config.toml` is installed, for the
|
||||||
same reason the `.deb` ships none — one next to the binaries is portable mode
|
same reason the `.deb` ships none — one next to the binaries is portable mode
|
||||||
(see [Configuration](#configuration)) and would override the personal config
|
(see [Configuration](#configuration)) and would override the personal config
|
||||||
of every account. The app writes `%APPDATA%\quicksearch\config.toml` on first
|
of every account. The app writes `%APPDATA%\quicksearch\config.toml` on first
|
||||||
|
|
@ -224,12 +226,32 @@ inside that folder.
|
||||||
`quicksearch` with no query arguments opens the app:
|
`quicksearch` with no query arguments opens the app:
|
||||||
|
|
||||||
- **Search**: results appear as you type; every keystroke cancels the
|
- **Search**: results appear as you type; every keystroke cancels the
|
||||||
previous search. One checkbox enables the two fuzzy passes. Sort by
|
previous search. One checkbox enables the two fuzzy passes, and once a
|
||||||
rank, name, path, size, or modified. Double-click a result to open it;
|
search has finished a button inside the right of the search box re-runs
|
||||||
|
it. Click a column heading to sort by it; **right-click any heading to
|
||||||
|
choose which columns are shown** — the path is always there, and size
|
||||||
|
and modified date start hidden, which is what buys the width the path
|
||||||
|
and the match get instead. The choice is saved (`[search.columns]`,
|
||||||
|
also in Settings → Search) and applies immediately. Sorting by a column
|
||||||
|
you then hide falls back to rank. Double-click a result to open it;
|
||||||
right-click it to reveal it in the file manager, open it, copy its
|
right-click it to reveal it in the file manager, open it, copy its
|
||||||
path, or build an ignore filter from it (session-only by default,
|
path, or build an ignore filter from it (session-only by default,
|
||||||
optionally persisted to the config). Result text can be selected and
|
optionally persisted to the config). Result text can be selected and
|
||||||
copied in place. Matches in file contents show highlighted snippets.
|
copied in place. A match in a file's **name** or **path** is
|
||||||
|
highlighted in that column; a match in its **contents** shows a
|
||||||
|
highlighted snippet in the Content Match column, with more of the
|
||||||
|
surrounding text on hover. Rows matched on name or path show a dash
|
||||||
|
there instead. With `[search] live_results` on (the default) the rows
|
||||||
|
actually on screen are watched, and what they show is read from the
|
||||||
|
files themselves: a rename, a deletion or an edit lands within a
|
||||||
|
second, whether or not indexing is running. The rows coming on screen
|
||||||
|
are also checked against the disk as they are watched, so one the index
|
||||||
|
was already out of date about corrects itself; the index is then
|
||||||
|
brought back in line for those files alone. Over a network share, where
|
||||||
|
the system reports no events, that check is all you get — the row is
|
||||||
|
right when it comes on screen and then holds still. Nothing is ever
|
||||||
|
added, removed or re-ordered underneath you; a file that disappears is
|
||||||
|
struck through where it sits. Editing the query drops every watch.
|
||||||
- **Manage Index**: full indexing status, Start/Stop/Automatic controls,
|
- **Manage Index**: full indexing status, Start/Stop/Automatic controls,
|
||||||
indexed folder list, full-text extension filters, ignore patterns, and
|
indexed folder list, full-text extension filters, ignore patterns, and
|
||||||
the indexing options. Stopping switches to manual mode and saves that
|
the indexing options. Stopping switches to manual mode and saves that
|
||||||
|
|
@ -244,25 +266,45 @@ inside that folder.
|
||||||
folder nothing has finished indexing reads "not yet indexed" rather
|
folder nothing has finished indexing reads "not yet indexed" rather
|
||||||
than zero, and because the figures come from completed runs they do
|
than zero, and because the figures come from completed runs they do
|
||||||
not move as live updates apply single changes in between.
|
not move as live updates apply single changes in between.
|
||||||
- **Duplicates**: files sharing a content hash, grouped.
|
- **Duplicates**: files sharing a content hash, grouped. That hash covers
|
||||||
|
each file's size and its first `processing.hash_length` bytes and nothing
|
||||||
|
else, which is the whole reason indexing is affordable — and the reason a
|
||||||
|
group is a strong suspicion rather than a fact. Right-click a group, or any
|
||||||
|
file in one, to settle it: every member is read through and compared byte
|
||||||
|
for byte, with progress and a Cancel button in a modal that then names each
|
||||||
|
file as identical, differing at a given byte, a different size, or
|
||||||
|
unreadable. Nothing is deleted or changed either way; the point is to know
|
||||||
|
before you delete something yourself.
|
||||||
- **Logs**: the lines the app would have printed to a terminal — warnings
|
- **Logs**: the lines the app would have printed to a terminal — warnings
|
||||||
from indexing, folder watching and opening files, newest last, with a
|
from indexing, folder watching and opening files, newest last, with a
|
||||||
filter box and Copy button. Launched from a desktop launcher (or on
|
filter box and Copy button. Launched from a desktop launcher (or on
|
||||||
Windows, where the app has no console at all) this is the only place
|
Windows, where the app has no console at all) this is the only place
|
||||||
they are visible.
|
they are visible.
|
||||||
- **Help**: an in-app quickstart — first indexing run, example queries,
|
- **Help**: an in-app quickstart — first indexing run, example queries,
|
||||||
what each tab does — pointing here for everything technical.
|
what each tab does — pointing here for everything technical. A brand-new
|
||||||
|
installation is shown a short click-through introduction covering the
|
||||||
|
same ground on its first launch; the Help tab brings it back. Upgrading
|
||||||
|
into this version does not raise it (see `[ui] tutorial_seen`).
|
||||||
|
- **Settings**: every configuration control in one place — the database
|
||||||
|
path, indexing and processing limits, search behaviour, the interface
|
||||||
|
(scale, shortcut, color scheme) and password protection. Each row
|
||||||
|
explains itself on hover. Edits are staged and applied together by
|
||||||
|
**Apply & Save**; leaving the tab with unapplied edits asks first. The
|
||||||
|
column choices and the password controls are the exceptions, acting the
|
||||||
|
moment they are used, since the Search tab's own header menu writes the
|
||||||
|
same settings. The indexed folder list and the indexing mode live on
|
||||||
|
Manage Index instead, next to the controls that act on them.
|
||||||
|
|
||||||
**Ctrl+Shift+F from anywhere** brings QuickSearch to the front, restoring
|
**Ctrl+Shift+F from anywhere** brings QuickSearch to the front, restoring
|
||||||
it if it was minimized, and puts the cursor in the search box with the
|
it if it was minimized, and puts the cursor in the search box with the
|
||||||
previous search selected, so the next thing you type is the new one. The
|
previous search selected, so the next thing you type is the new one. The
|
||||||
Options window's Interface section rebinds it — click the button and press
|
Settings tab's Interface section rebinds it — click the button and press
|
||||||
the keys — or switches it off. It is a system-wide shortcut, registered
|
the keys — or switches it off. It is a system-wide shortcut, registered
|
||||||
with Windows or with the X server, so it works while another application
|
with Windows or with the X server, so it works while another application
|
||||||
has focus. Wayland does not let an application claim a key, so there the
|
has focus. Wayland does not let an application claim a key, so there the
|
||||||
shortcut is registered with your desktop through the XDG desktop portal
|
shortcut is registered with your desktop through the XDG desktop portal
|
||||||
instead; your desktop then has the final say over which key it is, and its
|
instead; your desktop then has the final say over which key it is, and its
|
||||||
own keyboard settings are where to change it. The Options window says which
|
own keyboard settings are where to change it. The Settings tab says which
|
||||||
key it settled on. Wayland likewise gives no application a way to put itself
|
key it settled on. Wayland likewise gives no application a way to put itself
|
||||||
in front of what you are doing, so under it the shortcut selects the Search
|
in front of what you are doing, so under it the shortcut selects the Search
|
||||||
tab and the search box but leaves raising the window to the desktop; on X11
|
tab and the search box but leaves raising the window to the desktop; on X11
|
||||||
|
|
@ -301,7 +343,7 @@ processing; Windows Terminal has it, and older consoles get plain text.
|
||||||
|
|
||||||
The index contains the names and (by default) the full text of everything
|
The index contains the names and (by default) the full text of everything
|
||||||
it indexes — for most setups, your entire home directory. That is a lot of
|
it indexes — for most setups, your entire home directory. That is a lot of
|
||||||
concentrated risk in one file. **Options → Security → Enable password
|
concentrated risk in one file. **Settings → Security → Enable password
|
||||||
protection** encrypts the index on disk with SQLCipher; from then on
|
protection** encrypts the index on disk with SQLCipher; from then on
|
||||||
QuickSearch asks for the password every time it starts, in the GUI (an
|
QuickSearch asks for the password every time it starts, in the GUI (an
|
||||||
unlock screen before anything opens the index) and in the terminal (a
|
unlock screen before anything opens the index) and in the terminal (a
|
||||||
|
|
@ -315,6 +357,10 @@ rebuilds the index — there is no in-place conversion.
|
||||||
in the OS keychain — Secret Service/KWallet on Linux, Credential Manager
|
in the OS keychain — Secret Service/KWallet on Linux, Credential Manager
|
||||||
on Windows — and skips the prompt. Without a keychain daemon the option
|
on Windows — and skips the prompt. Without a keychain daemon the option
|
||||||
quietly falls back to prompting.
|
quietly falls back to prompting.
|
||||||
|
- **Show database key** asks for the password, then shows the raw SQLCipher
|
||||||
|
key as `0x…` (64 hex digits) with a copy button, for opening the index in
|
||||||
|
other SQLCipher tools. That key alone reads the index, so treat a copy of
|
||||||
|
it as carefully as the password.
|
||||||
- Scripts can set `QUICKSEARCH_PASSWORD` for non-interactive terminal
|
- Scripts can set `QUICKSEARCH_PASSWORD` for non-interactive terminal
|
||||||
search. Environment variables are readable by other processes of the
|
search. Environment variables are readable by other processes of the
|
||||||
same user (`/proc/<pid>/environ`) — prefer the keychain.
|
same user (`/proc/<pid>/environ`) — prefer the keychain.
|
||||||
|
|
@ -397,13 +443,13 @@ containing the binary, its config, and its index can be moved wholesale.
|
||||||
The GUI edits the config live; external edits apply on next start.
|
The GUI edits the config live; external edits apply on next start.
|
||||||
|
|
||||||
`[ui] search_hotkey` is the system-wide search shortcut, written the way
|
`[ui] search_hotkey` is the system-wide search shortcut, written the way
|
||||||
the Options window prints it (`Ctrl+Shift+F`): Ctrl, Alt and Shift in any
|
the Settings tab prints it (`Ctrl+Shift+F`): Ctrl, Alt and Shift in any
|
||||||
combination, plus one key, joined with `+`. An empty string switches it
|
combination, plus one key, joined with `+`. An empty string switches it
|
||||||
off. A value that is not a shortcut is not a config error — the app loads,
|
off. A value that is not a shortcut is not a config error — the app loads,
|
||||||
says so in the Options window, and runs without one.
|
says so on the Settings tab, and runs without one.
|
||||||
|
|
||||||
`[ui] color_scheme` is `dark` (the default) or `light`, changeable in the
|
`[ui] color_scheme` is `dark` (the default) or `light`, changeable on the
|
||||||
Options window and applied without a restart. It does not follow the
|
Settings tab and applied without a restart. It does not follow the
|
||||||
desktop's own light/dark setting: on Linux nothing in the window system
|
desktop's own light/dark setting: on Linux nothing in the window system
|
||||||
reports that, so the only way to know is to connect to the session message
|
reports that, so the only way to know is to connect to the session message
|
||||||
bus and subscribe to the user's settings feed — more of your session than a
|
bus and subscribe to the user's settings feed — more of your session than a
|
||||||
|
|
@ -499,7 +545,25 @@ Synchronous Rust: `std::thread` + `mpsc` channels, no async runtime.
|
||||||
bigger index — `indexing.content_extensions` remains the throttle. Files no larger than `processing.hash_length` skip that second
|
bigger index — `indexing.content_extensions` remains the throttle. Files no larger than `processing.hash_length` skip that second
|
||||||
pass entirely: the head the walk reads to hash them is already their
|
pass entirely: the head the walk reads to hash them is already their
|
||||||
whole content, so a plaintext body is extracted in the same `read` and
|
whole content, so a plaintext body is extracted in the same `read` and
|
||||||
stored complete. Every run ends — whether
|
stored complete. Every root runs its own pipeline — its own walker pool
|
||||||
|
and, once the walk ends, its own extraction pool — but every root's
|
||||||
|
*writes* go through one thread and one connection
|
||||||
|
(`indexing/pipeline.rs`), because that is what a single SQLite file
|
||||||
|
allows. That thread is where FTS5 tokenizes, up to `maximum_text_size`
|
||||||
|
of text per document inside the insert, and it is the run's dominant
|
||||||
|
cost. So its loop is scheduled around the walk, the disk-bound phase and
|
||||||
|
the one whose stall shows: each round serves every walking root first,
|
||||||
|
then one extracting root, and no turn runs past a 100 ms slice — an
|
||||||
|
extraction turn commits at the slice and carries the rows it did not
|
||||||
|
reach to its next turn. A walk therefore waits at most one slice per
|
||||||
|
round, which its walkers' channel absorbs, so a root walking a large tree
|
||||||
|
runs at its own rate while another root tokenizes big documents beside
|
||||||
|
it. What a root has left to extract is counted by its own content pass,
|
||||||
|
on that pass's read connection, rather than on the writer: on a large
|
||||||
|
root the count is seconds, and seconds of writer time is every other
|
||||||
|
root's walk standing still. Total write throughput is what one connection
|
||||||
|
tokenizing can do; the scheduling shares it fairly and keeps the walk
|
||||||
|
first, it does not raise it. Every run ends — whether
|
||||||
it completed or was stopped — with an optimize pass on its own connection:
|
it completed or was stopped — with an optimize pass on its own connection:
|
||||||
checkpoint, VACUUM if the file has at least 10% slack to reclaim, `PRAGMA
|
checkpoint, VACUUM if the file has at least 10% slack to reclaim, `PRAGMA
|
||||||
optimize`, checkpoint again. Progress streams through a polled
|
optimize`, checkpoint again. Progress streams through a polled
|
||||||
|
|
@ -555,6 +619,26 @@ Synchronous Rust: `std::thread` + `mpsc` channels, no async runtime.
|
||||||
handle and takes a single watch per root, filtering the events instead.
|
handle and takes a single watch per root, filtering the events instead.
|
||||||
Either way a tree too large to watch degrades to periodic reindexing
|
Either way a tree too large to watch degrades to periodic reindexing
|
||||||
rather than going silently stale.
|
rather than going silently stale.
|
||||||
|
- **Live results** (`live.rs`): a second, much smaller watcher, owned by the
|
||||||
|
frontend rather than the coordinator, pointed at the parent directories of
|
||||||
|
the result rows *currently on screen* once they have held still for a
|
||||||
|
moment. It watches directories, not the result files: editors save by
|
||||||
|
writing a temporary file and renaming it over the target, so the event
|
||||||
|
lands on the directory and a watch on the file is left holding an orphaned
|
||||||
|
inode. What a row shows is read from the **file**, never from the index —
|
||||||
|
metadata from `stat`, and for a content match the same MIME sniffing and
|
||||||
|
extractors the indexer uses, re-cut through the same `cascade::text_snippet`
|
||||||
|
the search itself does. That is what makes it work with indexing stopped.
|
||||||
|
Arming also sweeps each target once against the size and modified time the
|
||||||
|
row is displaying, which on a fresh result is what the index said: so
|
||||||
|
bringing a row on screen *is* a check of the index against the disk, and it
|
||||||
|
is the only thing that reports anything where the platform sends no events.
|
||||||
|
It still writes nothing itself; the paths it has just read go to
|
||||||
|
`IndexCoordinator::update_paths`, which applies them on the coordinator's
|
||||||
|
own thread — in any mode, so a stopped index does not drift from the screen
|
||||||
|
— leaving the single-writer rule intact. Caps at 64 directories and 256
|
||||||
|
rows, rate-limited per path, and dropped wholesale the moment the query is
|
||||||
|
edited.
|
||||||
- **Search** (`search/`): `SearchService` runs one worker thread; each
|
- **Search** (`search/`): `SearchService` runs one worker thread; each
|
||||||
query is a *generation*. New queries interrupt the in-flight SQLite
|
query is a *generation*. New queries interrupt the in-flight SQLite
|
||||||
statement (`InterruptHandle`) and stale generations stop cooperatively,
|
statement (`InterruptHandle`) and stale generations stop cooperatively,
|
||||||
|
|
@ -598,6 +682,21 @@ Synchronous Rust: `std::thread` + `mpsc` channels, no async runtime.
|
||||||
case-insensitive literal branch used to allocate a lowercased copy of its
|
case-insensitive literal branch used to allocate a lowercased copy of its
|
||||||
haystack, which the filename pass asked for twice per row of a full-table
|
haystack, which the filename pass asked for twice per row of a full-table
|
||||||
scan.
|
scan.
|
||||||
|
- **Duplicate verification** (`verify.rs`): the second opinion on a group from
|
||||||
|
`search/duplicates.rs`, which groups by `sha256(size ‖ head)` and so cannot
|
||||||
|
tell two pre-allocated disk images apart — same size, same zeroes at the
|
||||||
|
front, everything that distinguishes them in a footer. One lockstep pass:
|
||||||
|
open every member, drop the ones whose length already disagrees without
|
||||||
|
reading them, then read a chunk from the first that opened and the same
|
||||||
|
span from each of the others, reporting the offset of the first byte that
|
||||||
|
differs and dropping that file from the walk. Deliberately not a hash —
|
||||||
|
"the same digest" is a probabilistic answer, and a probabilistic answer is
|
||||||
|
what the head hash already gave. The reference is the first member that
|
||||||
|
*opens*, so one unreadable file costs its own verdict and nobody else's,
|
||||||
|
and termination follows what that file actually reads rather than the
|
||||||
|
length it claimed, so a file truncated mid-run degrades to a short
|
||||||
|
comparison. The read buffers share a fixed 8 MiB between them however many
|
||||||
|
members a group has, because a hardlink farm's group runs to thousands.
|
||||||
- **Baloo compatibility** (`cli.rs`, `mime.rs`): the read API this repo's
|
- **Baloo compatibility** (`cli.rs`, `mime.rs`): the read API this repo's
|
||||||
parent consumes — `status_for_path`, `list_failed`,
|
parent consumes — `status_for_path`, `list_failed`,
|
||||||
`index_size_breakdown`, `pending_content_count`, `clear_path` — plus a
|
`index_size_breakdown`, `pending_content_count`, `clear_path` — plus a
|
||||||
|
|
@ -630,14 +729,17 @@ core threads ─────────────▶ ctx.request_repaint() (w
|
||||||
```
|
```
|
||||||
|
|
||||||
Modules map one-to-one onto what you see: `app.rs` (shell and config
|
Modules map one-to-one onto what you see: `app.rs` (shell and config
|
||||||
routing, with `app/` submodules for the status bar, the security flow and
|
routing, with `app/` submodules for the status bar, the security flow, the
|
||||||
the confirmation modals), `search_tab.rs` (query strip and virtualized
|
confirmation modals and the duplicate-verification modal — the one place a
|
||||||
|
worker's progress is shown in a window rather than the status bar),
|
||||||
|
`search_tab.rs` (query strip and virtualized
|
||||||
results table; snippet rendering via `LayoutJob` byte ranges, the ignore
|
results table; snippet rendering via `LayoutJob` byte ranges, the ignore
|
||||||
dialog and the syntax help live in `search_tab/`), `manage_tab.rs` (status
|
dialog and the syntax help live in `search_tab/`), `manage_tab.rs` (status
|
||||||
detail + `tracker.rs` rate estimation, roots and filter editors),
|
detail + `tracker.rs` rate estimation, roots and filter editors),
|
||||||
`duplicates_tab.rs`, `logs_tab.rs` (a virtualized view of the core log
|
`duplicates_tab.rs`, `logs_tab.rs` (a virtualized view of the core log
|
||||||
ring), `options.rs` (draft-based settings editor shared between the window
|
ring), `settings_tab.rs` (the draft-based config editor, the second of the
|
||||||
and the Manage tab), `platform.rs` (open / reveal-in-file-manager, and the
|
two tabs that stage their edits behind an Apply & Save), `platform.rs`
|
||||||
|
(open / reveal-in-file-manager, and the
|
||||||
Windows stdio setup a window-subsystem process needs before anything
|
Windows stdio setup a window-subsystem process needs before anything
|
||||||
prints), `hotkey/` (the system-wide search shortcut: one key table feeding
|
prints), `hotkey/` (the system-wide search shortcut: one key table feeding
|
||||||
both a `RegisterHotKey` / `XGrabKey` registration and, on Wayland, an XDG
|
both a `RegisterHotKey` / `XGrabKey` registration and, on Wayland, an XDG
|
||||||
|
|
@ -650,12 +752,26 @@ microseconds regardless of row count.
|
||||||
|
|
||||||
- `cargo test -p quicksearch-core`: unit + integration suites (cascade
|
- `cargo test -p quicksearch-core`: unit + integration suites (cascade
|
||||||
ranking, cancellation, incremental indexing, coordinator modes, config
|
ranking, cancellation, incremental indexing, coordinator modes, config
|
||||||
resolution, fuzzy matcher vs. brute-force oracle).
|
resolution, fuzzy matcher vs. brute-force oracle, `verify.rs`'s byte-for-byte
|
||||||
|
comparison — the shared-head-different-tail case the head hash cannot see, an
|
||||||
|
unreadable first member, a difference past the first chunk, cancellation —
|
||||||
|
and `live.rs`'s event
|
||||||
|
classification, where the platform-specific rename and atomic-save shapes are
|
||||||
|
synthesized rather than provoked, so they are checked on every platform).
|
||||||
- `cargo test -p quicksearch-gui`: formatter/tracker/CLI-parsing units plus
|
- `cargo test -p quicksearch-gui`: formatter/tracker/CLI-parsing units plus
|
||||||
headless egui tests that drive the real widgets — building an input frame,
|
headless egui tests that drive the real widgets — building an input frame,
|
||||||
synthesizing clicks and reading back the painted text (`test_ui.rs`) — over
|
synthesizing clicks and reading back the painted text (`test_ui.rs`) — over
|
||||||
the search and manage tabs, the options editor, the unlock gate, the logs
|
the search, manage and settings tabs, the unlock gate, the logs
|
||||||
and duplicates tabs, and query highlighting.
|
and duplicates tabs, the first-start tour, and query highlighting. The search
|
||||||
|
tab's cover the column picker (including that the path column survives all
|
||||||
|
32 combinations of the others), which column a match is highlighted in, and
|
||||||
|
that the repeat-search button appearing inside the query box does not cost it
|
||||||
|
keyboard focus. The duplicates tab's open the real context menus and click
|
||||||
|
the entries inside them, so "the verification asks for the whole group, from
|
||||||
|
either menu, and not at all while one is running" is checked rather than
|
||||||
|
assumed; the verification modal is rendered in each of its states, and the
|
||||||
|
tour's footer is probed for where its three buttons actually landed rather
|
||||||
|
than for the numbers they were expected to land on.
|
||||||
- `cargo bench -p quicksearch-core --bench search` and `--bench index`: divan
|
- `cargo bench -p quicksearch-core --bench search` and `--bench index`: divan
|
||||||
microbenchmarks over the two hot paths. Each group runs *what the code does
|
microbenchmarks over the two hot paths. Each group runs *what the code does
|
||||||
today* against *the change being considered*, in one process on one corpus,
|
today* against *the change being considered*, in one process on one corpus,
|
||||||
|
|
|
||||||
|
|
@ -133,7 +133,7 @@ store_text_for_snippets = true
|
||||||
[security]
|
[security]
|
||||||
# Encrypt the index with a password (SQLCipher). The password is asked
|
# Encrypt the index with a password (SQLCipher). The password is asked
|
||||||
# for every time QuickSearch starts; turning this on or off deletes and
|
# for every time QuickSearch starts; turning this on or off deletes and
|
||||||
# rebuilds the index. Change it from the GUI (Options → Security), not by
|
# rebuilds the index. Change it from the GUI (Settings → Security), not by
|
||||||
# hand: enabling protection also generates the KDF salt below.
|
# hand: enabling protection also generates the KDF salt below.
|
||||||
password_protected = false
|
password_protected = false
|
||||||
# Store the derived key in the OS keychain (Secret Service / KWallet on
|
# Store the derived key in the OS keychain (Secret Service / KWallet on
|
||||||
|
|
@ -163,12 +163,18 @@ watch_cap_warned_roots = []
|
||||||
# registered with your desktop, which may assign a different key and lets
|
# registered with your desktop, which may assign a different key and lets
|
||||||
# you change it in its own keyboard settings.
|
# you change it in its own keyboard settings.
|
||||||
search_hotkey = "Ctrl+Shift+F"
|
search_hotkey = "Ctrl+Shift+F"
|
||||||
# 'dark' or 'light'. Applied as soon as it is changed in the Options
|
# 'dark' or 'light'. Applied as soon as it is changed on the Settings
|
||||||
# window. Your desktop's own light/dark setting is not consulted: reading
|
# tab. Your desktop's own light/dark setting is not consulted: reading
|
||||||
# it would mean connecting to your session's message bus and subscribing to
|
# it would mean connecting to your session's message bus and subscribing to
|
||||||
# your settings, which is more than a search tool should ask for. Anything
|
# your settings, which is more than a search tool should ask for. Anything
|
||||||
# other than 'light' is dark.
|
# other than 'light' is dark.
|
||||||
color_scheme = "dark"
|
color_scheme = "dark"
|
||||||
|
# Written by QuickSearch, not by you: whether the short introduction shown
|
||||||
|
# on a brand-new installation has been dismissed. Absent means this config
|
||||||
|
# predates that introduction - an installation that upgraded into this
|
||||||
|
# version, which is not offered it. The Help tab can show it again at any
|
||||||
|
# time.
|
||||||
|
tutorial_seen = false
|
||||||
|
|
||||||
[search]
|
[search]
|
||||||
# Start with the fuzzy passes enabled.
|
# Start with the fuzzy passes enabled.
|
||||||
|
|
@ -185,3 +191,32 @@ display_limit = 1000
|
||||||
results_per_page = 100
|
results_per_page = 100
|
||||||
# How long the GUI waits after the last keystroke before searching (ms).
|
# How long the GUI waits after the last keystroke before searching (ms).
|
||||||
debounce_ms = 150
|
debounce_ms = 150
|
||||||
|
# Watch the search results on screen and show renames, deletions and
|
||||||
|
# content changes as they happen. Only the rows actually visible are
|
||||||
|
# watched, and editing the query drops the watches. What a row shows is
|
||||||
|
# read from the file itself, so this works whether or not indexing is
|
||||||
|
# running — and the files it reads are then brought up to date in the
|
||||||
|
# index, so what is stored cannot drift from what you are looking at.
|
||||||
|
# Rows are also checked against the disk as they come on screen, which is
|
||||||
|
# all you get over a network share, where the system does not report other
|
||||||
|
# machines' writes. Nothing is ever added, removed or re-ordered while you
|
||||||
|
# read.
|
||||||
|
live_results = true
|
||||||
|
|
||||||
|
# Which columns the Search tab shows. The same choices are on the
|
||||||
|
# right-click menu of any column header, and in Settings → Search; both
|
||||||
|
# write here immediately, without an Apply.
|
||||||
|
#
|
||||||
|
# There is deliberately no 'path' key: the path is always shown, because
|
||||||
|
# it is the only column that identifies a result on its own.
|
||||||
|
[search.columns]
|
||||||
|
name = true
|
||||||
|
# The excerpt of a file's contents around the match. Rows that matched on
|
||||||
|
# their name or path show a dash there instead.
|
||||||
|
content_match = true
|
||||||
|
# Off by default: the width these take is usually better spent on the path
|
||||||
|
# and the matched text. Turning one on also makes it available to sort by;
|
||||||
|
# sorting by a column that is hidden falls back to sorting by rank.
|
||||||
|
size = false
|
||||||
|
modified = false
|
||||||
|
rank = true
|
||||||
|
|
|
||||||
|
|
@ -122,7 +122,6 @@ fn parallel(root: &str, config: &Config, db_path: &str) -> (usize, usize) {
|
||||||
config.clone(),
|
config.clone(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
) {
|
) {
|
||||||
let WalkEvent::File(file) = event else {
|
let WalkEvent::File(file) = event else {
|
||||||
|
|
|
||||||
|
|
@ -169,6 +169,47 @@ pub struct SearchConfig {
|
||||||
pub results_per_page: usize,
|
pub results_per_page: usize,
|
||||||
/// How long the GUI waits after the last keystroke before searching.
|
/// How long the GUI waits after the last keystroke before searching.
|
||||||
pub debounce_ms: u64,
|
pub debounce_ms: u64,
|
||||||
|
/// Watch the search results currently on screen and show renames,
|
||||||
|
/// deletions and content changes as they happen. Only the rows actually
|
||||||
|
/// visible are watched, and any edit to the query drops the watches.
|
||||||
|
/// What a row shows is read from the file itself, so this holds whether
|
||||||
|
/// or not indexing is running; the files it reads are then brought up to
|
||||||
|
/// date in the index, so what is stored cannot drift from what is on
|
||||||
|
/// screen. See [`crate::live`].
|
||||||
|
pub live_results: bool,
|
||||||
|
/// Which columns the Search tab shows.
|
||||||
|
pub columns: ColumnsConfig,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Which columns the Search tab shows, as picked from the right-click menu on
|
||||||
|
/// any column header or from the Settings tab.
|
||||||
|
///
|
||||||
|
/// The path column is deliberately not represented: it is always shown, so
|
||||||
|
/// "no columns at all" is not a state this can hold. Size and modified are off
|
||||||
|
/// by default — the width they cost is better spent on the path and the match,
|
||||||
|
/// and both are one click away.
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||||
|
#[serde(default)]
|
||||||
|
pub struct ColumnsConfig {
|
||||||
|
pub name: bool,
|
||||||
|
/// The matched excerpt from a file's contents. Rows that matched on their
|
||||||
|
/// name or path instead show a dash there.
|
||||||
|
pub content_match: bool,
|
||||||
|
pub size: bool,
|
||||||
|
pub modified: bool,
|
||||||
|
pub rank: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for ColumnsConfig {
|
||||||
|
fn default() -> Self {
|
||||||
|
ColumnsConfig {
|
||||||
|
name: true,
|
||||||
|
content_match: true,
|
||||||
|
size: false,
|
||||||
|
modified: false,
|
||||||
|
rank: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
impl SearchConfig {
|
impl SearchConfig {
|
||||||
|
|
@ -233,6 +274,8 @@ impl Default for SearchConfig {
|
||||||
display_limit: 1000,
|
display_limit: 1000,
|
||||||
results_per_page: 100,
|
results_per_page: 100,
|
||||||
debounce_ms: 150,
|
debounce_ms: 150,
|
||||||
|
live_results: true,
|
||||||
|
columns: ColumnsConfig::default(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -302,6 +345,21 @@ pub struct UiConfig {
|
||||||
/// recognises falls back to dark, where a typed-out enum would fail to
|
/// recognises falls back to dark, where a typed-out enum would fail to
|
||||||
/// deserialize and take the whole config file down with it.
|
/// deserialize and take the whole config file down with it.
|
||||||
pub color_scheme: String,
|
pub color_scheme: String,
|
||||||
|
/// Whether the first-start tour has been dismissed.
|
||||||
|
///
|
||||||
|
/// Three-valued on purpose. `None` means the key predates the tour — an
|
||||||
|
/// installation that upgraded into this version, which has already found
|
||||||
|
/// its way around — so only a config file this version *created* (which
|
||||||
|
/// gets `Some(false)` from [`UiConfig::default`]) is ever offered the tour.
|
||||||
|
/// A plain `bool` could not tell those apart.
|
||||||
|
///
|
||||||
|
/// The field-level `default` is load-bearing and not redundant with the
|
||||||
|
/// `#[serde(default)]` on the struct: that one fills a missing field from
|
||||||
|
/// `UiConfig::default()`, which says `Some(false)` — and would hand every
|
||||||
|
/// upgrading installation the tour. This one fills it from
|
||||||
|
/// `Option::default()`, which is `None`.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub tutorial_seen: Option<bool>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Default for UiConfig {
|
impl Default for UiConfig {
|
||||||
|
|
@ -311,6 +369,10 @@ impl Default for UiConfig {
|
||||||
watch_cap_warned_roots: Vec::new(),
|
watch_cap_warned_roots: Vec::new(),
|
||||||
search_hotkey: "Ctrl+Shift+F".to_string(),
|
search_hotkey: "Ctrl+Shift+F".to_string(),
|
||||||
color_scheme: "dark".to_string(),
|
color_scheme: "dark".to_string(),
|
||||||
|
// Not `None`: a config built from these defaults is a config being
|
||||||
|
// written for the first time, and that is exactly who the tour is
|
||||||
|
// for. `None` is reserved for a file that predates the key.
|
||||||
|
tutorial_seen: Some(false),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -985,3 +985,61 @@ fn fuzzy_edits_warning_only_above_the_threshold() {
|
||||||
assert!(msg.contains(&FUZZY_EDITS_WARN_ABOVE.to_string()));
|
assert!(msg.contains(&FUZZY_EDITS_WARN_ABOVE.to_string()));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `config_example.toml` is the documentation for every setting, so a key
|
||||||
|
/// renamed in the struct and not here would silently ship a config file that
|
||||||
|
/// does nothing. Parsing it also proves the `[search.columns]` sub-table is
|
||||||
|
/// spelled the way serde expects.
|
||||||
|
#[test]
|
||||||
|
fn the_documented_example_config_parses_to_the_defaults() {
|
||||||
|
let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||||
|
.join("../../config_example.toml")
|
||||||
|
.canonicalize()
|
||||||
|
.expect("config_example.toml sits at the repository root");
|
||||||
|
let text = std::fs::read_to_string(&path).expect("readable");
|
||||||
|
let parsed: Config = toml::from_str(&text).expect("config_example.toml parses");
|
||||||
|
|
||||||
|
// The example documents the shipped defaults for everything that has one
|
||||||
|
// that does not depend on the machine (paths and the hotkey do).
|
||||||
|
let d = Config::default();
|
||||||
|
assert_eq!(
|
||||||
|
parsed.search, d.search,
|
||||||
|
"[search] drifted from the defaults"
|
||||||
|
);
|
||||||
|
assert_eq!(parsed.processing, d.processing);
|
||||||
|
assert_eq!(parsed.ui.scale, d.ui.scale);
|
||||||
|
assert_eq!(parsed.ui.color_scheme, d.ui.color_scheme);
|
||||||
|
assert_eq!(parsed.ui.tutorial_seen, Some(false));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The tour is offered to an installation this version created, and to no
|
||||||
|
/// other. A config written before the key existed reads as `None`, which is
|
||||||
|
/// how "already found their way around" is distinguished from "brand new".
|
||||||
|
#[test]
|
||||||
|
fn only_a_freshly_written_config_asks_for_the_tour() {
|
||||||
|
assert_eq!(UiConfig::default().tutorial_seen, Some(false));
|
||||||
|
|
||||||
|
let older: Config = toml::from_str("[ui]\nscale = 1.1\n").expect("parses");
|
||||||
|
assert_eq!(
|
||||||
|
older.ui.tutorial_seen, None,
|
||||||
|
"a config predating the tour must not be offered it"
|
||||||
|
);
|
||||||
|
|
||||||
|
let dismissed: Config = toml::from_str("[ui]\ntutorial_seen = true\n").expect("parses");
|
||||||
|
assert_eq!(dismissed.ui.tutorial_seen, Some(true));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Size and modified cost more width than they earn for most searches.
|
||||||
|
#[test]
|
||||||
|
fn the_search_table_ships_without_size_or_modified() {
|
||||||
|
let cols = ColumnsConfig::default();
|
||||||
|
assert!(cols.name && cols.content_match && cols.rank);
|
||||||
|
assert!(!cols.size, "the size column is on by default");
|
||||||
|
assert!(!cols.modified, "the modified column is on by default");
|
||||||
|
|
||||||
|
// A `[search]` block written before the picker existed still gets them.
|
||||||
|
let older: Config = toml::from_str("[search]\ndisplay_limit = 500\n").expect("parses");
|
||||||
|
assert_eq!(older.search.columns, cols);
|
||||||
|
assert_eq!(older.search.display_limit, 500);
|
||||||
|
assert!(older.search.live_results, "live results default to on");
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,22 +1,24 @@
|
||||||
//! Parallel content extraction for one indexing root.
|
//! Parallel content extraction for one indexing root.
|
||||||
//!
|
//!
|
||||||
//! The second half of a root's pipeline, and the sibling of [`crate::walk`]:
|
//! The second half of a root's pipeline, and the sibling of [`crate::walk`]:
|
||||||
//! a pool of worker threads produces finished work over a bounded channel, and
|
//! a pool of worker threads produces finished work over a bounded channel,
|
||||||
//! the single writer drains it round-robin against every other root.
|
//! and the single writer drains it in time-bounded turns. Walking roots are
|
||||||
|
//! served first and only one extracting root per round, so a pass that is
|
||||||
|
//! producing faster than the writer can tokenize waits rather than holding up
|
||||||
|
//! anyone's walk — see `indexing::pipeline`.
|
||||||
//!
|
//!
|
||||||
//! **One feeder thread owns the only database connection**, paging through
|
//! **One feeder thread owns the only database connection**, paging through
|
||||||
//! the root's pending rows, while N workers do nothing but filesystem work. A
|
//! the root's pending rows, while N workers do nothing but filesystem work. A
|
||||||
//! connection per worker would multiply SQLite's page cache by the pool size
|
//! connection per worker would multiply SQLite's page cache by the pool size
|
||||||
//! (see [`crate::db::schema::PRAGMAS_WALK_READER`]).
|
//! (see [`crate::db::schema::PRAGMAS_WALK_READER`]).
|
||||||
|
|
||||||
use std::sync::atomic::AtomicBool;
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
use std::sync::{mpsc, Arc, Condvar, Mutex};
|
use std::sync::{mpsc, Arc, Condvar, Mutex};
|
||||||
use std::thread::JoinHandle;
|
use std::thread::JoinHandle;
|
||||||
|
|
||||||
use crate::config::Config;
|
use crate::config::Config;
|
||||||
use crate::extract::Registry;
|
use crate::extract::Registry;
|
||||||
use crate::file_handling::{decide_content, ContentOutcome, ExtractCursor};
|
use crate::file_handling::{decide_content, ContentOutcome, ExtractCursor, ExtractScope};
|
||||||
use crate::indexing::should_abort;
|
|
||||||
use crate::walk::{try_recv_next, TryNext, WorkerStats};
|
use crate::walk::{try_recv_next, TryNext, WorkerStats};
|
||||||
|
|
||||||
/// Finished rows waiting for the writer.
|
/// Finished rows waiting for the writer.
|
||||||
|
|
@ -69,6 +71,11 @@ struct Queue {
|
||||||
struct Shared {
|
struct Shared {
|
||||||
queue: Mutex<Queue>,
|
queue: Mutex<Queue>,
|
||||||
idle: Condvar,
|
idle: Condvar,
|
||||||
|
/// What the range held when the pass began: rows still to extract and
|
||||||
|
/// rows already done. Set by the feeder before it pages anything, so
|
||||||
|
/// `already_done + rows written this pass` stays exact; never set if the
|
||||||
|
/// feeder could not count.
|
||||||
|
totals: std::sync::OnceLock<ExtractScope>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Shared {
|
impl Shared {
|
||||||
|
|
@ -163,6 +170,16 @@ impl ContentPass {
|
||||||
self.stats.clone()
|
self.stats.clone()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The range's pending and already-done counts as they stood when the
|
||||||
|
/// pass began.
|
||||||
|
///
|
||||||
|
/// `None` until the feeder has counted — a scan that takes seconds on a
|
||||||
|
/// large root, which is why it happens here on the pass's own connection
|
||||||
|
/// and not on the indexer's writer — and forever if it could not.
|
||||||
|
pub fn totals(&self) -> Option<ExtractScope> {
|
||||||
|
self.shared.totals.get().copied()
|
||||||
|
}
|
||||||
|
|
||||||
/// Join the workers and report whether every one finished cleanly.
|
/// Join the workers and report whether every one finished cleanly.
|
||||||
/// See [`crate::walk::ParallelWalk::finish`].
|
/// See [`crate::walk::ParallelWalk::finish`].
|
||||||
pub fn finish(&mut self) -> bool {
|
pub fn finish(&mut self) -> bool {
|
||||||
|
|
@ -197,7 +214,7 @@ impl Drop for ContentPass {
|
||||||
/// A failed query ends the pass rather than retrying: the rows stay
|
/// A failed query ends the pass rather than retrying: the rows stay
|
||||||
/// `content_state = 0` and the next run picks them up, which is the same
|
/// `content_state = 0` and the next run picks them up, which is the same
|
||||||
/// outcome as being interrupted.
|
/// outcome as being interrupted.
|
||||||
fn feeder(shared: &Shared, db_path: &str, mut cursor: ExtractCursor, max_size: i64) {
|
fn feeder(shared: &Shared, db_path: &str, mut cursor: ExtractCursor, config: &Config) {
|
||||||
let conn = match crate::db::open::open_walk_reader(db_path) {
|
let conn = match crate::db::open::open_walk_reader(db_path) {
|
||||||
Ok(conn) => conn,
|
Ok(conn) => conn,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
|
|
@ -207,6 +224,18 @@ fn feeder(shared: &Shared, db_path: &str, mut cursor: ExtractCursor, max_size: i
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Before the first page, so nothing this pass writes is inside the count.
|
||||||
|
// The workers cannot run ahead of this: they block in `take` until the
|
||||||
|
// first page lands. A failure here costs the progress figure, not the
|
||||||
|
// pass.
|
||||||
|
match crate::file_handling::count_extract_scope(&conn, &cursor, config) {
|
||||||
|
Ok(totals) => {
|
||||||
|
let _ = shared.totals.set(totals);
|
||||||
|
}
|
||||||
|
Err(e) => crate::log_warn!("content reader: {}", e),
|
||||||
|
}
|
||||||
|
|
||||||
|
let max_size = crate::file_handling::max_text_file_size(config);
|
||||||
while shared.take_feed_slot().is_some() {
|
while shared.take_feed_slot().is_some() {
|
||||||
let page =
|
let page =
|
||||||
match crate::db::repo::pending_content_page(&conn, &cursor, max_size, FEED_PAGE as i64)
|
match crate::db::repo::pending_content_page(&conn, &cursor, max_size, FEED_PAGE as i64)
|
||||||
|
|
@ -244,14 +273,13 @@ fn worker(
|
||||||
registry: &Registry,
|
registry: &Registry,
|
||||||
config: &Config,
|
config: &Config,
|
||||||
stop_flag: &Arc<AtomicBool>,
|
stop_flag: &Arc<AtomicBool>,
|
||||||
suspend_flag: &Arc<AtomicBool>,
|
|
||||||
stats: &WorkerStats,
|
stats: &WorkerStats,
|
||||||
) {
|
) {
|
||||||
while let Some(row) = shared.take() {
|
while let Some(row) = shared.take() {
|
||||||
// Held for the whole of `decide_content`; that is the work the
|
// Held for the whole of `decide_content`; that is the work the
|
||||||
// progress line reports.
|
// progress line reports.
|
||||||
let _busy = stats.enter();
|
let _busy = stats.enter();
|
||||||
if should_abort(stop_flag, suspend_flag) {
|
if stop_flag.load(Ordering::Relaxed) {
|
||||||
shared.shutdown();
|
shared.shutdown();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -279,14 +307,13 @@ pub fn extract_content(
|
||||||
registry: Arc<Registry>,
|
registry: Arc<Registry>,
|
||||||
config: Config,
|
config: Config,
|
||||||
stop_flag: Arc<AtomicBool>,
|
stop_flag: Arc<AtomicBool>,
|
||||||
suspend_flag: Arc<AtomicBool>,
|
|
||||||
workers: usize,
|
workers: usize,
|
||||||
) -> ContentPass {
|
) -> ContentPass {
|
||||||
let shared = Arc::new(Shared {
|
let shared = Arc::new(Shared {
|
||||||
queue: Mutex::new(Queue::default()),
|
queue: Mutex::new(Queue::default()),
|
||||||
idle: Condvar::new(),
|
idle: Condvar::new(),
|
||||||
|
totals: std::sync::OnceLock::new(),
|
||||||
});
|
});
|
||||||
let max_size = i64::try_from(config.processing.maximum_text_file_size).unwrap_or(i64::MAX);
|
|
||||||
|
|
||||||
let (tx, rx) = mpsc::sync_channel(READY_CAP);
|
let (tx, rx) = mpsc::sync_channel(READY_CAP);
|
||||||
let stats = WorkerStats::new(workers.clamp(1, 64));
|
let stats = WorkerStats::new(workers.clamp(1, 64));
|
||||||
|
|
@ -294,19 +321,11 @@ pub fn extract_content(
|
||||||
.map(|_| {
|
.map(|_| {
|
||||||
let (shared, tx) = (shared.clone(), tx.clone());
|
let (shared, tx) = (shared.clone(), tx.clone());
|
||||||
let (registry, config) = (registry.clone(), config.clone());
|
let (registry, config) = (registry.clone(), config.clone());
|
||||||
let (stop_flag, suspend_flag) = (stop_flag.clone(), suspend_flag.clone());
|
let stop_flag = stop_flag.clone();
|
||||||
let stats = stats.clone();
|
let stats = stats.clone();
|
||||||
crate::platform::spawn_worker("qs-extract", move || {
|
crate::platform::spawn_worker("qs-extract", move || {
|
||||||
crate::platform::set_background_priority();
|
crate::platform::set_background_priority();
|
||||||
worker(
|
worker(&shared, &tx, ®istry, &config, &stop_flag, &stats)
|
||||||
&shared,
|
|
||||||
&tx,
|
|
||||||
®istry,
|
|
||||||
&config,
|
|
||||||
&stop_flag,
|
|
||||||
&suspend_flag,
|
|
||||||
&stats,
|
|
||||||
)
|
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
|
|
@ -318,7 +337,7 @@ pub fn extract_content(
|
||||||
let (shared, db_path, cursor) = (shared.clone(), db_path.to_string(), cursor.clone());
|
let (shared, db_path, cursor) = (shared.clone(), db_path.to_string(), cursor.clone());
|
||||||
crate::platform::spawn_worker("qs-feeder", move || {
|
crate::platform::spawn_worker("qs-feeder", move || {
|
||||||
crate::platform::set_background_priority();
|
crate::platform::set_background_priority();
|
||||||
feeder(&shared, &db_path, cursor, max_size)
|
feeder(&shared, &db_path, cursor, &config)
|
||||||
})
|
})
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -338,9 +357,23 @@ mod tests {
|
||||||
|
|
||||||
use crate::db::open_or_recreate;
|
use crate::db::open_or_recreate;
|
||||||
use crate::db::repo::{self, insert_file, NewFile};
|
use crate::db::repo::{self, insert_file, NewFile};
|
||||||
use crate::file_handling::{extract_scope_prepare, store_extracted};
|
use crate::file_handling::{store_extracted, ExtractScope, Stored};
|
||||||
use crate::mime::FileType;
|
use crate::mime::FileType;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
/// The removed `extract_scope_prepare`: the sweep on the writer, then the
|
||||||
|
/// count the content pass now does on its own connection. Composed here
|
||||||
|
/// because these tests want both halves in one call.
|
||||||
|
fn extract_scope_prepare(
|
||||||
|
conn_mutex: &Arc<Mutex<rusqlite::Connection>>,
|
||||||
|
cursor: &ExtractCursor,
|
||||||
|
config: &Config,
|
||||||
|
) -> Result<ExtractScope, String> {
|
||||||
|
let conn = crate::lock_ok(conn_mutex);
|
||||||
|
crate::file_handling::mark_oversize_pending_na(&conn, cursor, config)?;
|
||||||
|
crate::file_handling::count_extract_scope(&conn, cursor, config)
|
||||||
|
}
|
||||||
/// A path that does not exist yet — the caller builds the tree under it.
|
/// A path that does not exist yet — the caller builds the tree under it.
|
||||||
fn tmp(tag: &str) -> PathBuf {
|
fn tmp(tag: &str) -> PathBuf {
|
||||||
crate::testutil::scratch_dir(tag).join("tree")
|
crate::testutil::scratch_dir(tag).join("tree")
|
||||||
|
|
@ -391,7 +424,6 @@ mod tests {
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
workers,
|
workers,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
@ -450,9 +482,13 @@ mod tests {
|
||||||
assert_eq!(rows.len(), 3);
|
assert_eq!(rows.len(), 3);
|
||||||
|
|
||||||
let stop = Arc::new(AtomicBool::new(false));
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
|
let far = Instant::now() + Duration::from_secs(60);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
store_extracted(&conn_mutex, &rows, &stop, &config).unwrap(),
|
store_extracted(&conn_mutex, &rows, &stop, &config, far).unwrap(),
|
||||||
3
|
Stored {
|
||||||
|
consumed: 3,
|
||||||
|
written: 3
|
||||||
|
}
|
||||||
);
|
);
|
||||||
|
|
||||||
let state = |p: &Path| -> i64 {
|
let state = |p: &Path| -> i64 {
|
||||||
|
|
@ -501,6 +537,93 @@ mod tests {
|
||||||
let mut pass = pass_for(&tree, &db, "nonexistent", 4);
|
let mut pass = pass_for(&tree, &db, "nonexistent", 4);
|
||||||
assert!(drain(&mut pass).is_empty());
|
assert!(drain(&mut pass).is_empty());
|
||||||
assert!(pass.finish());
|
assert!(pass.finish());
|
||||||
|
// The pass still counted: an empty range is a known zero, not an
|
||||||
|
// unknown.
|
||||||
|
assert_eq!(
|
||||||
|
pass.totals(),
|
||||||
|
Some(ExtractScope {
|
||||||
|
pending: 0,
|
||||||
|
already_done: 0
|
||||||
|
})
|
||||||
|
);
|
||||||
|
std::fs::remove_dir_all(&tree).ok();
|
||||||
|
std::fs::remove_file(&db).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The pass counts its range on its own connection, before it pages —
|
||||||
|
/// which is what lets the writer thread stop doing it. The count is what
|
||||||
|
/// stood at the start: rows this pass writes are not inside it.
|
||||||
|
#[test]
|
||||||
|
fn the_pass_counts_its_range_before_it_starts() {
|
||||||
|
let (tree, db) = seed("totals", &[("r1", 3), ("r2", 2)]);
|
||||||
|
let mut pass = pass_for(&tree, &db, "r1", 2);
|
||||||
|
let rows = drain(&mut pass);
|
||||||
|
assert!(pass.finish());
|
||||||
|
assert_eq!(rows.len(), 3);
|
||||||
|
assert_eq!(
|
||||||
|
pass.totals(),
|
||||||
|
Some(ExtractScope {
|
||||||
|
pending: 3,
|
||||||
|
already_done: 0
|
||||||
|
}),
|
||||||
|
"only r1's rows, all of them pending when the pass began"
|
||||||
|
);
|
||||||
|
std::fs::remove_dir_all(&tree).ok();
|
||||||
|
std::fs::remove_file(&db).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The writer's turn is bounded by time, not by batch: `store_extracted`
|
||||||
|
/// stops at its deadline, tells the caller how far it got, and always
|
||||||
|
/// gets at least one row down so a caller looping on it cannot spin.
|
||||||
|
#[test]
|
||||||
|
fn store_extracted_honours_its_deadline_but_always_makes_progress() {
|
||||||
|
let (tree, db) = seed("deadline", &[("r1", 5)]);
|
||||||
|
let conn_mutex = Arc::new(Mutex::new(
|
||||||
|
open_or_recreate(db.to_str().unwrap(), "trigram").unwrap(),
|
||||||
|
));
|
||||||
|
let config = Config::default();
|
||||||
|
let mut pass = pass_for(&tree, &db, "r1", 2);
|
||||||
|
let rows = drain(&mut pass);
|
||||||
|
assert!(pass.finish());
|
||||||
|
assert_eq!(rows.len(), 5);
|
||||||
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
|
|
||||||
|
// A deadline already gone by: one row, then out.
|
||||||
|
let past = Instant::now() - Duration::from_secs(1);
|
||||||
|
assert_eq!(
|
||||||
|
store_extracted(&conn_mutex, &rows, &stop, &config, past).unwrap(),
|
||||||
|
Stored {
|
||||||
|
consumed: 1,
|
||||||
|
written: 1
|
||||||
|
}
|
||||||
|
);
|
||||||
|
// Plenty of time: the rest, in one call.
|
||||||
|
let far = Instant::now() + Duration::from_secs(60);
|
||||||
|
assert_eq!(
|
||||||
|
store_extracted(&conn_mutex, &rows[1..], &stop, &config, far).unwrap(),
|
||||||
|
Stored {
|
||||||
|
consumed: 4,
|
||||||
|
written: 4
|
||||||
|
}
|
||||||
|
);
|
||||||
|
let done: i64 = conn_mutex
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM files WHERE content_state = 1",
|
||||||
|
[],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(done, 5, "every row landed across the two calls");
|
||||||
|
|
||||||
|
// Stopped before it starts: nothing consumed, and the caller can tell.
|
||||||
|
stop.store(true, std::sync::atomic::Ordering::Relaxed);
|
||||||
|
assert_eq!(
|
||||||
|
store_extracted(&conn_mutex, &rows, &stop, &config, far).unwrap(),
|
||||||
|
Stored::default()
|
||||||
|
);
|
||||||
|
|
||||||
std::fs::remove_dir_all(&tree).ok();
|
std::fs::remove_dir_all(&tree).ok();
|
||||||
std::fs::remove_file(&db).ok();
|
std::fs::remove_file(&db).ok();
|
||||||
}
|
}
|
||||||
|
|
@ -514,7 +637,6 @@ mod tests {
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(AtomicBool::new(true)),
|
Arc::new(AtomicBool::new(true)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
);
|
);
|
||||||
assert!(drain(&mut pass).len() < 400);
|
assert!(drain(&mut pass).len() < 400);
|
||||||
|
|
|
||||||
|
|
@ -130,6 +130,7 @@ enum CoordCmd {
|
||||||
ConfigChanged(Config),
|
ConfigChanged(Config),
|
||||||
RebuildIndex,
|
RebuildIndex,
|
||||||
ClearIndex,
|
ClearIndex,
|
||||||
|
UpdatePaths(Vec<PathBuf>),
|
||||||
Shutdown,
|
Shutdown,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -228,6 +229,7 @@ impl IndexCoordinator {
|
||||||
watcher_rx: None,
|
watcher_rx: None,
|
||||||
watcher_gen: 0,
|
watcher_gen: 0,
|
||||||
pending: HashMap::new(),
|
pending: HashMap::new(),
|
||||||
|
targeted: HashMap::new(),
|
||||||
last_event_at: None,
|
last_event_at: None,
|
||||||
pending_since: None,
|
pending_since: None,
|
||||||
needs_full_run: false,
|
needs_full_run: false,
|
||||||
|
|
@ -303,6 +305,26 @@ impl IndexCoordinator {
|
||||||
let _ = self.cmd_tx.send(CoordCmd::ClearIndex);
|
let _ = self.cmd_tx.send(CoordCmd::ClearIndex);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Bring the index up to date for these paths and nothing else.
|
||||||
|
///
|
||||||
|
/// For [`crate::live`]: a frontend that has just read a displayed file
|
||||||
|
/// from disk hands the path here so the index agrees with what the user
|
||||||
|
/// is looking at. Deliberately **not** gated on [`IndexMode`] — the whole
|
||||||
|
/// point is that the rows on screen stay honest with indexing stopped —
|
||||||
|
/// but still applied on the coordinator's own thread, so the
|
||||||
|
/// single-writer rule holds and a full run is never raced.
|
||||||
|
///
|
||||||
|
/// Each path is re-read and rewritten only if its modified time has moved
|
||||||
|
/// (see [`crate::incremental::apply_fs_event`]), so submitting a path that
|
||||||
|
/// is already current costs a `stat` and a row lookup. A path that no
|
||||||
|
/// longer exists is removed from the index.
|
||||||
|
pub fn update_paths(&self, paths: Vec<PathBuf>) {
|
||||||
|
if paths.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let _ = self.cmd_tx.send(CoordCmd::UpdatePaths(paths));
|
||||||
|
}
|
||||||
|
|
||||||
/// Compare `config` against what the index was built with. Read-only.
|
/// Compare `config` against what the index was built with. Read-only.
|
||||||
pub fn check_config_validation(
|
pub fn check_config_validation(
|
||||||
&self,
|
&self,
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,12 @@ pub(super) struct Inner {
|
||||||
pub(super) watcher_rx: Option<mpsc::Receiver<(u64, Result<Watcher, WatchError>)>>,
|
pub(super) watcher_rx: Option<mpsc::Receiver<(u64, Result<Watcher, WatchError>)>>,
|
||||||
pub(super) watcher_gen: u64,
|
pub(super) watcher_gen: u64,
|
||||||
pub(super) pending: HashMap<PathBuf, FsEvent>,
|
pub(super) pending: HashMap<PathBuf, FsEvent>,
|
||||||
|
/// Paths a frontend asked for by name — see
|
||||||
|
/// [`IndexCoordinator::update_paths`]. Kept apart from [`Inner::pending`]
|
||||||
|
/// on purpose: this queue survives [`Inner::clear_pending`] and is applied
|
||||||
|
/// in manual mode, because it exists to keep the rows a user is *reading*
|
||||||
|
/// in step with the disk however the indexer is configured.
|
||||||
|
pub(super) targeted: HashMap<PathBuf, FsEvent>,
|
||||||
/// When the most recent event arrived; the burst is over once this is
|
/// When the most recent event arrived; the burst is over once this is
|
||||||
/// `pending_settle` old.
|
/// `pending_settle` old.
|
||||||
pub(super) last_event_at: Option<Instant>,
|
pub(super) last_event_at: Option<Instant>,
|
||||||
|
|
@ -158,6 +164,36 @@ impl Inner {
|
||||||
drop(shared);
|
drop(shared);
|
||||||
self.files_at = None;
|
self.files_at = None;
|
||||||
}
|
}
|
||||||
|
CoordCmd::UpdatePaths(paths) => {
|
||||||
|
// Only paths under an indexed root: the watcher never
|
||||||
|
// delivers anything else, so nothing downstream checks, and
|
||||||
|
// a file renamed *out* of every root would otherwise be
|
||||||
|
// written into the index at its new home. Roots in the same
|
||||||
|
// spelling `files.path` uses — the caller's paths are.
|
||||||
|
let prefixes: Vec<String> = self
|
||||||
|
.config
|
||||||
|
.normalized_indexing_paths()
|
||||||
|
.iter()
|
||||||
|
.map(|root| crate::file_handling::ExtractCursor::for_root(root).lo)
|
||||||
|
.collect();
|
||||||
|
for path in paths {
|
||||||
|
let spelled = path.to_string_lossy();
|
||||||
|
if !prefixes.iter().any(|lo| spelled.starts_with(lo.as_str())) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Existence decides the verb. The caller knows a file
|
||||||
|
// changed, not what it changed into, and a `Modify` for a
|
||||||
|
// path that is gone would be silently skipped rather than
|
||||||
|
// removing the row.
|
||||||
|
let event = if path.is_file() {
|
||||||
|
FsEvent::Modify(path)
|
||||||
|
} else {
|
||||||
|
FsEvent::Remove(path)
|
||||||
|
};
|
||||||
|
enqueue(&mut self.targeted, event);
|
||||||
|
}
|
||||||
|
self.was_busy = true;
|
||||||
|
}
|
||||||
CoordCmd::Shutdown => unreachable!("handled in run()"),
|
CoordCmd::Shutdown => unreachable!("handled in run()"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -197,6 +233,14 @@ impl Inner {
|
||||||
|
|
||||||
self.refresh_file_count();
|
self.refresh_file_count();
|
||||||
|
|
||||||
|
// Ahead of both the reconcile and the mode gate, and ahead of the
|
||||||
|
// settle window the watcher queue waits out: these are rows a user is
|
||||||
|
// looking at right now, there are at most a screenful, and a stopped
|
||||||
|
// indexer is exactly when the frontend most needs them to be current.
|
||||||
|
if !self.targeted.is_empty() {
|
||||||
|
self.apply_targeted();
|
||||||
|
}
|
||||||
|
|
||||||
// Ahead of the mode gate: a config edit is reconciled in manual mode
|
// Ahead of the mode gate: a config edit is reconciled in manual mode
|
||||||
// too.
|
// too.
|
||||||
if self.pending_work.is_some() {
|
if self.pending_work.is_some() {
|
||||||
|
|
@ -469,6 +513,71 @@ impl Inner {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Apply the by-name queue: the paths a frontend is displaying.
|
||||||
|
///
|
||||||
|
/// Shaped like [`Inner::apply_pending`] — removals first, same budget —
|
||||||
|
/// but it never escalates to [`Inner::needs_full_run`]. A frontend reads
|
||||||
|
/// what it shows from the file itself, so a failure here leaves the screen
|
||||||
|
/// correct and only the index behind; reindexing the world over that would
|
||||||
|
/// be wildly out of proportion.
|
||||||
|
fn apply_targeted(&mut self) {
|
||||||
|
self.was_busy = true;
|
||||||
|
let mut conn = match self.ensure_write_conn() {
|
||||||
|
Ok(conn) => conn,
|
||||||
|
Err(e) => {
|
||||||
|
crate::log_warn!("coordinator: targeted update unavailable: {}", e);
|
||||||
|
self.targeted.clear();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let deadline = Instant::now() + APPLY_BUDGET;
|
||||||
|
let chunk = self.config.processing.batch_size.max(1);
|
||||||
|
|
||||||
|
// Removals lead for the same reason they do in `apply_pending`: the
|
||||||
|
// queue is an unordered map, and a rename enqueues both halves.
|
||||||
|
let removals: Vec<PathBuf> = self
|
||||||
|
.targeted
|
||||||
|
.iter()
|
||||||
|
.filter(|(_, ev)| is_removal(ev))
|
||||||
|
.map(|(p, _)| p.clone())
|
||||||
|
.collect();
|
||||||
|
for batch in removals.chunks(chunk) {
|
||||||
|
if let Err(e) = crate::incremental::remove_paths(&mut conn, batch, chunk) {
|
||||||
|
crate::log_warn!("coordinator: targeted remove: {}", e);
|
||||||
|
}
|
||||||
|
for path in batch {
|
||||||
|
self.targeted.remove(path);
|
||||||
|
}
|
||||||
|
if Instant::now() >= deadline {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if Instant::now() < deadline {
|
||||||
|
let upserts: Vec<PathBuf> = self
|
||||||
|
.targeted
|
||||||
|
.iter()
|
||||||
|
.filter(|(_, ev)| !is_removal(ev))
|
||||||
|
.map(|(p, _)| p.clone())
|
||||||
|
.collect();
|
||||||
|
for path in upserts {
|
||||||
|
let Some(ev) = self.targeted.remove(&path) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if let Err(e) =
|
||||||
|
apply_fs_event(&mut conn, &ev, &self.config, &self.ignore, &self.registry)
|
||||||
|
{
|
||||||
|
crate::log_warn!("coordinator: targeted apply {:?}: {}", ev, e);
|
||||||
|
}
|
||||||
|
if Instant::now() >= deadline {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
self.write_conn = Some(conn);
|
||||||
|
}
|
||||||
|
|
||||||
fn ensure_write_conn(&mut self) -> Result<Connection, String> {
|
fn ensure_write_conn(&mut self) -> Result<Connection, String> {
|
||||||
if let Some(conn) = self.write_conn.take() {
|
if let Some(conn) = self.write_conn.take() {
|
||||||
return Ok(conn);
|
return Ok(conn);
|
||||||
|
|
@ -799,7 +908,7 @@ impl Inner {
|
||||||
);
|
);
|
||||||
let mut shared = crate::lock_ok(&self.shared);
|
let mut shared = crate::lock_ok(&self.shared);
|
||||||
shared.mode = self.mode;
|
shared.mode = self.mode;
|
||||||
shared.queued_events = self.pending.len();
|
shared.queued_events = self.pending.len() + self.targeted.len();
|
||||||
shared.reconcile = reconcile;
|
shared.reconcile = reconcile;
|
||||||
drop(shared);
|
drop(shared);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -505,6 +505,149 @@ fn a_run_it_schedules_itself_wakes_the_frontend() {
|
||||||
coord.shutdown();
|
coord.shutdown();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- targeted updates (see `IndexCoordinator::update_paths`) --------------
|
||||||
|
|
||||||
|
impl Fixture {
|
||||||
|
/// The `mtime` the index holds for one path, or `None` if it has no row.
|
||||||
|
fn stored_mtime(&self, path: &std::path::Path) -> Option<i64> {
|
||||||
|
let conn = db::open_existing(&self.db.to_string_lossy(), false).ok()?;
|
||||||
|
conn.query_row(
|
||||||
|
"SELECT mtime FROM files WHERE path = ?1",
|
||||||
|
[path.to_string_lossy().as_ref()],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.ok()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The point of the whole thing: the frontend has just read a file the user is
|
||||||
|
/// looking at, and the index catches up even though indexing is stopped — with
|
||||||
|
/// no watcher running and no full run scheduled.
|
||||||
|
#[test]
|
||||||
|
fn update_paths_indexes_one_file_with_indexing_stopped() {
|
||||||
|
let f = Fixture::new(false);
|
||||||
|
std::fs::write(f.dir.join("seed.txt"), "initial content").unwrap();
|
||||||
|
f.seed_index();
|
||||||
|
assert_eq!(f.file_count(), 1);
|
||||||
|
|
||||||
|
let coord = start_coord(f.config.clone());
|
||||||
|
let added = f.dir.join("added.txt");
|
||||||
|
std::fs::write(&added, "written while indexing was stopped").unwrap();
|
||||||
|
coord.update_paths(vec![added.clone()]);
|
||||||
|
|
||||||
|
wait_for("targeted insert", Duration::from_secs(20), || {
|
||||||
|
f.stored_mtime(&added).is_some()
|
||||||
|
});
|
||||||
|
assert_eq!(
|
||||||
|
coord.state().mode,
|
||||||
|
IndexMode::ManualStopped,
|
||||||
|
"a targeted update started a run"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
coord.state().last_full_index.is_some(),
|
||||||
|
"the seed stamp was disturbed"
|
||||||
|
);
|
||||||
|
coord.shutdown();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The same call is how a row is *validated*: submitting a path the index
|
||||||
|
/// already agrees with must not rewrite it, which is what makes it cheap
|
||||||
|
/// enough for the frontend to submit whatever it just looked at.
|
||||||
|
#[test]
|
||||||
|
fn update_paths_leaves_a_row_that_already_agrees_alone() {
|
||||||
|
let f = Fixture::new(false);
|
||||||
|
let file = f.dir.join("steady.txt");
|
||||||
|
std::fs::write(&file, "unchanged").unwrap();
|
||||||
|
f.seed_index();
|
||||||
|
let before = f.stored_mtime(&file).expect("seeded");
|
||||||
|
|
||||||
|
let coord = start_coord(f.config.clone());
|
||||||
|
coord.update_paths(vec![file.clone()]);
|
||||||
|
// No state change to wait on, so wait out a few ticks instead.
|
||||||
|
std::thread::sleep(Duration::from_secs(3));
|
||||||
|
|
||||||
|
assert_eq!(f.stored_mtime(&file), Some(before));
|
||||||
|
assert_eq!(f.file_count(), 1);
|
||||||
|
coord.shutdown();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A path outside every indexed root is not the index's to hold, however it
|
||||||
|
/// was submitted: a result renamed into an un-indexed folder must not follow
|
||||||
|
/// the row into the index at its new home.
|
||||||
|
#[test]
|
||||||
|
fn update_paths_ignores_a_path_outside_every_root() {
|
||||||
|
let f = Fixture::new(false);
|
||||||
|
std::fs::write(f.dir.join("seed.txt"), "initial content").unwrap();
|
||||||
|
f.seed_index();
|
||||||
|
assert_eq!(f.file_count(), 1);
|
||||||
|
|
||||||
|
// A sibling of the indexed tree, under the same scratch parent.
|
||||||
|
let outside = f.dir.parent().unwrap().join("elsewhere");
|
||||||
|
std::fs::create_dir_all(&outside).unwrap();
|
||||||
|
let stray = outside.join("moved-here.txt");
|
||||||
|
std::fs::write(&stray, "renamed out of the index").unwrap();
|
||||||
|
|
||||||
|
let coord = start_coord(f.config.clone());
|
||||||
|
coord.update_paths(vec![stray.clone()]);
|
||||||
|
std::thread::sleep(Duration::from_secs(3));
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
f.stored_mtime(&stray),
|
||||||
|
None,
|
||||||
|
"an un-indexed folder gained a row"
|
||||||
|
);
|
||||||
|
assert_eq!(f.file_count(), 1);
|
||||||
|
coord.shutdown();
|
||||||
|
std::fs::remove_dir_all(&outside).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A row whose file has gone leaves the index too — the frontend hands over
|
||||||
|
/// the path, not a verb, so the coordinator decides from what is on disk.
|
||||||
|
#[test]
|
||||||
|
fn update_paths_removes_a_row_whose_file_is_gone() {
|
||||||
|
let f = Fixture::new(false);
|
||||||
|
let file = f.dir.join("doomed.txt");
|
||||||
|
std::fs::write(&file, "not for long").unwrap();
|
||||||
|
f.seed_index();
|
||||||
|
assert!(f.stored_mtime(&file).is_some());
|
||||||
|
|
||||||
|
let coord = start_coord(f.config.clone());
|
||||||
|
std::fs::remove_file(&file).unwrap();
|
||||||
|
coord.update_paths(vec![file.clone()]);
|
||||||
|
|
||||||
|
wait_for("targeted remove", Duration::from_secs(20), || {
|
||||||
|
f.stored_mtime(&file).is_none()
|
||||||
|
});
|
||||||
|
coord.shutdown();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The single-writer rule still holds: a targeted update submitted while a
|
||||||
|
/// full run owns the database waits for it rather than opening a second
|
||||||
|
/// writer beside it.
|
||||||
|
#[test]
|
||||||
|
fn update_paths_waits_for_a_full_run_rather_than_racing_it() {
|
||||||
|
let f = Fixture::new(false);
|
||||||
|
for i in 0..400 {
|
||||||
|
std::fs::write(f.dir.join(format!("f{i}.txt")), "body").unwrap();
|
||||||
|
}
|
||||||
|
let coord = start_coord(f.config.clone());
|
||||||
|
coord.reindex_now();
|
||||||
|
|
||||||
|
let added = f.dir.join("late.txt");
|
||||||
|
std::fs::write(&added, "submitted mid-run").unwrap();
|
||||||
|
coord.update_paths(vec![added.clone()]);
|
||||||
|
|
||||||
|
wait_for("run finished", Duration::from_secs(60), || {
|
||||||
|
coord.state().last_full_index.is_some()
|
||||||
|
});
|
||||||
|
wait_for(
|
||||||
|
"targeted insert after the run",
|
||||||
|
Duration::from_secs(20),
|
||||||
|
|| f.stored_mtime(&added).is_some(),
|
||||||
|
);
|
||||||
|
coord.shutdown();
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn auto_mode_runs_initial_index_and_applies_watcher_events() {
|
fn auto_mode_runs_initial_index_and_applies_watcher_events() {
|
||||||
let f = Fixture::new(true);
|
let f = Fixture::new(true);
|
||||||
|
|
|
||||||
|
|
@ -217,6 +217,85 @@ pub fn set_content_done(
|
||||||
set_state_clearing_failure(tx, file_id, STATE_DONE, "update DONE")
|
set_state_clearing_failure(tx, file_id, STATE_DONE, "update DONE")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Reusable decode buffer and decompression context for the readers of
|
||||||
|
/// `documents_text` — the read side's mirror of [`DocEncoder`].
|
||||||
|
///
|
||||||
|
/// Shared by the cascade's full-text passes and by [`crate::live`], which
|
||||||
|
/// re-reads one row's body when a file under a visible result changes.
|
||||||
|
///
|
||||||
|
/// `zstd::decode_all` builds and tears down a `ZSTD_DCtx` *and* allocates a
|
||||||
|
/// fresh output `Vec` on every call, and it is called once per candidate row.
|
||||||
|
/// One context and one buffer, reused across a whole scan, make that a
|
||||||
|
/// per-scan cost instead of a per-row one.
|
||||||
|
pub struct DocDecoder {
|
||||||
|
dctx: zstd::bulk::Decompressor<'static>,
|
||||||
|
buf: Vec<u8>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where [`DocDecoder::decode`]'s buffer starts before it has seen a document.
|
||||||
|
/// Most extracted text is well under this, so the doubling below rarely runs.
|
||||||
|
const INITIAL_DOC_CAPACITY: usize = 64 * 1024;
|
||||||
|
|
||||||
|
/// Where the doubling stops. Stored text is capped at
|
||||||
|
/// `processing.maximum_text_size` (256 KiB by default), so this is far above
|
||||||
|
/// any legitimate document even if that setting is raised — past it, a failure
|
||||||
|
/// is a corrupt frame rather than a buffer that is too small.
|
||||||
|
const MAX_DOC_CAPACITY: usize = 64 * 1024 * 1024;
|
||||||
|
|
||||||
|
impl DocDecoder {
|
||||||
|
pub fn new() -> Result<Self, String> {
|
||||||
|
Ok(DocDecoder {
|
||||||
|
dctx: zstd::bulk::Decompressor::new().map_err(|e| e.to_string())?,
|
||||||
|
buf: Vec::new(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decompress `blob` and borrow the result as text.
|
||||||
|
///
|
||||||
|
/// Returns `None` for a corrupt frame or non-UTF-8 content. Nothing is
|
||||||
|
/// copied: the indexer stores UTF-8, so the bytes are borrowed in place
|
||||||
|
/// rather than run through `String::from_utf8_lossy(..).into_owned()`,
|
||||||
|
/// which duplicated the whole document even when it was already valid.
|
||||||
|
pub fn decode(&mut self, blob: &[u8]) -> Option<&str> {
|
||||||
|
self.buf.clear();
|
||||||
|
// `decompress_to_buffer` writes into spare capacity and fails rather
|
||||||
|
// than growing, so the room has to be there first.
|
||||||
|
//
|
||||||
|
// The frame header would say how much is needed, but the indexer
|
||||||
|
// writes with `zstd::encode_all`, which is *stream*-based and so
|
||||||
|
// records no content size — `get_frame_content_size` says `None` for
|
||||||
|
// every row this ever sees. Falling back to `zstd::decode_all` there
|
||||||
|
// looked harmless and was not: it builds a streaming decoder per call,
|
||||||
|
// which measured as one ~2.4 MiB allocation per document and 27 of the
|
||||||
|
// 30 GiB a fuzzy search moved through the allocator.
|
||||||
|
//
|
||||||
|
// So grow this buffer instead and keep reusing it. It settles at the
|
||||||
|
// largest document in the scan within the first few rows, after which
|
||||||
|
// decoding a row allocates nothing at all.
|
||||||
|
if let Ok(Some(size)) = zstd::zstd_safe::get_frame_content_size(blob) {
|
||||||
|
self.buf.reserve(usize::try_from(size).ok()?);
|
||||||
|
}
|
||||||
|
loop {
|
||||||
|
if self.buf.capacity() == 0 {
|
||||||
|
self.buf.reserve(INITIAL_DOC_CAPACITY);
|
||||||
|
}
|
||||||
|
match self.dctx.decompress_to_buffer(blob, &mut self.buf) {
|
||||||
|
Ok(_) => break,
|
||||||
|
// Too small, or corrupt — the bulk API cannot tell us which.
|
||||||
|
// Growing is only worth trying while the buffer is still
|
||||||
|
// smaller than any document could legitimately be.
|
||||||
|
Err(_) if self.buf.capacity() < MAX_DOC_CAPACITY => {
|
||||||
|
let bigger = self.buf.capacity().saturating_mul(2);
|
||||||
|
self.buf.clear();
|
||||||
|
self.buf.reserve(bigger);
|
||||||
|
}
|
||||||
|
Err(_) => return None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::str::from_utf8(&self.buf).ok()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Level 3 hits ~3-5× on English prose at high throughput (hundreds of
|
/// Level 3 hits ~3-5× on English prose at high throughput (hundreds of
|
||||||
/// MB/s); level 9+ would shave a few percent more at 10× the CPU cost, and
|
/// MB/s); level 9+ would shave a few percent more at 10× the CPU cost, and
|
||||||
/// readers decompress far faster than writers compress.
|
/// readers decompress far faster than writers compress.
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,7 @@
|
||||||
//! | [`PRAGMAS_SEARCH`] | search worker | held across a typing session | 32 MiB |
|
//! | [`PRAGMAS_SEARCH`] | search worker | held across a typing session | 32 MiB |
|
||||||
//! | [`PRAGMAS_READONLY`] | one-shot readers | a single query | 4 MiB |
|
//! | [`PRAGMAS_READONLY`] | one-shot readers | a single query | 4 MiB |
|
||||||
//! | [`PRAGMAS_MAINTENANCE`] | VACUUM | one bulk copy | 8 MiB |
|
//! | [`PRAGMAS_MAINTENANCE`] | VACUUM | one bulk copy | 8 MiB |
|
||||||
//! | [`PRAGMAS_WALK_READER`] | per-root row prefetch | the walk | 1 MiB |
|
//! | [`PRAGMAS_WALK_READER`] | per-root walk prefetch and content feeder | the run | 1 MiB |
|
||||||
//!
|
//!
|
||||||
//! `PRAGMA mmap_size` is absent from all of them: SQLCipher's codec disables
|
//! `PRAGMA mmap_size` is absent from all of them: SQLCipher's codec disables
|
||||||
//! mmap at runtime only when a key is set, mapped pages still count in
|
//! mmap at runtime only when a key is set, mapped pages still count in
|
||||||
|
|
@ -124,12 +124,19 @@ pub const PRAGMAS_READONLY: &str = "
|
||||||
PRAGMA foreign_keys = ON;
|
PRAGMA foreign_keys = ON;
|
||||||
";
|
";
|
||||||
|
|
||||||
/// Pragmas for a walk's row-prefetch connection.
|
/// Pragmas for a root's own reader: the walk's row prefetch, and then the
|
||||||
|
/// content pass's feeder ([`crate::content`]), which reuses this profile for
|
||||||
|
/// the rest of the run.
|
||||||
///
|
///
|
||||||
/// One of these exists per indexing root, so the cache size is multiplied by
|
/// Two of these can exist per indexing root, so the cache size is multiplied
|
||||||
/// the root count. 1 MiB holds the upper levels of `idx_files_parent` hot,
|
/// by the root count. 1 MiB is sized for the walk's queries, which each read
|
||||||
/// which is all these queries touch: each is a single index range lookup,
|
/// one range of `idx_files_parent` once and never revisit it. The feeder's
|
||||||
/// and the pages under it are read once and not revisited.
|
/// paging is the same shape, but its one-off `count_extract_scope` at pass
|
||||||
|
/// start is not: that scans the root's whole path range fetching a row per
|
||||||
|
/// entry, so on a large root it is a cold read all the way through. It is
|
||||||
|
/// deliberately here rather than on the writer — the writer holding still for
|
||||||
|
/// it stopped every other root's walk — and this is the connection that pays
|
||||||
|
/// for that, once per root.
|
||||||
pub const PRAGMAS_WALK_READER: &str = "
|
pub const PRAGMAS_WALK_READER: &str = "
|
||||||
PRAGMA busy_timeout = 5000;
|
PRAGMA busy_timeout = 5000;
|
||||||
PRAGMA cache_size = -1024;
|
PRAGMA cache_size = -1024;
|
||||||
|
|
|
||||||
|
|
@ -41,11 +41,6 @@ impl ExtractedContent {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn with_property(mut self, key: impl Into<String>, value: impl Into<String>) -> Self {
|
|
||||||
self.properties.insert(key.into(), value.into());
|
|
||||||
self
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Convert properties into the `Vec<(String, String)>` shape expected by
|
/// Convert properties into the `Vec<(String, String)>` shape expected by
|
||||||
/// [`crate::db::repo::set_content_done`]. Keys are sorted for determinism
|
/// [`crate::db::repo::set_content_done`]. Keys are sorted for determinism
|
||||||
/// in tests and snapshots.
|
/// in tests and snapshots.
|
||||||
|
|
@ -300,9 +295,9 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn properties_sorted_is_deterministic() {
|
fn properties_sorted_is_deterministic() {
|
||||||
let c = ExtractedContent::with_text("hi")
|
let mut c = ExtractedContent::with_text("hi");
|
||||||
.with_property("b", "2")
|
c.properties.insert("b".into(), "2".into());
|
||||||
.with_property("a", "1");
|
c.properties.insert("a".into(), "1".into());
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
c.properties_sorted(),
|
c.properties_sorted(),
|
||||||
vec![
|
vec![
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,6 @@ use rusqlite::Connection;
|
||||||
use super::*;
|
use super::*;
|
||||||
use crate::config::Config;
|
use crate::config::Config;
|
||||||
use crate::db::repo::{self};
|
use crate::db::repo::{self};
|
||||||
use crate::indexing::should_abort;
|
|
||||||
|
|
||||||
/// The compressed sidecar for one row, or `None` where there is none to write
|
/// The compressed sidecar for one row, or `None` where there is none to write
|
||||||
/// — an empty body, or `store_text_for_snippets` turned off.
|
/// — an empty body, or `store_text_for_snippets` turned off.
|
||||||
|
|
@ -29,12 +28,13 @@ type Body = Result<Option<Vec<u8>>, String>;
|
||||||
///
|
///
|
||||||
/// What that lock does *not* gate, so the benefit is not overclaimed: search
|
/// What that lock does *not* gate, so the benefit is not overclaimed: search
|
||||||
/// holds its own connection (`db::open::open_search_reader`) and the database
|
/// holds its own connection (`db::open::open_search_reader`) and the database
|
||||||
/// is WAL, where a reader never blocks on a writer. `conn_mutex` serializes
|
/// is WAL, where a reader never blocks on a writer. Nor does it separate one
|
||||||
/// the indexer against itself — one root's content stores against another's
|
/// root from another — every root's writes already run on the single writer
|
||||||
/// walk inserts, the scope reconciler's slices, and WAL checkpointing. A
|
/// thread, so two of them are never inside the lock at once. What it actually
|
||||||
/// whole-tree wall-clock run is dominated by FTS5 trigram tokenization and
|
/// serializes the run against is WAL checkpointing, which `run_indexing`
|
||||||
/// does not move measurably from this change; it is contention that improves,
|
/// forces from the same thread between turns. A whole-tree wall-clock run is
|
||||||
/// not throughput.
|
/// dominated by FTS5 trigram tokenization and does not move measurably from
|
||||||
|
/// this change; it is the length of the hold that improves, not throughput.
|
||||||
fn compress_bodies<'a>(
|
fn compress_bodies<'a>(
|
||||||
texts: impl Iterator<Item = Option<&'a str>>,
|
texts: impl Iterator<Item = Option<&'a str>>,
|
||||||
config: &Config,
|
config: &Config,
|
||||||
|
|
@ -198,15 +198,13 @@ pub fn process_batch_inserts(
|
||||||
/// Delete the rows a completed run found no file behind, in chunked
|
/// Delete the rows a completed run found no file behind, in chunked
|
||||||
/// transactions. Returns how many went.
|
/// transactions. Returns how many went.
|
||||||
///
|
///
|
||||||
/// `should_abort` *blocks* while the indexer is suspended, so it must only be
|
/// The stop flag is checked between chunks and again per path, never with a
|
||||||
/// observed between chunks with nothing held — checking it mid-transaction
|
/// transaction open: a chunk either commits whole or is not begun, so a stop
|
||||||
/// pins the shared connection for the whole suspension and freezes the GUI.
|
/// cannot leave the index half-reconciled.
|
||||||
/// The stop flag, which never blocks, guards the inner loop.
|
|
||||||
pub fn cleanup_stale_index_entries(
|
pub fn cleanup_stale_index_entries(
|
||||||
conn_mutex: &Arc<Mutex<Connection>>,
|
conn_mutex: &Arc<Mutex<Connection>>,
|
||||||
stale_paths: &[String],
|
stale_paths: &[String],
|
||||||
stop_flag: &Arc<AtomicBool>,
|
stop_flag: &Arc<AtomicBool>,
|
||||||
suspend_flag: &Arc<AtomicBool>,
|
|
||||||
config: &Config,
|
config: &Config,
|
||||||
) -> Result<usize, String> {
|
) -> Result<usize, String> {
|
||||||
if stale_paths.is_empty() {
|
if stale_paths.is_empty() {
|
||||||
|
|
@ -216,8 +214,8 @@ pub fn cleanup_stale_index_entries(
|
||||||
let mut deleted_count = 0usize;
|
let mut deleted_count = 0usize;
|
||||||
|
|
||||||
for batch in stale_paths.chunks(chunk) {
|
for batch in stale_paths.chunks(chunk) {
|
||||||
// Outside the lock, so a suspend parks here rather than mid-transaction.
|
// Outside the lock, so a stop is seen before a transaction is begun.
|
||||||
if should_abort(stop_flag, suspend_flag) {
|
if stop_flag.load(Ordering::Relaxed) {
|
||||||
return Ok(deleted_count);
|
return Ok(deleted_count);
|
||||||
}
|
}
|
||||||
let conn = crate::lock_ok(conn_mutex);
|
let conn = crate::lock_ok(conn_mutex);
|
||||||
|
|
@ -241,7 +239,7 @@ pub fn cleanup_stale_index_entries(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if deleted_count > 0 && !should_abort(stop_flag, suspend_flag) {
|
if deleted_count > 0 && !stop_flag.load(Ordering::Relaxed) {
|
||||||
let conn = crate::lock_ok(conn_mutex);
|
let conn = crate::lock_ok(conn_mutex);
|
||||||
fts_finalize_after_text_indexing(&conn);
|
fts_finalize_after_text_indexing(&conn);
|
||||||
}
|
}
|
||||||
|
|
@ -296,71 +294,107 @@ pub struct ExtractScope {
|
||||||
pub already_done: usize,
|
pub already_done: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Prepare a root's extraction scope: flip oversize pending rows to NA
|
/// The `maximum_text_file_size` bound as the SQL below compares it.
|
||||||
/// (idempotent) and count what is pending vs. already extracted in the range.
|
pub(crate) fn max_text_file_size(config: &Config) -> i64 {
|
||||||
|
i64::try_from(config.processing.maximum_text_file_size).unwrap_or(i64::MAX)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Flip a root's oversize pending rows to NA. Idempotent.
|
||||||
///
|
///
|
||||||
/// The oversize sweep covers what walk-time decisions cannot: a
|
/// Covers what walk-time decisions cannot: a `maximum_text_file_size`
|
||||||
/// `maximum_text_file_size` *lowered* between runs (which does not force a
|
/// *lowered* between runs (which does not force a rebuild), and rows left
|
||||||
/// rebuild), and rows left pending by an older build.
|
/// pending by an older build. Rows this misses would stay pending forever, so
|
||||||
pub fn extract_scope_prepare(
|
/// it runs on the writer before a root's content pass starts.
|
||||||
conn_mutex: &Arc<Mutex<Connection>>,
|
pub fn mark_oversize_pending_na(
|
||||||
|
conn: &Connection,
|
||||||
cursor: &ExtractCursor,
|
cursor: &ExtractCursor,
|
||||||
config: &Config,
|
config: &Config,
|
||||||
) -> Result<ExtractScope, String> {
|
) -> Result<(), String> {
|
||||||
let max_size = i64::try_from(config.processing.maximum_text_file_size).unwrap_or(i64::MAX);
|
|
||||||
let conn = crate::lock_ok(conn_mutex);
|
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"UPDATE files SET content_state = 3 \
|
"UPDATE files SET content_state = 3 \
|
||||||
WHERE content_state = 0 AND size > ?1 AND path >= ?2 AND path < ?3",
|
WHERE content_state = 0 AND size > ?1 AND path >= ?2 AND path < ?3",
|
||||||
rusqlite::params![max_size, cursor.lo, cursor.hi],
|
rusqlite::params![max_text_file_size(config), cursor.lo, cursor.hi],
|
||||||
)
|
)
|
||||||
.map_err(|e| format!("mark oversize files NA: {}", e))?;
|
.map_err(|e| format!("mark oversize files NA: {}", e))?;
|
||||||
let pending: i64 = conn
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Count what a root's range holds: rows still to extract this run, and rows
|
||||||
|
/// whose text is already searchable from earlier runs.
|
||||||
|
///
|
||||||
|
/// One range scan for both figures. Deliberately callable on any connection
|
||||||
|
/// — the content pass runs it on its own read connection rather than on the
|
||||||
|
/// indexer's writer, because on a large root it takes seconds, and seconds of
|
||||||
|
/// writer time is every other root's walk standing still.
|
||||||
|
pub fn count_extract_scope(
|
||||||
|
conn: &Connection,
|
||||||
|
cursor: &ExtractCursor,
|
||||||
|
config: &Config,
|
||||||
|
) -> Result<ExtractScope, String> {
|
||||||
|
let (pending, already_done): (i64, i64) = conn
|
||||||
.query_row(
|
.query_row(
|
||||||
"SELECT COUNT(*) FROM files \
|
"SELECT COALESCE(SUM(content_state = 0 AND size <= ?1), 0), \
|
||||||
WHERE content_state = 0 AND size <= ?1 AND path >= ?2 AND path < ?3",
|
COALESCE(SUM(content_state = 1), 0) \
|
||||||
rusqlite::params![max_size, cursor.lo, cursor.hi],
|
FROM files WHERE path >= ?2 AND path < ?3",
|
||||||
|row| row.get(0),
|
rusqlite::params![max_text_file_size(config), cursor.lo, cursor.hi],
|
||||||
|
|row| Ok((row.get(0)?, row.get(1)?)),
|
||||||
)
|
)
|
||||||
.map_err(|e| format!("Failed to count pending text files: {}", e))?;
|
.map_err(|e| format!("Failed to count text files: {}", e))?;
|
||||||
let already_done: i64 = conn
|
|
||||||
.query_row(
|
|
||||||
"SELECT COUNT(*) FROM files \
|
|
||||||
WHERE content_state = 1 AND path >= ?1 AND path < ?2",
|
|
||||||
rusqlite::params![cursor.lo, cursor.hi],
|
|
||||||
|row| row.get(0),
|
|
||||||
)
|
|
||||||
.map_err(|e| format!("Failed to count extracted files: {}", e))?;
|
|
||||||
Ok(ExtractScope {
|
Ok(ExtractScope {
|
||||||
pending: pending.max(0) as usize,
|
pending: pending.max(0) as usize,
|
||||||
already_done: already_done.max(0) as usize,
|
already_done: already_done.max(0) as usize,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Write a batch of already-extracted rows — the cheap half of the content
|
/// Rows per compression chunk and per transaction inside [`store_extracted`].
|
||||||
/// pass, and all that runs with the connection held. Chunked so each
|
|
||||||
/// transaction stays short.
|
|
||||||
///
|
///
|
||||||
/// Returns how many rows were written. A row whose write fails is logged and
|
/// Half of what a writer turn may hand in (`pipeline::READY_TOPUP` is 64), so
|
||||||
/// skipped rather than failing the run: its `content_state` stays pending, so
|
/// a full turn commits twice rather than once — short holds of the connection
|
||||||
/// the next run retries it.
|
/// being the point. It also bounds the compression thrown away when the
|
||||||
|
/// deadline cuts a chunk short, to at most `STORE_CHUNK - 1` bodies.
|
||||||
|
///
|
||||||
|
/// Note the two buffers are additive: a root can hold `READY_TOPUP` extracted
|
||||||
|
/// rows waiting for the writer *and* `content::READY_CAP` more in its
|
||||||
|
/// channel, so in-flight text per root is bounded by their sum, not by either
|
||||||
|
/// alone.
|
||||||
|
const STORE_CHUNK: usize = 32;
|
||||||
|
|
||||||
|
/// What one [`store_extracted`] call did with the rows it was handed.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||||
|
pub struct Stored {
|
||||||
|
/// Rows the caller must now drop from its buffer, written or not.
|
||||||
|
pub consumed: usize,
|
||||||
|
/// Rows whose write succeeded — whose `content_state` moved.
|
||||||
|
pub written: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Write already-extracted rows — the cheap half of the content pass, and all
|
||||||
|
/// that runs with the connection held — until `deadline`.
|
||||||
|
///
|
||||||
|
/// This is where a document's FTS5 trigram tokenization happens, up to
|
||||||
|
/// `maximum_text_size` of it per row, and it is the writer thread's dominant
|
||||||
|
/// cost. The deadline is checked after every row, so a turn on the writer
|
||||||
|
/// overruns it by at most one document; the rows not reached are left for the
|
||||||
|
/// caller to hand back next turn. At least one row is always consumed unless
|
||||||
|
/// the run is already stopped, so a caller looping on this cannot spin.
|
||||||
|
///
|
||||||
|
/// A row whose write fails is logged and consumed rather than failing the
|
||||||
|
/// run: its `content_state` stays pending, so the next run retries it.
|
||||||
pub fn store_extracted(
|
pub fn store_extracted(
|
||||||
conn_mutex: &Arc<Mutex<Connection>>,
|
conn_mutex: &Arc<Mutex<Connection>>,
|
||||||
rows: &[crate::content::ExtractedRow],
|
rows: &[crate::content::ExtractedRow],
|
||||||
stop_flag: &Arc<AtomicBool>,
|
stop_flag: &Arc<AtomicBool>,
|
||||||
config: &Config,
|
config: &Config,
|
||||||
) -> Result<usize, String> {
|
deadline: std::time::Instant,
|
||||||
if rows.is_empty() {
|
) -> Result<Stored, String> {
|
||||||
return Ok(0);
|
let mut done = Stored::default();
|
||||||
}
|
for chunk in rows.chunks(STORE_CHUNK) {
|
||||||
let mut written = 0usize;
|
|
||||||
for batch in rows.chunks(config.processing.batch_size.max(1)) {
|
|
||||||
if stop_flag.load(Ordering::Relaxed) {
|
if stop_flag.load(Ordering::Relaxed) {
|
||||||
return Ok(written);
|
break;
|
||||||
}
|
}
|
||||||
// Outside the lock — see `compress_bodies`.
|
// Outside the lock — see `compress_bodies`.
|
||||||
let bodies = compress_bodies(
|
let bodies = compress_bodies(
|
||||||
batch
|
chunk
|
||||||
.iter()
|
.iter()
|
||||||
.map(|r| crate::file_handling::outcome_body(&r.outcome)),
|
.map(|r| crate::file_handling::outcome_body(&r.outcome)),
|
||||||
config,
|
config,
|
||||||
|
|
@ -369,16 +403,34 @@ pub fn store_extracted(
|
||||||
let tx = conn
|
let tx = conn
|
||||||
.unchecked_transaction()
|
.unchecked_transaction()
|
||||||
.map_err(|e| format!("Failed to begin transaction: {}", e))?;
|
.map_err(|e| format!("Failed to begin transaction: {}", e))?;
|
||||||
for (i, row) in batch.iter().enumerate() {
|
let mut cut = false;
|
||||||
let zstd = body_or_skip!(bodies, i, row.name);
|
for (i, row) in chunk.iter().enumerate() {
|
||||||
if let Err(e) = store_content_outcome(&tx, row.file_id, &row.name, &row.outcome, zstd) {
|
// Counted before anything can skip it: consumed is what the
|
||||||
crate::log_warn!("content indexing for {}: {}", row.name, e);
|
// caller drains, and a row that failed still has to leave.
|
||||||
continue;
|
done.consumed += 1;
|
||||||
|
match &bodies[i] {
|
||||||
|
Err(e) => crate::log_warn!("compress text for {}: {}", row.name, e),
|
||||||
|
Ok(zstd) => match store_content_outcome(
|
||||||
|
&tx,
|
||||||
|
row.file_id,
|
||||||
|
&row.name,
|
||||||
|
&row.outcome,
|
||||||
|
zstd.as_deref(),
|
||||||
|
) {
|
||||||
|
Ok(()) => done.written += 1,
|
||||||
|
Err(e) => crate::log_warn!("content indexing for {}: {}", row.name, e),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if stop_flag.load(Ordering::Relaxed) || std::time::Instant::now() >= deadline {
|
||||||
|
cut = true;
|
||||||
|
break;
|
||||||
}
|
}
|
||||||
written += 1;
|
|
||||||
}
|
}
|
||||||
tx.commit()
|
tx.commit()
|
||||||
.map_err(|e| format!("Failed to commit transaction: {}", e))?;
|
.map_err(|e| format!("Failed to commit transaction: {}", e))?;
|
||||||
|
if cut {
|
||||||
|
break;
|
||||||
}
|
}
|
||||||
Ok(written)
|
}
|
||||||
|
Ok(done)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,17 @@ use super::*;
|
||||||
use std::sync::atomic::{AtomicBool, Ordering};
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
|
||||||
/// A path that does not exist yet — these tests build the tree themselves.
|
/// A path that does not exist yet — these tests build the tree themselves.
|
||||||
|
/// The removed `extract_scope_prepare`: the oversize sweep, then the count.
|
||||||
|
fn extract_scope_prepare(
|
||||||
|
conn_mutex: &std::sync::Arc<std::sync::Mutex<rusqlite::Connection>>,
|
||||||
|
cursor: &ExtractCursor,
|
||||||
|
config: &Config,
|
||||||
|
) -> Result<ExtractScope, String> {
|
||||||
|
let conn = crate::lock_ok(conn_mutex);
|
||||||
|
super::mark_oversize_pending_na(&conn, cursor, config)?;
|
||||||
|
super::count_extract_scope(&conn, cursor, config)
|
||||||
|
}
|
||||||
|
|
||||||
fn tmp(tag: &str) -> std::path::PathBuf {
|
fn tmp(tag: &str) -> std::path::PathBuf {
|
||||||
crate::testutil::scratch_dir(tag).join("tree")
|
crate::testutil::scratch_dir(tag).join("tree")
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -11,11 +11,12 @@ mod count_and_extract_tests;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests;
|
mod tests;
|
||||||
|
|
||||||
pub(crate) use batch::store_inline_text;
|
|
||||||
pub use batch::{
|
pub use batch::{
|
||||||
cleanup_stale_index_entries, extract_scope_prepare, process_batch_inserts,
|
cleanup_stale_index_entries, count_extract_scope, mark_oversize_pending_na,
|
||||||
process_batch_updates, store_extracted, ExtractCursor, ExtractScope,
|
process_batch_inserts, process_batch_updates, store_extracted, ExtractCursor, ExtractScope,
|
||||||
|
Stored,
|
||||||
};
|
};
|
||||||
|
pub(crate) use batch::{max_text_file_size, store_inline_text};
|
||||||
pub use counting::count_tree_entries_fast;
|
pub use counting::count_tree_entries_fast;
|
||||||
pub use paths::{db_key_for_missing_path, filtered_dirs, filtered_walk, UnreadableDirs};
|
pub use paths::{db_key_for_missing_path, filtered_dirs, filtered_walk, UnreadableDirs};
|
||||||
pub(crate) use paths::{normalize_root_string, path_to_db_string, warn_if_unrepresentable};
|
pub(crate) use paths::{normalize_root_string, path_to_db_string, warn_if_unrepresentable};
|
||||||
|
|
|
||||||
|
|
@ -159,25 +159,6 @@ fn remove_path(conn: &mut Connection, path: &Path) -> Result<(), String> {
|
||||||
remove_paths(conn, std::slice::from_ref(&path.to_path_buf()), 1)
|
remove_paths(conn, std::slice::from_ref(&path.to_path_buf()), 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Drop removals that a removal of one of their ancestors already covers.
|
|
||||||
///
|
|
||||||
/// `rm -rf dir/` reports `dir` *and* every file beneath it; removing `dir`
|
|
||||||
/// sweeps its whole path range, so each descendant event is duplicate work.
|
|
||||||
/// For callers holding a raw removal set — the coordinator collapses on
|
|
||||||
/// arrival instead (`collapse_pending_removals`). Containment is
|
|
||||||
/// component-wise, per [`crate::file_handling::UnreadableDirs::covers`].
|
|
||||||
pub fn collapse_removal_roots(paths: Vec<std::path::PathBuf>) -> Vec<std::path::PathBuf> {
|
|
||||||
if paths.len() < 2 {
|
|
||||||
return paths;
|
|
||||||
}
|
|
||||||
let all: std::collections::HashSet<&Path> = paths.iter().map(|p| p.as_path()).collect();
|
|
||||||
paths
|
|
||||||
.iter()
|
|
||||||
.filter(|p| !p.ancestors().skip(1).any(|a| all.contains(a)))
|
|
||||||
.cloned()
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Delete `paths` and everything indexed beneath them, in transactions of at
|
/// Delete `paths` and everything indexed beneath them, in transactions of at
|
||||||
/// most `chunk` paths.
|
/// most `chunk` paths.
|
||||||
///
|
///
|
||||||
|
|
@ -314,47 +295,6 @@ mod tests {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn collapse(paths: &[&str]) -> Vec<String> {
|
|
||||||
let mut out: Vec<String> =
|
|
||||||
collapse_removal_roots(paths.iter().map(std::path::PathBuf::from).collect())
|
|
||||||
.iter()
|
|
||||||
.map(|p| p.to_string_lossy().into_owned())
|
|
||||||
.collect();
|
|
||||||
out.sort();
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn removal_roots_collapse_to_the_shallowest_ancestor() {
|
|
||||||
assert_eq!(
|
|
||||||
collapse(&["/dir", "/dir/a.txt", "/dir/b/c.txt", "/dir/b"]),
|
|
||||||
vec!["/dir"]
|
|
||||||
);
|
|
||||||
|
|
||||||
// Component-wise, so a name-prefix sibling is not swallowed.
|
|
||||||
assert_eq!(
|
|
||||||
collapse(&["/a/b", "/a/bc"]),
|
|
||||||
vec!["/a/b", "/a/bc"],
|
|
||||||
"/a/bc does not live under /a/b"
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
collapse(&["/a/b", "/a/b.txt"]),
|
|
||||||
vec!["/a/b", "/a/b.txt"],
|
|
||||||
"a sibling file sorting between a dir and its children survives"
|
|
||||||
);
|
|
||||||
|
|
||||||
// Unrelated removals all survive; order of input does not matter.
|
|
||||||
assert_eq!(
|
|
||||||
collapse(&["/x/deep/f", "/y", "/x"]),
|
|
||||||
vec!["/x", "/y"],
|
|
||||||
"/x/deep/f is covered by /x, /y is independent"
|
|
||||||
);
|
|
||||||
|
|
||||||
// Degenerate inputs.
|
|
||||||
assert!(collapse(&[]).is_empty());
|
|
||||||
assert_eq!(collapse(&["/only"]), vec!["/only"]);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The collapse must not change what ends up deleted — only how much work
|
/// The collapse must not change what ends up deleted — only how much work
|
||||||
/// it takes to get there.
|
/// it takes to get there.
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -378,9 +318,10 @@ mod tests {
|
||||||
format!("{}/a.txt", canonical_tree).into(),
|
format!("{}/a.txt", canonical_tree).into(),
|
||||||
format!("{}/deep/b.txt", canonical_tree).into(),
|
format!("{}/deep/b.txt", canonical_tree).into(),
|
||||||
];
|
];
|
||||||
let roots = collapse_removal_roots(reported);
|
// Collapsed to its root the way the coordinator collapses an
|
||||||
assert_eq!(roots.len(), 1, "one range covers the whole tree");
|
// arriving queue (`collapse_pending_removals`): one range covers the
|
||||||
|
// whole tree, which is what makes `remove_paths` cheap.
|
||||||
|
let roots = vec![reported[0].clone()];
|
||||||
remove_paths(&mut f.conn, &roots, 200).unwrap();
|
remove_paths(&mut f.conn, &roots, 200).unwrap();
|
||||||
assert_eq!(f.counts(), (1, 1, 1), "only tree2 survives");
|
assert_eq!(f.counts(), (1, 1, 1), "only tree2 survives");
|
||||||
let survivor = f.canonical(&f.dir.join("tree2").join("keep.txt"));
|
let survivor = f.canonical(&f.dir.join("tree2").join("keep.txt"));
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ use std::collections::HashMap;
|
||||||
use std::sync::atomic::{AtomicBool, Ordering};
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
use std::sync::{mpsc, Arc, Mutex};
|
use std::sync::{mpsc, Arc, Mutex};
|
||||||
use std::thread;
|
use std::thread;
|
||||||
use std::time::{Duration, Instant};
|
use std::time::Instant;
|
||||||
|
|
||||||
use crate::config::Config;
|
use crate::config::Config;
|
||||||
use crate::db;
|
use crate::db;
|
||||||
|
|
@ -84,7 +84,6 @@ pub struct IndexingService {
|
||||||
status: Arc<Mutex<IndexingStatus>>,
|
status: Arc<Mutex<IndexingStatus>>,
|
||||||
command_tx: mpsc::Sender<IndexingCommand>,
|
command_tx: mpsc::Sender<IndexingCommand>,
|
||||||
db_connection: Arc<Mutex<Option<Arc<Mutex<Connection>>>>>,
|
db_connection: Arc<Mutex<Option<Arc<Mutex<Connection>>>>>,
|
||||||
suspend_flag: Arc<AtomicBool>,
|
|
||||||
/// The long single statement a run is inside, if any: the prologue's
|
/// The long single statement a run is inside, if any: the prologue's
|
||||||
/// reconcile scan or the epilogue's VACUUM — never both, so one slot
|
/// reconcile scan or the epilogue's VACUUM — never both, so one slot
|
||||||
/// serves and [`IndexingService::cancel_db_work`] reaches either.
|
/// serves and [`IndexingService::cancel_db_work`] reaches either.
|
||||||
|
|
@ -92,9 +91,6 @@ pub struct IndexingService {
|
||||||
_handle: thread::JoinHandle<()>,
|
_handle: thread::JoinHandle<()>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Polling interval for `should_abort` while suspended.
|
|
||||||
const SUSPEND_POLL_MS: u64 = 100;
|
|
||||||
|
|
||||||
/// `indexing.root_workers` rekeyed from the spellings the user typed to the
|
/// `indexing.root_workers` rekeyed from the spellings the user typed to the
|
||||||
/// canonical roots the indexer walks, so an override survives a `~`, a
|
/// canonical roots the indexer walks, so an override survives a `~`, a
|
||||||
/// trailing slash, a relative path or a symlinked root. Entries naming a
|
/// trailing slash, a relative path or a symlinked root. Entries naming a
|
||||||
|
|
@ -112,39 +108,21 @@ fn resolved_root_workers(config: &Config) -> HashMap<String, usize> {
|
||||||
.collect()
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Combined stop/suspend check used by worker loops; `true` iff the caller
|
|
||||||
/// should abort. While suspended (and not stopped) it parks the thread in
|
|
||||||
/// short sleeps until `resume()`.
|
|
||||||
pub(crate) fn should_abort(stop: &Arc<AtomicBool>, suspend: &Arc<AtomicBool>) -> bool {
|
|
||||||
loop {
|
|
||||||
if stop.load(Ordering::Relaxed) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
if !suspend.load(Ordering::Relaxed) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
thread::sleep(Duration::from_millis(SUSPEND_POLL_MS));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl IndexingService {
|
impl IndexingService {
|
||||||
pub fn new() -> Self {
|
pub fn new() -> Self {
|
||||||
let status = Arc::new(Mutex::new(IndexingStatus::Idle));
|
let status = Arc::new(Mutex::new(IndexingStatus::Idle));
|
||||||
let (command_tx, command_rx) = mpsc::channel();
|
let (command_tx, command_rx) = mpsc::channel();
|
||||||
let db_connection = Arc::new(Mutex::new(None));
|
let db_connection = Arc::new(Mutex::new(None));
|
||||||
let suspend_flag = Arc::new(AtomicBool::new(false));
|
|
||||||
let interrupt: Arc<db::InterruptSlot> = Arc::new(db::InterruptSlot::default());
|
let interrupt: Arc<db::InterruptSlot> = Arc::new(db::InterruptSlot::default());
|
||||||
|
|
||||||
let status_clone = status.clone();
|
let status_clone = status.clone();
|
||||||
let db_connection_clone = db_connection.clone();
|
let db_connection_clone = db_connection.clone();
|
||||||
let suspend_clone = suspend_flag.clone();
|
|
||||||
let interrupt_clone = interrupt.clone();
|
let interrupt_clone = interrupt.clone();
|
||||||
let handle = thread::spawn(move || {
|
let handle = thread::spawn(move || {
|
||||||
Self::indexing_thread(
|
Self::indexing_thread(
|
||||||
status_clone,
|
status_clone,
|
||||||
command_rx,
|
command_rx,
|
||||||
db_connection_clone,
|
db_connection_clone,
|
||||||
suspend_clone,
|
|
||||||
interrupt_clone,
|
interrupt_clone,
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
@ -153,7 +131,6 @@ impl IndexingService {
|
||||||
status,
|
status,
|
||||||
command_tx,
|
command_tx,
|
||||||
db_connection,
|
db_connection,
|
||||||
suspend_flag,
|
|
||||||
interrupt,
|
interrupt,
|
||||||
_handle: handle,
|
_handle: handle,
|
||||||
}
|
}
|
||||||
|
|
@ -171,22 +148,6 @@ impl IndexingService {
|
||||||
db::interrupt(&self.interrupt)
|
db::interrupt(&self.interrupt)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Pause the indexer: worker loops calling [`should_abort`] block until
|
|
||||||
/// [`resume`](Self::resume). Does not stop the worker.
|
|
||||||
pub fn suspend(&self) {
|
|
||||||
self.suspend_flag.store(true, Ordering::Relaxed);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Resume indexing after [`suspend`](Self::suspend). No-op if not
|
|
||||||
/// suspended.
|
|
||||||
pub fn resume(&self) {
|
|
||||||
self.suspend_flag.store(false, Ordering::Relaxed);
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn is_suspended(&self) -> bool {
|
|
||||||
self.suspend_flag.load(Ordering::Relaxed)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Start indexing one or more roots; all walk concurrently, funnelling
|
/// Start indexing one or more roots; all walk concurrently, funnelling
|
||||||
/// into one writer thread. Duplicate roots collapse to one walk, and a
|
/// into one writer thread. Duplicate roots collapse to one walk, and a
|
||||||
/// file reachable from more than one is written once. Returns `Err` if a
|
/// file reachable from more than one is written once. Returns `Err` if a
|
||||||
|
|
@ -301,11 +262,6 @@ impl IndexingService {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Force graceful shutdown - used for signal handling
|
|
||||||
pub fn graceful_shutdown(&self) -> Result<(), String> {
|
|
||||||
self.stop_indexing()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Check if configuration changes require index recreation. A pure
|
/// Check if configuration changes require index recreation. A pure
|
||||||
/// *read* check that never wipes; a missing or incompatible DB means
|
/// *read* check that never wipes; a missing or incompatible DB means
|
||||||
/// there is nothing to validate.
|
/// there is nothing to validate.
|
||||||
|
|
@ -367,7 +323,6 @@ impl IndexingService {
|
||||||
status: Arc<Mutex<IndexingStatus>>,
|
status: Arc<Mutex<IndexingStatus>>,
|
||||||
command_rx: mpsc::Receiver<IndexingCommand>,
|
command_rx: mpsc::Receiver<IndexingCommand>,
|
||||||
db_connection: Arc<Mutex<Option<Arc<Mutex<Connection>>>>>,
|
db_connection: Arc<Mutex<Option<Arc<Mutex<Connection>>>>>,
|
||||||
suspend_flag: Arc<AtomicBool>,
|
|
||||||
interrupt: Arc<db::InterruptSlot>,
|
interrupt: Arc<db::InterruptSlot>,
|
||||||
) {
|
) {
|
||||||
let stop_flag = Arc::new(AtomicBool::new(false));
|
let stop_flag = Arc::new(AtomicBool::new(false));
|
||||||
|
|
@ -396,7 +351,6 @@ impl IndexingService {
|
||||||
let config_owned = config.clone();
|
let config_owned = config.clone();
|
||||||
|
|
||||||
let db_connection_clone = db_connection.clone();
|
let db_connection_clone = db_connection.clone();
|
||||||
let suspend_clone = suspend_flag.clone();
|
|
||||||
let interrupt_clone = interrupt.clone();
|
let interrupt_clone = interrupt.clone();
|
||||||
indexing_handle = Some(thread::spawn(move || {
|
indexing_handle = Some(thread::spawn(move || {
|
||||||
// The writer thread: every DB write and every text
|
// The writer thread: every DB write and every text
|
||||||
|
|
@ -407,7 +361,6 @@ impl IndexingService {
|
||||||
&paths_owned,
|
&paths_owned,
|
||||||
&db_path_owned,
|
&db_path_owned,
|
||||||
&stop_flag_clone,
|
&stop_flag_clone,
|
||||||
&suspend_clone,
|
|
||||||
&config_owned,
|
&config_owned,
|
||||||
&db_connection_clone,
|
&db_connection_clone,
|
||||||
&interrupt_clone,
|
&interrupt_clone,
|
||||||
|
|
|
||||||
|
|
@ -13,9 +13,9 @@ use crate::db;
|
||||||
use crate::db::repo;
|
use crate::db::repo;
|
||||||
use crate::extract::Registry;
|
use crate::extract::Registry;
|
||||||
use crate::file_handling::{
|
use crate::file_handling::{
|
||||||
cleanup_stale_index_entries, count_tree_entries_fast, extract_scope_prepare,
|
cleanup_stale_index_entries, count_tree_entries_fast, fts_finalize_after_text_indexing,
|
||||||
fts_finalize_after_text_indexing, normalize_root_string, process_batch_inserts,
|
mark_oversize_pending_na, normalize_root_string, process_batch_inserts, process_batch_updates,
|
||||||
process_batch_updates, store_extracted, ExtractCursor, FileIndexAction, OwnedNewFile,
|
store_extracted, ExtractCursor, ExtractScope, FileIndexAction, OwnedNewFile,
|
||||||
};
|
};
|
||||||
use crate::walk::{thread_count_for, walk_indexable_files, ParallelWalk, TryNext, WalkEvent};
|
use crate::walk::{thread_count_for, walk_indexable_files, ParallelWalk, TryNext, WalkEvent};
|
||||||
|
|
||||||
|
|
@ -84,6 +84,20 @@ impl Drop for CancelOnDrop {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Writer time one root's turn may take before the round moves on.
|
||||||
|
///
|
||||||
|
/// This is the bound on how long any root can hold up the others. Before
|
||||||
|
/// there was one, an extraction turn ran to the end of whatever was ready —
|
||||||
|
/// half a second to two seconds of FTS5 trigram tokenization for a batch of
|
||||||
|
/// large documents — while a walking root's rows sat in its channel and its
|
||||||
|
/// walkers parked behind them. Reads as "4/4 workers busy, no progress".
|
||||||
|
const TURN_SLICE: Duration = Duration::from_millis(100);
|
||||||
|
|
||||||
|
/// Most extracted rows a root holds back between turns. Not `quantum`: a row
|
||||||
|
/// carries up to `maximum_text_size` of text, and 500 of those would be
|
||||||
|
/// 128 MiB per root. At 64 it is 16 MiB.
|
||||||
|
const READY_TOPUP: usize = 64;
|
||||||
|
|
||||||
/// One root's in-flight indexing state, owned by the writer loop.
|
/// One root's in-flight indexing state, owned by the writer loop.
|
||||||
pub(super) struct RootPipeline {
|
pub(super) struct RootPipeline {
|
||||||
pub(super) root: String,
|
pub(super) root: String,
|
||||||
|
|
@ -100,8 +114,14 @@ pub(super) struct RootPipeline {
|
||||||
pub(super) phase: RootPhase,
|
pub(super) phase: RootPhase,
|
||||||
/// The running content pass, once this root's walk has finished.
|
/// The running content pass, once this root's walk has finished.
|
||||||
pub(super) content: Option<crate::content::ContentPass>,
|
pub(super) content: Option<crate::content::ContentPass>,
|
||||||
pub(super) extract_total: usize,
|
/// Extracted rows pulled off the pass and not yet written. A turn writes
|
||||||
pub(super) extracted: usize,
|
/// for its slice, not for its batch, so it may leave some behind.
|
||||||
|
pub(super) ready: Vec<crate::content::ExtractedRow>,
|
||||||
|
/// Rows this run's content pass has written for this root.
|
||||||
|
pub(super) written: usize,
|
||||||
|
/// The pass's range counts, cached once known so a `Done` root still has
|
||||||
|
/// them after its pass is gone.
|
||||||
|
pub(super) totals: Option<ExtractScope>,
|
||||||
pub(super) current_file: Option<String>,
|
pub(super) current_file: Option<String>,
|
||||||
/// When this root's current phase began, for the one line each phase logs
|
/// When this root's current phase began, for the one line each phase logs
|
||||||
/// when it ends.
|
/// when it ends.
|
||||||
|
|
@ -173,8 +193,16 @@ impl RootPipeline {
|
||||||
stats.map_or((0, 0), |s| (s.active(), s.total()))
|
stats.map_or((0, 0), |s| (s.active(), s.total()))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn snapshot(&self) -> RootProgress {
|
/// The pass's counts, from the cache or — until the cache is filled — from
|
||||||
|
/// the pass itself.
|
||||||
|
fn extract_totals(&self) -> Option<ExtractScope> {
|
||||||
|
self.totals
|
||||||
|
.or_else(|| self.content.as_ref().and_then(|p| p.totals()))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(super) fn snapshot(&self) -> RootProgress {
|
||||||
let (active_workers, total_workers) = self.worker_counts();
|
let (active_workers, total_workers) = self.worker_counts();
|
||||||
|
let totals = self.extract_totals();
|
||||||
RootProgress {
|
RootProgress {
|
||||||
root: self.root.clone(),
|
root: self.root.clone(),
|
||||||
phase: self.phase,
|
phase: self.phase,
|
||||||
|
|
@ -183,30 +211,58 @@ impl RootPipeline {
|
||||||
0 => None,
|
0 => None,
|
||||||
n => Some(n),
|
n => Some(n),
|
||||||
},
|
},
|
||||||
extracted: self.extracted,
|
// Earlier runs' rows count once the pass has counted them; until
|
||||||
extract_total: self.extract_total,
|
// then only this run's, so the figure never goes backwards.
|
||||||
|
extracted: totals.map_or(self.written, |t| t.already_done + self.written),
|
||||||
|
extract_total: totals.map(|t| t.pending + t.already_done),
|
||||||
current_file: self.current_file.clone(),
|
current_file: self.current_file.clone(),
|
||||||
active_workers,
|
active_workers,
|
||||||
total_workers,
|
total_workers,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Drain up to one quantum of walk events into the pending batches,
|
/// Drain walk events into the pending batches for up to one slice,
|
||||||
/// finishing the walk if it ends. Returns whether anything happened.
|
/// finishing the walk if it ends. Returns whether anything happened.
|
||||||
fn service_walking(&mut self, cx: &mut RunCx<'_>) -> Result<bool, String> {
|
///
|
||||||
|
/// Batches still land per quantum; the slice only decides how many of
|
||||||
|
/// them one turn may write. A walk slower than the writer ends its turn at
|
||||||
|
/// `Empty` well inside the slice; only a walk that has the writer
|
||||||
|
/// saturated uses all of it.
|
||||||
|
pub(super) fn service_walking(&mut self, cx: &mut RunCx<'_>) -> Result<bool, String> {
|
||||||
|
let deadline = Instant::now() + cx.slice;
|
||||||
let mut took = 0usize;
|
let mut took = 0usize;
|
||||||
let mut finished = false;
|
let mut finished = false;
|
||||||
while took < cx.quantum {
|
while !finished {
|
||||||
|
let quantum_end = took + cx.quantum;
|
||||||
|
let more = self.walk_quantum(cx, &mut took, quantum_end, &mut finished)?;
|
||||||
|
if !more || Instant::now() >= deadline {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(finished || took > 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One quantum of [`RootPipeline::service_walking`]. Returns whether the
|
||||||
|
/// channel still had events when the quantum ended — false on `Empty` or
|
||||||
|
/// on the walk finishing.
|
||||||
|
fn walk_quantum(
|
||||||
|
&mut self,
|
||||||
|
cx: &mut RunCx<'_>,
|
||||||
|
took: &mut usize,
|
||||||
|
quantum_end: usize,
|
||||||
|
finished: &mut bool,
|
||||||
|
) -> Result<bool, String> {
|
||||||
|
while *took < quantum_end {
|
||||||
match self.walk.try_next() {
|
match self.walk.try_next() {
|
||||||
TryNext::Item(WalkEvent::Stale(paths)) => {
|
TryNext::Item(WalkEvent::Stale(paths)) => {
|
||||||
took += 1;
|
*took += 1;
|
||||||
// Applied at the end of the run: deleting mid-walk would
|
// Applied at the end of the run: deleting mid-walk would
|
||||||
// break "a stopped run deletes nothing", and an aliased
|
// break "a stopped run deletes nothing", and an aliased
|
||||||
// sighting that exempts a path may still be ahead.
|
// sighting that exempts a path may still be ahead.
|
||||||
cx.stale_candidates.extend(paths);
|
cx.stale_candidates.extend(paths);
|
||||||
}
|
}
|
||||||
TryNext::Item(WalkEvent::File(file)) => {
|
TryNext::Item(WalkEvent::File(file)) => {
|
||||||
took += 1;
|
*took += 1;
|
||||||
self.walked += 1;
|
self.walked += 1;
|
||||||
if self.walked.is_multiple_of(64) {
|
if self.walked.is_multiple_of(64) {
|
||||||
self.current_file = Some(file.path.clone());
|
self.current_file = Some(file.path.clone());
|
||||||
|
|
@ -247,15 +303,15 @@ impl RootPipeline {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
TryNext::Empty => break,
|
TryNext::Empty => return Ok(false),
|
||||||
TryNext::Finished => {
|
TryNext::Finished => {
|
||||||
self.finish_walk(cx)?;
|
self.finish_walk(cx)?;
|
||||||
finished = true;
|
*finished = true;
|
||||||
break;
|
return Ok(false);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Ok(finished || took > 0)
|
Ok(true)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The walk ended: land the buffered batches, then either hand the root
|
/// The walk ended: land the buffered batches, then either hand the root
|
||||||
|
|
@ -314,42 +370,60 @@ impl RootPipeline {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
let cursor = ExtractCursor::for_root(&self.root);
|
let cursor = ExtractCursor::for_root(&self.root);
|
||||||
let scope = extract_scope_prepare(&cx.conn_mutex, &cursor, cx.config)?;
|
// Only the sweep runs on the writer. Counting the range is the
|
||||||
// Progress counts the root's whole searchable set: files extracted
|
// pass's own job, on its own connection: on a large root it is
|
||||||
// in earlier runs start the counter, so an unchanged root shows
|
// seconds, and here that was seconds of every other root's walk
|
||||||
// "X of X" rather than "0 of 0".
|
// standing still.
|
||||||
self.extract_total = scope.pending + scope.already_done;
|
{
|
||||||
self.extracted = scope.already_done;
|
let conn = crate::lock_ok(&cx.conn_mutex);
|
||||||
if scope.pending == 0 {
|
mark_oversize_pending_na(&conn, &cursor, cx.config)?;
|
||||||
self.phase = RootPhase::Done;
|
}
|
||||||
} else {
|
self.totals = None;
|
||||||
// Starts only now: the rows have to exist before the feeder
|
self.written = 0;
|
||||||
// can page over them.
|
self.ready.clear();
|
||||||
|
// Starts only now: the rows have to exist before the feeder can
|
||||||
|
// page over them. Started even when nothing may be pending — the
|
||||||
|
// count that would say so is the pass's — and an empty range
|
||||||
|
// finishes on its own next turn.
|
||||||
self.content = Some(crate::content::extract_content(
|
self.content = Some(crate::content::extract_content(
|
||||||
cx.db_path,
|
cx.db_path,
|
||||||
&cursor,
|
&cursor,
|
||||||
cx.registry.clone(),
|
cx.registry.clone(),
|
||||||
cx.config.clone(),
|
cx.config.clone(),
|
||||||
cx.stop_flag.clone(),
|
cx.stop_flag.clone(),
|
||||||
cx.suspend_flag.clone(),
|
|
||||||
self.workers,
|
self.workers,
|
||||||
));
|
));
|
||||||
self.phase = RootPhase::Extracting;
|
self.phase = RootPhase::Extracting;
|
||||||
}
|
}
|
||||||
}
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Drain up to a quantum of finished extraction work, then write it;
|
/// Write finished extraction work for up to one slice; extraction itself
|
||||||
/// extraction runs on this root's own pool. Returns whether anything
|
/// runs on this root's own pool. Returns whether anything happened.
|
||||||
/// happened.
|
///
|
||||||
fn service_extracting(&mut self, cx: &mut RunCx<'_>) -> Result<bool, String> {
|
/// Rows the slice does not reach stay in `ready` for the next turn, and
|
||||||
let pass = self.content.as_mut().expect("extracting root has a pass");
|
/// the pass is not declared done until they have all landed.
|
||||||
let mut batch: Vec<crate::content::ExtractedRow> = Vec::new();
|
pub(super) fn service_extracting(&mut self, cx: &mut RunCx<'_>) -> Result<bool, String> {
|
||||||
|
let deadline = Instant::now() + cx.slice;
|
||||||
let mut finished = false;
|
let mut finished = false;
|
||||||
while batch.len() < cx.quantum {
|
let mut consumed = 0usize;
|
||||||
|
// Disjoint borrows: the pass is held across the store.
|
||||||
|
let Self {
|
||||||
|
content,
|
||||||
|
ready,
|
||||||
|
written,
|
||||||
|
totals,
|
||||||
|
current_file,
|
||||||
|
..
|
||||||
|
} = self;
|
||||||
|
let pass = content.as_mut().expect("extracting root has a pass");
|
||||||
|
if totals.is_none() {
|
||||||
|
*totals = pass.totals();
|
||||||
|
}
|
||||||
|
loop {
|
||||||
|
while ready.len() < READY_TOPUP {
|
||||||
match pass.try_next() {
|
match pass.try_next() {
|
||||||
TryNext::Item(row) => batch.push(row),
|
TryNext::Item(row) => ready.push(row),
|
||||||
TryNext::Empty => break,
|
TryNext::Empty => break,
|
||||||
TryNext::Finished => {
|
TryNext::Finished => {
|
||||||
finished = true;
|
finished = true;
|
||||||
|
|
@ -357,12 +431,28 @@ impl RootPipeline {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if let Some(row) = batch.last() {
|
if ready.is_empty() {
|
||||||
self.current_file = Some(row.name.clone());
|
break;
|
||||||
|
}
|
||||||
|
let stored = store_extracted(&cx.conn_mutex, ready, cx.stop_flag, cx.config, deadline)?;
|
||||||
|
if stored.consumed > 0 {
|
||||||
|
// The last row *written*, not the last fetched: with leftovers
|
||||||
|
// the two can be a slice apart.
|
||||||
|
*current_file = Some(ready[stored.consumed - 1].name.clone());
|
||||||
|
}
|
||||||
|
ready.drain(..stored.consumed);
|
||||||
|
*written += stored.written;
|
||||||
|
consumed += stored.consumed;
|
||||||
|
// Stopped, out of time, or still holding rows the deadline cut
|
||||||
|
// short — the next turn takes it from here.
|
||||||
|
if stored.consumed == 0 || !ready.is_empty() || Instant::now() >= deadline {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if finished && ready.is_empty() {
|
||||||
|
if totals.is_none() {
|
||||||
|
*totals = pass.totals();
|
||||||
}
|
}
|
||||||
let took = batch.len();
|
|
||||||
self.extracted += store_extracted(&cx.conn_mutex, &batch, cx.stop_flag, cx.config)?;
|
|
||||||
if finished {
|
|
||||||
// Join before deciding; see `ParallelWalk::finish`.
|
// Join before deciding; see `ParallelWalk::finish`.
|
||||||
if !pass.finish() {
|
if !pass.finish() {
|
||||||
crate::log_warn!("a content worker for {} terminated abnormally", self.root);
|
crate::log_warn!("a content worker for {} terminated abnormally", self.root);
|
||||||
|
|
@ -370,39 +460,67 @@ impl RootPipeline {
|
||||||
self.content = None;
|
self.content = None;
|
||||||
self.phase = RootPhase::Done;
|
self.phase = RootPhase::Done;
|
||||||
let extract_time = self.phase_elapsed();
|
let extract_time = self.phase_elapsed();
|
||||||
|
// Quiet for the pass that found nothing to do: every root passes
|
||||||
|
// through here now, changed or not.
|
||||||
|
if self.written > 0 {
|
||||||
crate::log_info!(
|
crate::log_info!(
|
||||||
"{}: content done — {}",
|
"{}: content done — {}",
|
||||||
self.root,
|
self.root,
|
||||||
phase_summary(self.extracted, "files with text", extract_time)
|
phase_summary(self.written, "files with text", extract_time)
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
Ok(finished || took > 0)
|
}
|
||||||
|
Ok(finished || consumed > 0)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One run's shared environment and cross-root state, threaded through the
|
/// One run's shared environment and cross-root state, threaded through the
|
||||||
/// per-phase [`RootPipeline`] service methods.
|
/// per-phase [`RootPipeline`] service methods.
|
||||||
struct RunCx<'a> {
|
pub(super) struct RunCx<'a> {
|
||||||
|
pub(super) conn_mutex: Arc<Mutex<Connection>>,
|
||||||
|
pub(super) config: &'a Config,
|
||||||
|
pub(super) db_path: &'a str,
|
||||||
|
pub(super) stop_flag: &'a Arc<AtomicBool>,
|
||||||
|
/// Shared with every root's walk workers, which use it to finish small
|
||||||
|
/// text files without handing them to the content pass.
|
||||||
|
pub(super) registry: Arc<Registry>,
|
||||||
|
pub(super) quantum: usize,
|
||||||
|
/// See [`TURN_SLICE`]; a field so tests can shrink it.
|
||||||
|
pub(super) slice: Duration,
|
||||||
|
/// 128-bit path digests, not paths: at millions of files, owning every
|
||||||
|
/// path string again was the single largest allocation in a run. See
|
||||||
|
/// `walk::path_digest`.
|
||||||
|
pub(super) seen_paths: HashSet<u128>,
|
||||||
|
/// Rows the per-directory reconciliation found no file behind, plus
|
||||||
|
/// whatever the vanished-directory sweep adds once the walks end.
|
||||||
|
pub(super) stale_candidates: Vec<String>,
|
||||||
|
/// Paths reached by resolving a symlink, whose row lives under a parent
|
||||||
|
/// that may be outside every root.
|
||||||
|
pub(super) aliased_paths: HashSet<String>,
|
||||||
|
pub(super) stale_cleanup_ok: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<'a> RunCx<'a> {
|
||||||
|
pub(super) fn new(
|
||||||
conn_mutex: Arc<Mutex<Connection>>,
|
conn_mutex: Arc<Mutex<Connection>>,
|
||||||
config: &'a Config,
|
config: &'a Config,
|
||||||
db_path: &'a str,
|
db_path: &'a str,
|
||||||
stop_flag: &'a Arc<AtomicBool>,
|
stop_flag: &'a Arc<AtomicBool>,
|
||||||
suspend_flag: &'a Arc<AtomicBool>,
|
) -> RunCx<'a> {
|
||||||
/// Shared with every root's walk workers, which use it to finish small
|
RunCx {
|
||||||
/// text files without handing them to the content pass.
|
conn_mutex,
|
||||||
registry: Arc<Registry>,
|
config,
|
||||||
quantum: usize,
|
db_path,
|
||||||
/// 128-bit path digests, not paths: at millions of files, owning every
|
stop_flag,
|
||||||
/// path string again was the single largest allocation in a run. See
|
registry: Arc::new(Registry::default_set()),
|
||||||
/// `walk::path_digest`.
|
quantum: config.processing.batch_size.max(1),
|
||||||
seen_paths: HashSet<u128>,
|
slice: TURN_SLICE,
|
||||||
/// Rows the per-directory reconciliation found no file behind, plus
|
seen_paths: HashSet::new(),
|
||||||
/// whatever the vanished-directory sweep adds once the walks end.
|
stale_candidates: Vec::new(),
|
||||||
stale_candidates: Vec<String>,
|
aliased_paths: HashSet::new(),
|
||||||
/// Paths reached by resolving a symlink, whose row lives under a parent
|
stale_cleanup_ok: true,
|
||||||
/// that may be outside every root.
|
}
|
||||||
aliased_paths: HashSet<String>,
|
}
|
||||||
stale_cleanup_ok: bool,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Publish a status snapshot. Never clobbers Stopping — the command thread
|
/// Publish a status snapshot. Never clobbers Stopping — the command thread
|
||||||
|
|
@ -447,7 +565,6 @@ fn build_pipeline(
|
||||||
cx.config.clone(),
|
cx.config.clone(),
|
||||||
cx.registry.clone(),
|
cx.registry.clone(),
|
||||||
cx.stop_flag.clone(),
|
cx.stop_flag.clone(),
|
||||||
cx.suspend_flag.clone(),
|
|
||||||
workers,
|
workers,
|
||||||
);
|
);
|
||||||
|
|
||||||
|
|
@ -491,8 +608,9 @@ fn build_pipeline(
|
||||||
phase: RootPhase::Walking,
|
phase: RootPhase::Walking,
|
||||||
workers,
|
workers,
|
||||||
content: None,
|
content: None,
|
||||||
extract_total: 0,
|
ready: Vec::new(),
|
||||||
extracted: 0,
|
written: 0,
|
||||||
|
totals: None,
|
||||||
current_file: None,
|
current_file: None,
|
||||||
phase_started: Instant::now(),
|
phase_started: Instant::now(),
|
||||||
})
|
})
|
||||||
|
|
@ -557,7 +675,6 @@ fn cleanup_stale(pipelines: &mut [RootPipeline], cx: &mut RunCx<'_>) -> Result<(
|
||||||
&cx.conn_mutex,
|
&cx.conn_mutex,
|
||||||
stale_paths.as_slice(),
|
stale_paths.as_slice(),
|
||||||
cx.stop_flag,
|
cx.stop_flag,
|
||||||
cx.suspend_flag,
|
|
||||||
cx.config,
|
cx.config,
|
||||||
)?;
|
)?;
|
||||||
crate::log_info!(
|
crate::log_info!(
|
||||||
|
|
@ -575,7 +692,6 @@ impl IndexingService {
|
||||||
paths: &[String],
|
paths: &[String],
|
||||||
db_path: &str,
|
db_path: &str,
|
||||||
stop_flag: &Arc<AtomicBool>,
|
stop_flag: &Arc<AtomicBool>,
|
||||||
suspend_flag: &Arc<AtomicBool>,
|
|
||||||
config: &Config,
|
config: &Config,
|
||||||
db_connection: &Arc<Mutex<Option<Arc<Mutex<Connection>>>>>,
|
db_connection: &Arc<Mutex<Option<Arc<Mutex<Connection>>>>>,
|
||||||
interrupt: &db::InterruptSlot,
|
interrupt: &db::InterruptSlot,
|
||||||
|
|
@ -637,19 +753,7 @@ impl IndexingService {
|
||||||
let count_cancel = Arc::new(AtomicBool::new(false));
|
let count_cancel = Arc::new(AtomicBool::new(false));
|
||||||
let _count_guard = CancelOnDrop(count_cancel.clone());
|
let _count_guard = CancelOnDrop(count_cancel.clone());
|
||||||
|
|
||||||
let mut cx = RunCx {
|
let mut cx = RunCx::new(conn_mutex, config, db_path, stop_flag);
|
||||||
conn_mutex,
|
|
||||||
config,
|
|
||||||
db_path,
|
|
||||||
stop_flag,
|
|
||||||
suspend_flag,
|
|
||||||
registry: Arc::new(Registry::default_set()),
|
|
||||||
quantum: config.processing.batch_size.max(1),
|
|
||||||
seen_paths: HashSet::new(),
|
|
||||||
stale_candidates: Vec::new(),
|
|
||||||
aliased_paths: HashSet::new(),
|
|
||||||
stale_cleanup_ok: true,
|
|
||||||
};
|
|
||||||
|
|
||||||
// Read stored counts up front, under one lock, before the walks
|
// Read stored counts up front, under one lock, before the walks
|
||||||
// compete for the connection.
|
// compete for the connection.
|
||||||
|
|
@ -689,10 +793,19 @@ impl IndexingService {
|
||||||
};
|
};
|
||||||
let mut checkpoint_at = wal_cap;
|
let mut checkpoint_at = wal_cap;
|
||||||
|
|
||||||
// Round-robin with skipping: each round takes at most one quantum
|
// Walks first, one slice each, then a single extraction slice.
|
||||||
// from every root that has work ready.
|
//
|
||||||
|
// The walk is the disk-bound phase and the one whose stall shows: its
|
||||||
|
// workers can only run as far ahead as their channel, so a writer that
|
||||||
|
// does not come back to it soon enough parks a whole pool behind one
|
||||||
|
// root's tokenizing. Serving every walking root before any extraction
|
||||||
|
// caps a walk's wait at one slice per round; taking one extraction
|
||||||
|
// slice per round, not one per root, keeps that cap independent of
|
||||||
|
// how many roots are extracting — while still handing extraction a
|
||||||
|
// slice every round, so it is never starved either. Any root's turn
|
||||||
|
// ends early the moment it has nothing ready.
|
||||||
loop {
|
loop {
|
||||||
if should_abort(stop_flag, suspend_flag) {
|
if stop_flag.load(Ordering::Relaxed) {
|
||||||
aborted = true;
|
aborted = true;
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
@ -700,11 +813,16 @@ impl IndexingService {
|
||||||
let n = pipelines.len();
|
let n = pipelines.len();
|
||||||
for k in 0..n {
|
for k in 0..n {
|
||||||
let p = &mut pipelines[(rr + k) % n];
|
let p = &mut pipelines[(rr + k) % n];
|
||||||
progressed |= match p.phase {
|
if p.phase == RootPhase::Walking {
|
||||||
RootPhase::Walking => p.service_walking(&mut cx)?,
|
progressed |= p.service_walking(&mut cx)?;
|
||||||
RootPhase::Extracting => p.service_extracting(&mut cx)?,
|
}
|
||||||
RootPhase::Done => false,
|
}
|
||||||
};
|
for k in 0..n {
|
||||||
|
let p = &mut pipelines[(rr + k) % n];
|
||||||
|
if p.phase == RootPhase::Extracting {
|
||||||
|
progressed |= p.service_extracting(&mut cx)?;
|
||||||
|
break;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
rr = rr.wrapping_add(1);
|
rr = rr.wrapping_add(1);
|
||||||
|
|
||||||
|
|
@ -766,15 +884,19 @@ impl IndexingService {
|
||||||
}
|
}
|
||||||
|
|
||||||
if aborted {
|
if aborted {
|
||||||
// Buffered records are valid work — land them before leaving.
|
// Nothing is landed on the way out, and there used to be a loop
|
||||||
for p in &mut pipelines {
|
// here that looked as though it did: `aborted` implies the stop
|
||||||
process_batch_updates(&cx.conn_mutex, &p.pending_updates, stop_flag, config)?;
|
// flag is set, and both batch writers return on it before their
|
||||||
p.pending_updates.clear();
|
// first chunk, so it wrote nothing. What a stop drops is each
|
||||||
process_batch_inserts(&cx.conn_mutex, &p.pending_inserts, stop_flag, config)?;
|
// root's part-filled insert/update batch (under `batch_size`
|
||||||
p.pending_inserts.clear();
|
// rows) and whatever extraction had ready — all of it still
|
||||||
}
|
// `content_state = 0` or absent, so the next run finds it again.
|
||||||
// No stale cleanup: a partial walk's seen set would delete most
|
// That is what "a stopped run promises nothing" already means,
|
||||||
// of the index.
|
// and it is cheaper than tokenizing a slice's worth of documents
|
||||||
|
// while someone waits for the window to close.
|
||||||
|
//
|
||||||
|
// No stale cleanup either: a partial walk's seen set would delete
|
||||||
|
// most of the index.
|
||||||
report_run_warnings();
|
report_run_warnings();
|
||||||
crate::log_info!(
|
crate::log_info!(
|
||||||
"indexing stopped after {:.1}s",
|
"indexing stopped after {:.1}s",
|
||||||
|
|
|
||||||
|
|
@ -26,12 +26,15 @@ pub struct RootProgress {
|
||||||
/// counts tree *entries* and so reads high. Read it through
|
/// counts tree *entries* and so reads high. Read it through
|
||||||
/// [`RootProgress::walk_denominator`].
|
/// [`RootProgress::walk_denominator`].
|
||||||
pub walk_total: Option<usize>,
|
pub walk_total: Option<usize>,
|
||||||
/// Rows with searchable text: extracted in earlier runs plus this one.
|
/// Rows with searchable text: this run's, plus earlier runs' once
|
||||||
|
/// `extract_total` is known.
|
||||||
pub extracted: usize,
|
pub extracted: usize,
|
||||||
/// The root's whole searchable set: pending + already-extracted rows when
|
/// The root's whole searchable set: pending + already-extracted rows when
|
||||||
/// the walk finished — the count of files that have or will have text,
|
/// the walk finished — the count of files that have or will have text,
|
||||||
/// not of files under the root.
|
/// not of files under the root. `None` until the root's content pass has
|
||||||
pub extract_total: usize,
|
/// counted its range: a scan that takes seconds on a large root, and one
|
||||||
|
/// that used to run on the writer thread with every other root waiting.
|
||||||
|
pub extract_total: Option<usize>,
|
||||||
pub current_file: Option<String>,
|
pub current_file: Option<String>,
|
||||||
/// Threads busy right now / pool size, for the pool this root's current
|
/// Threads busy right now / pool size, for the pool this root's current
|
||||||
/// phase is running. Both zero once the root is done: its threads are
|
/// phase is running. Both zero once the root is done: its threads are
|
||||||
|
|
@ -80,14 +83,22 @@ impl OverallProgress {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Aggregate every root's progress into the one pair the status bar shows.
|
/// Aggregate every root's progress into the one pair the status bar shows.
|
||||||
/// `extract_total` is exact the moment a root's walk ends; before that a
|
///
|
||||||
/// root contributes only its walk.
|
/// A root contributes its extraction half only once `extract_total` is known
|
||||||
|
/// — both to `processed` and to `total`, so the two stay in step. Until then
|
||||||
|
/// (during the walk, and for the moments after it while the pass counts) it
|
||||||
|
/// contributes its walk alone.
|
||||||
pub fn overall_progress(roots: &[RootProgress]) -> OverallProgress {
|
pub fn overall_progress(roots: &[RootProgress]) -> OverallProgress {
|
||||||
let processed = roots.iter().map(|r| r.walked + r.extracted).sum();
|
let processed = roots
|
||||||
|
.iter()
|
||||||
|
.map(|r| r.walked + r.extract_total.map_or(0, |_| r.extracted))
|
||||||
|
.sum();
|
||||||
let mut total = Some(0usize);
|
let mut total = Some(0usize);
|
||||||
for r in roots {
|
for r in roots {
|
||||||
match (total, r.walk_denominator()) {
|
match (total, r.walk_denominator()) {
|
||||||
(Some(acc), Some(walk)) => total = Some(acc + walk + r.extract_total),
|
(Some(acc), Some(walk)) => {
|
||||||
|
total = Some(acc + walk + r.extract_total.unwrap_or(0));
|
||||||
|
}
|
||||||
_ => {
|
_ => {
|
||||||
total = None;
|
total = None;
|
||||||
break;
|
break;
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,7 @@ use crate::extract::Registry;
|
||||||
use crate::file_handling::ExtractCursor;
|
use crate::file_handling::ExtractCursor;
|
||||||
use crate::walk::walk_indexable_files;
|
use crate::walk::walk_indexable_files;
|
||||||
use std::sync::atomic::AtomicUsize;
|
use std::sync::atomic::AtomicUsize;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
fn tmp_dir(tag: &str) -> std::path::PathBuf {
|
fn tmp_dir(tag: &str) -> std::path::PathBuf {
|
||||||
// Canonical: the temp dir itself may sit behind a symlink
|
// Canonical: the temp dir itself may sit behind a symlink
|
||||||
|
|
@ -86,7 +87,6 @@ fn worker_counts_follow_the_phase() {
|
||||||
|
|
||||||
let root = dir.to_string_lossy().into_owned();
|
let root = dir.to_string_lossy().into_owned();
|
||||||
let stop = Arc::new(AtomicBool::new(false));
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
let suspend = Arc::new(AtomicBool::new(false));
|
|
||||||
let walk = walk_indexable_files(
|
let walk = walk_indexable_files(
|
||||||
std::slice::from_ref(&root),
|
std::slice::from_ref(&root),
|
||||||
false,
|
false,
|
||||||
|
|
@ -96,7 +96,6 @@ fn worker_counts_follow_the_phase() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
stop.clone(),
|
stop.clone(),
|
||||||
suspend.clone(),
|
|
||||||
3,
|
3,
|
||||||
);
|
);
|
||||||
// An empty range, so the pass ends immediately — but its pool size is
|
// An empty range, so the pass ends immediately — but its pool size is
|
||||||
|
|
@ -107,7 +106,6 @@ fn worker_counts_follow_the_phase() {
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Config::default(),
|
Config::default(),
|
||||||
stop,
|
stop,
|
||||||
suspend,
|
|
||||||
2,
|
2,
|
||||||
);
|
);
|
||||||
|
|
||||||
|
|
@ -123,8 +121,9 @@ fn worker_counts_follow_the_phase() {
|
||||||
phase: RootPhase::Walking,
|
phase: RootPhase::Walking,
|
||||||
phase_started: Instant::now(),
|
phase_started: Instant::now(),
|
||||||
content: Some(content),
|
content: Some(content),
|
||||||
extract_total: 0,
|
ready: Vec::new(),
|
||||||
extracted: 0,
|
written: 0,
|
||||||
|
totals: None,
|
||||||
current_file: None,
|
current_file: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -138,6 +137,157 @@ fn worker_counts_follow_the_phase() {
|
||||||
std::fs::remove_dir_all(&dir).ok();
|
std::fs::remove_dir_all(&dir).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The writer's extraction turn is bounded by its slice, not by what is
|
||||||
|
/// ready: rows the slice does not reach are carried to the next turn, and the
|
||||||
|
/// root is not `Done` until they have all landed. Pinned with a zero slice,
|
||||||
|
/// under which every turn writes exactly one row.
|
||||||
|
#[test]
|
||||||
|
fn an_extracting_turn_lands_its_leftovers_one_slice_at_a_time() {
|
||||||
|
use super::pipeline::RunCx;
|
||||||
|
use crate::content::ExtractedRow;
|
||||||
|
use crate::db::repo::{insert_file, NewFile};
|
||||||
|
use crate::file_handling::ContentOutcome;
|
||||||
|
use crate::mime::FileType;
|
||||||
|
|
||||||
|
let dir = tmp_dir("slice-leftovers");
|
||||||
|
let db_path = dir.join("index.db").to_string_lossy().into_owned();
|
||||||
|
let mut conn = db::open_or_recreate(&db_path, "trigram").unwrap();
|
||||||
|
let tree = dir.join("tree");
|
||||||
|
std::fs::create_dir_all(&tree).unwrap();
|
||||||
|
// Five rows the walk would have written, whose extracted text is
|
||||||
|
// hand-built below rather than read back — the pass is not the subject.
|
||||||
|
let mut ready: Vec<ExtractedRow> = Vec::new();
|
||||||
|
{
|
||||||
|
let tx = conn.transaction().unwrap();
|
||||||
|
for i in 0..5 {
|
||||||
|
let path = tree.join(format!("f{}.txt", i));
|
||||||
|
std::fs::write(&path, "sphinx of black quartz").unwrap();
|
||||||
|
let file_id = insert_file(
|
||||||
|
&tx,
|
||||||
|
&NewFile {
|
||||||
|
name: &format!("f{}.txt", i),
|
||||||
|
path: &path.to_string_lossy(),
|
||||||
|
parent: &tree.to_string_lossy(),
|
||||||
|
size: 22,
|
||||||
|
mtime: 1,
|
||||||
|
inode: None,
|
||||||
|
device_id: None,
|
||||||
|
mime: Some("text/plain"),
|
||||||
|
ftype: FileType::TEXT,
|
||||||
|
hash: None,
|
||||||
|
needs_content: true,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
.expect("unique path");
|
||||||
|
ready.push(ExtractedRow {
|
||||||
|
file_id,
|
||||||
|
name: format!("f{}.txt", i),
|
||||||
|
outcome: ContentOutcome::Done {
|
||||||
|
text: format!("sphinx of black quartz {}", i),
|
||||||
|
properties: Vec::new(),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
tx.commit().unwrap();
|
||||||
|
}
|
||||||
|
let conn_mutex = Arc::new(Mutex::new(conn));
|
||||||
|
|
||||||
|
let root = dir.to_string_lossy().into_owned();
|
||||||
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
|
let config = Config::default();
|
||||||
|
let walk = walk_indexable_files(
|
||||||
|
std::slice::from_ref(&root),
|
||||||
|
false,
|
||||||
|
false,
|
||||||
|
crate::config::IgnoreSet::compile(&[]).unwrap(),
|
||||||
|
&db_path,
|
||||||
|
config.clone(),
|
||||||
|
Arc::new(Registry::default_set()),
|
||||||
|
stop.clone(),
|
||||||
|
1,
|
||||||
|
);
|
||||||
|
// An empty range: the pass reports `Finished` on its own, and the turn
|
||||||
|
// has to keep going past that until `ready` is empty.
|
||||||
|
let content = crate::content::extract_content(
|
||||||
|
&db_path,
|
||||||
|
&ExtractCursor::for_root(&dir.join("nothing").to_string_lossy()),
|
||||||
|
Arc::new(Registry::default_set()),
|
||||||
|
config.clone(),
|
||||||
|
stop.clone(),
|
||||||
|
1,
|
||||||
|
);
|
||||||
|
let mut p = RootPipeline {
|
||||||
|
root,
|
||||||
|
walk,
|
||||||
|
count_total: Arc::new(AtomicUsize::new(0)),
|
||||||
|
workers: 1,
|
||||||
|
pending_updates: Vec::new(),
|
||||||
|
pending_inserts: Vec::new(),
|
||||||
|
walked: 0,
|
||||||
|
walk_clean: true,
|
||||||
|
phase: RootPhase::Extracting,
|
||||||
|
phase_started: Instant::now(),
|
||||||
|
content: Some(content),
|
||||||
|
ready,
|
||||||
|
written: 0,
|
||||||
|
totals: None,
|
||||||
|
current_file: None,
|
||||||
|
};
|
||||||
|
let mut cx = RunCx::new(conn_mutex.clone(), &config, &db_path, &stop);
|
||||||
|
cx.slice = Duration::ZERO;
|
||||||
|
|
||||||
|
let mut turns = 0;
|
||||||
|
while p.phase == RootPhase::Extracting {
|
||||||
|
turns += 1;
|
||||||
|
assert!(
|
||||||
|
turns < 200,
|
||||||
|
"the root never finished: {} written",
|
||||||
|
p.written
|
||||||
|
);
|
||||||
|
let before = p.written;
|
||||||
|
let progressed = p.service_extracting(&mut cx).unwrap();
|
||||||
|
assert!(
|
||||||
|
p.written - before <= 1,
|
||||||
|
"a zero slice wrote {} rows in one turn",
|
||||||
|
p.written - before
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
p.phase != RootPhase::Done || p.ready.is_empty(),
|
||||||
|
"Done with {} rows still to write",
|
||||||
|
p.ready.len()
|
||||||
|
);
|
||||||
|
if !progressed {
|
||||||
|
// The empty pass has not reported `Finished` yet.
|
||||||
|
std::thread::sleep(Duration::from_millis(1));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert_eq!(p.written, 5);
|
||||||
|
assert!(p.ready.is_empty());
|
||||||
|
assert!(
|
||||||
|
turns >= 5,
|
||||||
|
"five rows cannot land in {} zero-slice turns",
|
||||||
|
turns
|
||||||
|
);
|
||||||
|
let done: i64 = conn_mutex
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM files WHERE content_state = 1",
|
||||||
|
[],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(done, 5, "every row reached the index");
|
||||||
|
// The empty pass counted its (empty) range, so the totals are known and
|
||||||
|
// the snapshot reports this run's rows on top of the range's zero.
|
||||||
|
assert_eq!(p.snapshot().extracted, 5);
|
||||||
|
assert_eq!(p.snapshot().extract_total, Some(0));
|
||||||
|
|
||||||
|
drop(p);
|
||||||
|
std::fs::remove_dir_all(&dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
/// One full run over `config`'s roots, driven directly so the caller owns
|
/// One full run over `config`'s roots, driven directly so the caller owns
|
||||||
/// the stop flag. Returns when the run does.
|
/// the stop flag. Returns when the run does.
|
||||||
fn run_with(config: &Config, db_path: &str, stop: &Arc<AtomicBool>) -> Result<(), String> {
|
fn run_with(config: &Config, db_path: &str, stop: &Arc<AtomicBool>) -> Result<(), String> {
|
||||||
|
|
@ -146,7 +296,6 @@ fn run_with(config: &Config, db_path: &str, stop: &Arc<AtomicBool>) -> Result<()
|
||||||
&config.paths.indexing_paths,
|
&config.paths.indexing_paths,
|
||||||
db_path,
|
db_path,
|
||||||
stop,
|
stop,
|
||||||
&Arc::new(AtomicBool::new(false)),
|
|
||||||
config,
|
config,
|
||||||
&Arc::new(Mutex::new(None)),
|
&Arc::new(Mutex::new(None)),
|
||||||
&db::InterruptSlot::default(),
|
&db::InterruptSlot::default(),
|
||||||
|
|
@ -374,7 +523,7 @@ fn progress(phase: RootPhase, walked: usize, walk_total: Option<usize>) -> RootP
|
||||||
walked,
|
walked,
|
||||||
walk_total,
|
walk_total,
|
||||||
extracted: 0,
|
extracted: 0,
|
||||||
extract_total: 0,
|
extract_total: None,
|
||||||
current_file: None,
|
current_file: None,
|
||||||
active_workers: 0,
|
active_workers: 0,
|
||||||
total_workers: 0,
|
total_workers: 0,
|
||||||
|
|
@ -426,7 +575,7 @@ fn overall_progress_sums_both_halves_of_every_root() {
|
||||||
let mut walking = progress(RootPhase::Walking, 100, Some(1000));
|
let mut walking = progress(RootPhase::Walking, 100, Some(1000));
|
||||||
let mut extracting = progress(RootPhase::Extracting, 500, Some(9999));
|
let mut extracting = progress(RootPhase::Extracting, 500, Some(9999));
|
||||||
extracting.extracted = 200;
|
extracting.extracted = 200;
|
||||||
extracting.extract_total = 400;
|
extracting.extract_total = Some(400);
|
||||||
walking.extracted = 0;
|
walking.extracted = 0;
|
||||||
|
|
||||||
let o = overall_progress(&[walking, extracting]);
|
let o = overall_progress(&[walking, extracting]);
|
||||||
|
|
@ -435,6 +584,19 @@ fn overall_progress_sums_both_halves_of_every_root() {
|
||||||
assert_eq!(o.total, Some(1900));
|
assert_eq!(o.total, Some(1900));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A root whose content pass has not counted its range yet contributes only
|
||||||
|
/// its walk — to both halves, so processed and total stay in step and the
|
||||||
|
/// bar cannot jump when the count lands.
|
||||||
|
#[test]
|
||||||
|
fn an_uncounted_extraction_contributes_only_its_walk() {
|
||||||
|
let mut counting = progress(RootPhase::Extracting, 500, None);
|
||||||
|
counting.extracted = 7;
|
||||||
|
counting.extract_total = None;
|
||||||
|
let o = overall_progress(&[counting]);
|
||||||
|
assert_eq!(o.processed, 500);
|
||||||
|
assert_eq!(o.total, Some(500));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn one_uncounted_walking_root_leaves_the_whole_total_unknown() {
|
fn one_uncounted_walking_root_leaves_the_whole_total_unknown() {
|
||||||
let known = progress(RootPhase::Done, 10, Some(10));
|
let known = progress(RootPhase::Done, 10, Some(10));
|
||||||
|
|
@ -472,7 +634,7 @@ fn a_finished_run_reaches_exactly_one_hundred_percent() {
|
||||||
.map(|&(walked, extracted)| {
|
.map(|&(walked, extracted)| {
|
||||||
let mut p = progress(RootPhase::Done, walked, Some(walked * 2));
|
let mut p = progress(RootPhase::Done, walked, Some(walked * 2));
|
||||||
p.extracted = extracted;
|
p.extracted = extracted;
|
||||||
p.extract_total = extracted;
|
p.extract_total = Some(extracted);
|
||||||
p
|
p
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
|
|
@ -496,6 +658,9 @@ fn a_run_with_nothing_to_do_has_no_fraction_to_show() {
|
||||||
fn the_fraction_never_exceeds_one() {
|
fn the_fraction_never_exceeds_one() {
|
||||||
let mut p = progress(RootPhase::Done, 10, None);
|
let mut p = progress(RootPhase::Done, 10, None);
|
||||||
p.extracted = 100;
|
p.extracted = 100;
|
||||||
|
// A counted scope the writes then overran; an uncounted one would be
|
||||||
|
// left out of both halves and prove nothing here.
|
||||||
|
p.extract_total = Some(0);
|
||||||
let o = overall_progress(&[p]);
|
let o = overall_progress(&[p]);
|
||||||
assert_eq!(o.processed, 110);
|
assert_eq!(o.processed, 110);
|
||||||
assert_eq!(o.total, Some(10));
|
assert_eq!(o.total, Some(10));
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,7 @@ pub mod extract;
|
||||||
pub mod file_handling;
|
pub mod file_handling;
|
||||||
pub mod incremental;
|
pub mod incremental;
|
||||||
pub mod indexing;
|
pub mod indexing;
|
||||||
|
pub mod live;
|
||||||
pub mod log;
|
pub mod log;
|
||||||
pub mod mime;
|
pub mod mime;
|
||||||
pub mod platform;
|
pub mod platform;
|
||||||
|
|
@ -19,6 +20,7 @@ pub mod snippet;
|
||||||
#[doc(hidden)]
|
#[doc(hidden)]
|
||||||
pub mod testutil;
|
pub mod testutil;
|
||||||
pub mod textenc;
|
pub mod textenc;
|
||||||
|
pub mod verify;
|
||||||
pub mod walk;
|
pub mod walk;
|
||||||
pub mod watcher;
|
pub mod watcher;
|
||||||
|
|
||||||
|
|
|
||||||
734
crates/quicksearch-core/src/live.rs
Normal file
734
crates/quicksearch-core/src/live.rs
Normal file
|
|
@ -0,0 +1,734 @@
|
||||||
|
//! Watching the search results a frontend is actually showing.
|
||||||
|
//!
|
||||||
|
//! What a row shows is read from the *file*, never from the index. That is
|
||||||
|
//! what lets this work with indexing stopped, with a file outside every
|
||||||
|
//! indexed root, or against a row the indexer has not caught up with yet —
|
||||||
|
//! and it is the whole point of the feature: the list on screen describes the
|
||||||
|
//! disk, not a snapshot of it.
|
||||||
|
//!
|
||||||
|
//! Nothing here writes to the index. Keeping it so is what lets this run
|
||||||
|
//! alongside the indexer without a second writer; a frontend that wants the
|
||||||
|
//! index brought back in line with what it just displayed hands the paths to
|
||||||
|
//! [`crate::coordinator::IndexCoordinator::update_paths`], which does the
|
||||||
|
//! write on its own thread.
|
||||||
|
//!
|
||||||
|
//! # Why directories, not files
|
||||||
|
//!
|
||||||
|
//! The obvious design is a watch per result file. It does not work. Editors
|
||||||
|
//! save by writing a temporary file and renaming it over the target, so the
|
||||||
|
//! event lands on the *directory* and the old inode — the one a file watch is
|
||||||
|
//! attached to — is simply orphaned. A file watch also cannot report the new
|
||||||
|
//! name of a rename. Watching the deduplicated set of parent directories
|
||||||
|
//! `NonRecursive` sees both, on inotify and on `ReadDirectoryChangesW` alike.
|
||||||
|
//!
|
||||||
|
//! # Why this is not [`crate::watcher`]
|
||||||
|
//!
|
||||||
|
//! That module exists to cover *subtrees*: it walks each root registering
|
||||||
|
//! every directory beneath it, adds directories that appear later, and backs a
|
||||||
|
//! 128k budget with an all-or-nothing guarantee. Pointing it at a result's
|
||||||
|
//! parent would register that parent's whole tree. This is a flat, fixed, tiny
|
||||||
|
//! set with no growth and no budget, and its timings are a tenth of that one's
|
||||||
|
//! — the indexer can afford to coalesce for thirty seconds, a cursor blinking
|
||||||
|
//! next to a stale filename cannot.
|
||||||
|
//!
|
||||||
|
//! # What an event turns into
|
||||||
|
//!
|
||||||
|
//! A rename is applied from the event itself. A content change is answered by
|
||||||
|
//! reading the file: `metadata` for size and modified time, and — for a row
|
||||||
|
//! whose cell shows body text — the same MIME sniffing and extractors the
|
||||||
|
//! indexer uses, re-cut through the same [`crate::search::cascade::text_snippet`]
|
||||||
|
//! (or, for a fuzzy hit, [`crate::search::cascade::fuzzy_snippet`]) the
|
||||||
|
//! search itself uses.
|
||||||
|
//!
|
||||||
|
//! Arming also sweeps every target once, comparing the file on disk against
|
||||||
|
//! the size and modified time the row is *currently displaying*. Since a fresh
|
||||||
|
//! result carries what the index said, that sweep is exactly a check of the
|
||||||
|
//! index against the disk, and it is what makes a row corrected while it was
|
||||||
|
//! scrolled out of view right itself the moment it comes back. It is also the
|
||||||
|
//! only thing that works on a filesystem the platform reports no events for.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::io::Read;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::sync::mpsc;
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::thread::{self, JoinHandle};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use notify::{
|
||||||
|
Config as NotifyConfig, Event as NotifyEvent, EventKind, RecommendedWatcher, RecursiveMode,
|
||||||
|
Watcher as NotifyWatcher,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::config::Config;
|
||||||
|
use crate::extract::Registry;
|
||||||
|
use crate::query::split::split_for_cascade;
|
||||||
|
use crate::search::fuzzy::{edit_budget, Bitap};
|
||||||
|
use crate::search::ContentTier;
|
||||||
|
use crate::snippet::Snippet;
|
||||||
|
|
||||||
|
/// How long events for one path are pooled before being acted on, so the
|
||||||
|
/// several notify emits behind a single save collapse into one update.
|
||||||
|
const SETTLE: Duration = Duration::from_millis(150);
|
||||||
|
|
||||||
|
/// Floor on how often any one path may produce an update. A file being written
|
||||||
|
/// in a loop — a log, a build artifact — cannot spin the UI.
|
||||||
|
const MIN_INTERVAL: Duration = Duration::from_millis(750);
|
||||||
|
|
||||||
|
/// How often the loop wakes while it has work pending. It blocks outright when
|
||||||
|
/// it has none, so an idle QuickSearch does not tick at all.
|
||||||
|
const TICK: Duration = Duration::from_millis(50);
|
||||||
|
|
||||||
|
/// Ceiling on updates emitted per tick, so a directory-wide change (an
|
||||||
|
/// unpack, a `chmod -R`) drains over several frames instead of one.
|
||||||
|
const MAX_PER_TICK: usize = 4;
|
||||||
|
|
||||||
|
/// Most watches to register. Results cluster hard — a query's hits usually
|
||||||
|
/// share a handful of directories — so this is generous for the visible rows
|
||||||
|
/// while staying negligible against the indexer's 128k budget.
|
||||||
|
const MAX_DIRS: usize = 64;
|
||||||
|
|
||||||
|
/// Most rows to track, whatever the frontend asks for.
|
||||||
|
const MAX_TARGETS: usize = 256;
|
||||||
|
|
||||||
|
/// One row the frontend is showing and wants kept current.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct Target {
|
||||||
|
/// The row's path, spelled exactly as the search returned it. Event paths
|
||||||
|
/// are compared against this byte for byte — see the note in [`watch`].
|
||||||
|
///
|
||||||
|
/// [`watch`]: LiveWatcher::watch
|
||||||
|
pub path: String,
|
||||||
|
/// `Some` when this row displays text from the file's *body*, and so
|
||||||
|
/// needs its snippet re-cut when the file changes — and how, since an
|
||||||
|
/// exact-tier window is cut around the literal term and a fuzzy one
|
||||||
|
/// around a bitap match the literal is usually absent from. `None` for a
|
||||||
|
/// filename or path match, which costs one `metadata` call per change and
|
||||||
|
/// never opens the file.
|
||||||
|
pub text: Option<ContentTier>,
|
||||||
|
/// The size the row is displaying. The arm-time sweep compares the file
|
||||||
|
/// against this, so on a fresh result — where it is whatever the index
|
||||||
|
/// said — the sweep doubles as a check of the index against the disk.
|
||||||
|
pub size: u64,
|
||||||
|
/// The modified time the row is displaying; see [`Target::size`].
|
||||||
|
pub mtime: i64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a change did to a row's Content Match window.
|
||||||
|
///
|
||||||
|
/// Three states, not an `Option`, because "I did not look" and "I looked and
|
||||||
|
/// it is not there any more" have to reach the frontend as different answers.
|
||||||
|
/// Blanking a cell because the file was too large to re-read would lose a
|
||||||
|
/// window the search legitimately found.
|
||||||
|
#[derive(Debug, Clone, PartialEq)]
|
||||||
|
pub enum WindowUpdate {
|
||||||
|
/// Nothing to say: not a body-text row, or its body could not be re-read
|
||||||
|
/// (too large, no extractor, unreadable). The cell keeps what it has.
|
||||||
|
Unchanged,
|
||||||
|
/// Re-cut from the file as it is now.
|
||||||
|
Cut(Snippet),
|
||||||
|
/// The body was read and the query is no longer in it. The cell has
|
||||||
|
/// nothing to show and falls back to its dash.
|
||||||
|
NoMatch,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A ready-to-apply change to one row on screen.
|
||||||
|
#[derive(Debug, Clone, PartialEq)]
|
||||||
|
pub enum LiveUpdate {
|
||||||
|
/// The file moved. `path` is the row's old path — the frontend's key.
|
||||||
|
Renamed {
|
||||||
|
path: String,
|
||||||
|
to: String,
|
||||||
|
name: String,
|
||||||
|
},
|
||||||
|
/// The file's contents changed, as read from the file itself.
|
||||||
|
Changed {
|
||||||
|
path: String,
|
||||||
|
size: u64,
|
||||||
|
mtime: i64,
|
||||||
|
window: WindowUpdate,
|
||||||
|
},
|
||||||
|
/// The file is no longer there. Reversible: the directory watch stays, so
|
||||||
|
/// a file recreated at the same path reports [`LiveUpdate::Changed`].
|
||||||
|
Gone { path: String },
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LiveUpdate {
|
||||||
|
/// The row this update is keyed by — the path the frontend knows it as.
|
||||||
|
pub fn path(&self) -> &str {
|
||||||
|
match self {
|
||||||
|
LiveUpdate::Renamed { path, .. }
|
||||||
|
| LiveUpdate::Changed { path, .. }
|
||||||
|
| LiveUpdate::Gone { path } => path,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What one settled event window decided about a path, before the filesystem
|
||||||
|
/// is consulted.
|
||||||
|
#[derive(Debug, Clone, PartialEq)]
|
||||||
|
enum Op {
|
||||||
|
Changed,
|
||||||
|
Renamed(PathBuf),
|
||||||
|
/// Provisional: on Linux the `From` half of a rename arrives before the
|
||||||
|
/// paired event that names the destination, so this may still be upgraded
|
||||||
|
/// to [`Op::Renamed`] inside the same window.
|
||||||
|
Gone,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Commands and events share one channel so the loop can block on `recv()`
|
||||||
|
/// whenever nothing is pending.
|
||||||
|
enum Msg {
|
||||||
|
Event(NotifyEvent),
|
||||||
|
Watch {
|
||||||
|
query: String,
|
||||||
|
targets: Vec<Target>,
|
||||||
|
/// Boxed: this is by far the largest variant, and a `Watch` is rare
|
||||||
|
/// next to the events sharing the channel with it.
|
||||||
|
config: Box<Config>,
|
||||||
|
},
|
||||||
|
Clear,
|
||||||
|
Stop,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Handle on the watcher thread. Dropping it stops the thread.
|
||||||
|
pub struct LiveWatcher {
|
||||||
|
tx: mpsc::Sender<Msg>,
|
||||||
|
handle: Option<JoinHandle<()>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LiveWatcher {
|
||||||
|
/// Spawn the watcher. `notify` is called after every update is queued, so
|
||||||
|
/// an egui frontend can `request_repaint`; pass a no-op for headless use.
|
||||||
|
pub fn start(notify: Arc<dyn Fn() + Send + Sync>) -> (LiveWatcher, mpsc::Receiver<LiveUpdate>) {
|
||||||
|
let (tx, rx) = mpsc::channel::<Msg>();
|
||||||
|
let (update_tx, update_rx) = mpsc::channel::<LiveUpdate>();
|
||||||
|
let event_tx = tx.clone();
|
||||||
|
let handle = thread::Builder::new()
|
||||||
|
.name("qs-live".into())
|
||||||
|
.spawn(move || {
|
||||||
|
Loop {
|
||||||
|
rx,
|
||||||
|
event_tx,
|
||||||
|
update_tx,
|
||||||
|
notify,
|
||||||
|
watcher: None,
|
||||||
|
targets: HashMap::new(),
|
||||||
|
pending: HashMap::new(),
|
||||||
|
last_emit: HashMap::new(),
|
||||||
|
orphan_to: Vec::new(),
|
||||||
|
settle_at: None,
|
||||||
|
query: None,
|
||||||
|
fuzzy: None,
|
||||||
|
config: None,
|
||||||
|
registry: Registry::default_set(),
|
||||||
|
}
|
||||||
|
.run()
|
||||||
|
})
|
||||||
|
.expect("spawn live watcher");
|
||||||
|
(
|
||||||
|
LiveWatcher {
|
||||||
|
tx,
|
||||||
|
handle: Some(handle),
|
||||||
|
},
|
||||||
|
update_rx,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Replace the watched set wholesale.
|
||||||
|
///
|
||||||
|
/// `query` is the search these rows came from; it is what a re-cut snippet
|
||||||
|
/// is marked against. `config` supplies the extraction limits and filters,
|
||||||
|
/// so a snippet cut here is the text the indexer would have stored.
|
||||||
|
/// Registration happens on the watcher's own thread, so this never blocks
|
||||||
|
/// the caller on a spun-down disk or a stale mount.
|
||||||
|
pub fn watch(&self, query: &str, targets: Vec<Target>, config: &Config) {
|
||||||
|
let _ = self.tx.send(Msg::Watch {
|
||||||
|
query: query.to_string(),
|
||||||
|
targets,
|
||||||
|
config: Box::new(config.clone()),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Drop every watch and forget every pending update.
|
||||||
|
pub fn clear(&self) {
|
||||||
|
let _ = self.tx.send(Msg::Clear);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stop the thread and join it. Idempotent; [`Drop`] calls it.
|
||||||
|
pub fn stop(&mut self) {
|
||||||
|
let _ = self.tx.send(Msg::Stop);
|
||||||
|
if let Some(handle) = self.handle.take() {
|
||||||
|
let _ = handle.join();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for LiveWatcher {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
self.stop();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Loop {
|
||||||
|
rx: mpsc::Receiver<Msg>,
|
||||||
|
event_tx: mpsc::Sender<Msg>,
|
||||||
|
update_tx: mpsc::Sender<LiveUpdate>,
|
||||||
|
notify: Arc<dyn Fn() + Send + Sync>,
|
||||||
|
watcher: Option<RecommendedWatcher>,
|
||||||
|
targets: HashMap<String, Target>,
|
||||||
|
pending: HashMap<String, Op>,
|
||||||
|
last_emit: HashMap<String, Instant>,
|
||||||
|
/// Rename destinations seen in this window whose source is not a target —
|
||||||
|
/// the other half of a Windows rename, which carries no pairing cookie.
|
||||||
|
orphan_to: Vec<PathBuf>,
|
||||||
|
settle_at: Option<Instant>,
|
||||||
|
query: Option<crate::query::split::CascadeQuery>,
|
||||||
|
/// The fuzzy matcher for `query`, built at arm time — building it is the
|
||||||
|
/// cost, running it is cheap — for re-cutting [`ContentTier::Fuzzy`]
|
||||||
|
/// rows. `None` when the term does not fuzz (too short, wildcarded, or a
|
||||||
|
/// zero edit budget), which is when the fuzzy pass would not have run.
|
||||||
|
fuzzy: Option<Bitap>,
|
||||||
|
config: Option<Box<Config>>,
|
||||||
|
/// Built once and reused: the extractors are stateless, and the frontend
|
||||||
|
/// re-arms often enough that rebuilding the table per arm would be waste.
|
||||||
|
registry: Registry,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Loop {
|
||||||
|
fn run(mut self) {
|
||||||
|
loop {
|
||||||
|
// Block outright when there is nothing to time out on: an idle
|
||||||
|
// window costs no wakeups at all.
|
||||||
|
let msg = match self.settle_at {
|
||||||
|
None => match self.rx.recv() {
|
||||||
|
Ok(msg) => Some(msg),
|
||||||
|
Err(_) => return,
|
||||||
|
},
|
||||||
|
Some(deadline) => {
|
||||||
|
let wait = deadline.saturating_duration_since(Instant::now());
|
||||||
|
match self.rx.recv_timeout(wait.min(TICK)) {
|
||||||
|
Ok(msg) => Some(msg),
|
||||||
|
Err(mpsc::RecvTimeoutError::Timeout) => None,
|
||||||
|
Err(mpsc::RecvTimeoutError::Disconnected) => return,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
match msg {
|
||||||
|
Some(Msg::Stop) => return,
|
||||||
|
Some(Msg::Clear) => self.reset(),
|
||||||
|
Some(Msg::Watch {
|
||||||
|
query,
|
||||||
|
targets,
|
||||||
|
config,
|
||||||
|
}) => self.rearm(&query, targets, config),
|
||||||
|
Some(Msg::Event(event)) => {
|
||||||
|
classify(
|
||||||
|
&event,
|
||||||
|
&self.targets,
|
||||||
|
&mut self.pending,
|
||||||
|
&mut self.orphan_to,
|
||||||
|
);
|
||||||
|
if !self.pending.is_empty() && self.settle_at.is_none() {
|
||||||
|
self.settle_at = Some(Instant::now() + SETTLE);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None => {}
|
||||||
|
}
|
||||||
|
self.flush_settled();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn reset(&mut self) {
|
||||||
|
self.watcher = None;
|
||||||
|
self.targets.clear();
|
||||||
|
self.pending.clear();
|
||||||
|
self.last_emit.clear();
|
||||||
|
self.orphan_to.clear();
|
||||||
|
self.settle_at = None;
|
||||||
|
self.query = None;
|
||||||
|
self.fuzzy = None;
|
||||||
|
self.config = None;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Point the watcher at a new set of rows, dropping everything about the
|
||||||
|
/// old one, then check each one against the disk. Registration failures
|
||||||
|
/// are per-directory and silent beyond the log: this is a cosmetic
|
||||||
|
/// feature, and a modal about a missing highlight would be worse than the
|
||||||
|
/// missing highlight.
|
||||||
|
fn rearm(&mut self, query: &str, targets: Vec<Target>, config: Box<Config>) {
|
||||||
|
self.reset();
|
||||||
|
if targets.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.query = split_for_cascade(query).ok();
|
||||||
|
// The same construction the fuzzy pass makes for a scan, so a fuzzy
|
||||||
|
// row is re-cut against exactly what matched it.
|
||||||
|
self.fuzzy = self.query.as_ref().and_then(|q| {
|
||||||
|
if q.pattern.is_wildcard() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let folded = q.term.to_ascii_lowercase();
|
||||||
|
let k = edit_budget(folded.len(), config.search.fuzzy_max_edits)?;
|
||||||
|
Bitap::new(folded.as_bytes(), k)
|
||||||
|
});
|
||||||
|
self.config = Some(config);
|
||||||
|
|
||||||
|
let mut watcher = {
|
||||||
|
let tx = self.event_tx.clone();
|
||||||
|
let sink = move |res: notify::Result<NotifyEvent>| {
|
||||||
|
if let Ok(event) = res {
|
||||||
|
let _ = tx.send(Msg::Event(event));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
match RecommendedWatcher::new(sink, NotifyConfig::default()) {
|
||||||
|
Ok(w) => w,
|
||||||
|
Err(e) => {
|
||||||
|
crate::log_warn!("live results: no watcher available: {}", e);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut dirs: Vec<PathBuf> = Vec::new();
|
||||||
|
for target in targets.into_iter().take(MAX_TARGETS) {
|
||||||
|
// Derived from the row's own path, never canonicalized: notify
|
||||||
|
// builds each event path as `watched_dir.join(name)`, so leaving
|
||||||
|
// this spelled as the index spells it is what lets event paths be
|
||||||
|
// compared to `Target::path` as plain strings.
|
||||||
|
let Some(dir) = Path::new(&target.path).parent().map(Path::to_path_buf) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if !dirs.contains(&dir) {
|
||||||
|
if dirs.len() >= MAX_DIRS {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if let Err(e) = watcher.watch(&dir, RecursiveMode::NonRecursive) {
|
||||||
|
// Deliberately not the indexer's all-or-nothing: partial
|
||||||
|
// coverage of a display nicety is fine. Rows under this
|
||||||
|
// directory are dropped rather than swept — a row nothing
|
||||||
|
// can follow is better left alone than corrected once and
|
||||||
|
// then silently frozen.
|
||||||
|
crate::log_warn!("live results: not watching {}: {}", dir.display(), e);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
dirs.push(dir);
|
||||||
|
}
|
||||||
|
self.targets.insert(target.path.clone(), target);
|
||||||
|
}
|
||||||
|
if self.targets.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.watcher = Some(watcher);
|
||||||
|
self.sweep();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Compare every target against the disk once and emit what disagrees.
|
||||||
|
///
|
||||||
|
/// Nothing here waits for an event, which is the point: a row whose file
|
||||||
|
/// changed while it was scrolled out of view — or one whose index row was
|
||||||
|
/// simply out of date when the search returned it — is corrected the
|
||||||
|
/// moment it is watched.
|
||||||
|
fn sweep(&mut self) {
|
||||||
|
let mut updates: Vec<LiveUpdate> = Vec::new();
|
||||||
|
for target in self.targets.values() {
|
||||||
|
match std::fs::metadata(&target.path) {
|
||||||
|
Ok(meta) if meta.is_file() => {
|
||||||
|
let (size, mtime) = (meta.len(), mtime_of(&meta));
|
||||||
|
if size == target.size && mtime == target.mtime {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
updates.push(self.changed_update(&target.path, target.text, size, mtime));
|
||||||
|
}
|
||||||
|
// Unreadable counts as gone: the row cannot be shown as
|
||||||
|
// current when we cannot see the file at all.
|
||||||
|
_ => updates.push(LiveUpdate::Gone {
|
||||||
|
path: target.path.clone(),
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let now = Instant::now();
|
||||||
|
for update in updates {
|
||||||
|
// Recorded so an event arriving right behind the sweep — a save
|
||||||
|
// still in flight when the row came on screen — does not repeat
|
||||||
|
// the same answer a moment later.
|
||||||
|
self.last_emit.insert(update.path().to_string(), now);
|
||||||
|
self.note_emitted(&update);
|
||||||
|
self.send(update);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Turn the settled window's decisions into updates.
|
||||||
|
fn flush_settled(&mut self) {
|
||||||
|
let Some(at) = self.settle_at else { return };
|
||||||
|
if Instant::now() < at {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.settle_at = None;
|
||||||
|
|
||||||
|
// Windows never reports the two halves of a rename as one event and
|
||||||
|
// gives no cookie to pair them by. When exactly one target went away
|
||||||
|
// and exactly one unclaimed destination appeared in the same window,
|
||||||
|
// they are the same file; anything more ambiguous resolves to the
|
||||||
|
// truthful "gone".
|
||||||
|
let orphans = std::mem::take(&mut self.orphan_to);
|
||||||
|
let gone: Vec<String> = self
|
||||||
|
.pending
|
||||||
|
.iter()
|
||||||
|
.filter(|(_, op)| **op == Op::Gone)
|
||||||
|
.map(|(path, _)| path.clone())
|
||||||
|
.collect();
|
||||||
|
if gone.len() == 1 && orphans.len() == 1 {
|
||||||
|
self.pending
|
||||||
|
.insert(gone[0].clone(), Op::Renamed(orphans[0].clone()));
|
||||||
|
}
|
||||||
|
|
||||||
|
let now = Instant::now();
|
||||||
|
let ready: Vec<(String, Op)> = self
|
||||||
|
.pending
|
||||||
|
.iter()
|
||||||
|
.filter(|(path, _)| {
|
||||||
|
self.last_emit
|
||||||
|
.get(*path)
|
||||||
|
.is_none_or(|t| now.duration_since(*t) >= MIN_INTERVAL)
|
||||||
|
})
|
||||||
|
.take(MAX_PER_TICK)
|
||||||
|
.map(|(path, op)| (path.clone(), op.clone()))
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
for (path, op) in ready {
|
||||||
|
self.pending.remove(&path);
|
||||||
|
self.last_emit.insert(path.clone(), now);
|
||||||
|
self.apply(path, op);
|
||||||
|
}
|
||||||
|
if !self.pending.is_empty() {
|
||||||
|
self.settle_at = Some(now + SETTLE);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn apply(&mut self, path: String, op: Op) {
|
||||||
|
let update = match op {
|
||||||
|
Op::Gone => LiveUpdate::Gone { path },
|
||||||
|
Op::Renamed(to) => {
|
||||||
|
let name = to
|
||||||
|
.file_name()
|
||||||
|
.map(|n| n.to_string_lossy().into_owned())
|
||||||
|
.unwrap_or_default();
|
||||||
|
LiveUpdate::Renamed {
|
||||||
|
path,
|
||||||
|
to: to.to_string_lossy().into_owned(),
|
||||||
|
name,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Op::Changed => {
|
||||||
|
// Read from the file, not from the index: this has to land
|
||||||
|
// whatever the indexer is doing, or not doing.
|
||||||
|
match std::fs::metadata(&path) {
|
||||||
|
Ok(meta) if meta.is_file() => {
|
||||||
|
let text = self.targets.get(&path).and_then(|t| t.text);
|
||||||
|
self.changed_update(&path, text, meta.len(), mtime_of(&meta))
|
||||||
|
}
|
||||||
|
_ => LiveUpdate::Gone { path },
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
self.note_emitted(&update);
|
||||||
|
self.send(update);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the file at `path` implies for the row showing it.
|
||||||
|
fn changed_update(
|
||||||
|
&self,
|
||||||
|
path: &str,
|
||||||
|
text: Option<ContentTier>,
|
||||||
|
size: u64,
|
||||||
|
mtime: i64,
|
||||||
|
) -> LiveUpdate {
|
||||||
|
LiveUpdate::Changed {
|
||||||
|
path: path.to_string(),
|
||||||
|
size,
|
||||||
|
mtime,
|
||||||
|
window: text.map_or(WindowUpdate::Unchanged, |tier| {
|
||||||
|
self.window_from_disk(path, size, tier)
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Re-cut this row's Content Match window from the file on disk.
|
||||||
|
///
|
||||||
|
/// Deliberately the indexer's own path — [`crate::mime::guess_mime_from_head`]
|
||||||
|
/// then [`crate::file_handling::decide_content`], which already applies
|
||||||
|
/// `content_extensions` and the `maximum_text_size` truncation — so the
|
||||||
|
/// text a window is cut from is the text the index would have stored, and
|
||||||
|
/// a refreshed row cannot disagree with a re-run search about anything but
|
||||||
|
/// timing. Then the tier's own matcher, for the same reason.
|
||||||
|
fn window_from_disk(&self, path: &str, size: u64, tier: ContentTier) -> WindowUpdate {
|
||||||
|
let (Some(config), Some(query)) = (self.config.as_deref(), self.query.as_ref()) else {
|
||||||
|
return WindowUpdate::Unchanged;
|
||||||
|
};
|
||||||
|
// The indexer would not have stored text for a file this large, so
|
||||||
|
// neither does the row — reading it would stall this thread over a
|
||||||
|
// window nobody can see the whole of anyway.
|
||||||
|
if size > config.processing.maximum_text_file_size {
|
||||||
|
return WindowUpdate::Unchanged;
|
||||||
|
}
|
||||||
|
let file = Path::new(path);
|
||||||
|
let Some(head) = read_head(file, config.processing.hash_length) else {
|
||||||
|
return WindowUpdate::Unchanged;
|
||||||
|
};
|
||||||
|
let mime = crate::mime::guess_mime_from_head(file, &head);
|
||||||
|
// Extractors run third-party parsers over whatever the file now
|
||||||
|
// holds. One that panics must not take this thread — and with it every
|
||||||
|
// live row for the rest of the session — down with it.
|
||||||
|
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
|
||||||
|
crate::file_handling::decide_content(path, mime.as_deref(), &self.registry, config)
|
||||||
|
}));
|
||||||
|
let outcome = match outcome {
|
||||||
|
Ok(outcome) => outcome,
|
||||||
|
Err(_) => {
|
||||||
|
crate::log_warn!("live results: extracting {} panicked", path);
|
||||||
|
return WindowUpdate::Unchanged;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let Some(text) = crate::file_handling::outcome_body(&outcome) else {
|
||||||
|
return WindowUpdate::Unchanged;
|
||||||
|
};
|
||||||
|
let folded = text.to_ascii_lowercase();
|
||||||
|
let cut = match tier {
|
||||||
|
ContentTier::Exact => {
|
||||||
|
// A literal term always yields a window, marked or not,
|
||||||
|
// because the passes only ever call this for a body FTS
|
||||||
|
// already matched. Here the body may genuinely have stopped
|
||||||
|
// matching, and an unmarked window is how that reads.
|
||||||
|
crate::search::cascade::text_snippet(&query.pattern, text, &folded)
|
||||||
|
.filter(|snip| !snip.ranges.is_empty())
|
||||||
|
}
|
||||||
|
ContentTier::Fuzzy => match &self.fuzzy {
|
||||||
|
Some(bitap) => {
|
||||||
|
crate::search::cascade::fuzzy_snippet(bitap, text, &folded).map(|(_, s)| s)
|
||||||
|
}
|
||||||
|
// The term does not fuzz, so a fuzzy row cannot be re-judged;
|
||||||
|
// leaving it is the honest reading.
|
||||||
|
None => return WindowUpdate::Unchanged,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
match cut {
|
||||||
|
Some(snip) => WindowUpdate::Cut(snip),
|
||||||
|
None => WindowUpdate::NoMatch,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Keep the target's baseline in step with what the frontend was just
|
||||||
|
/// told, so a later sweep over the same arm does not repeat itself.
|
||||||
|
fn note_emitted(&mut self, update: &LiveUpdate) {
|
||||||
|
let LiveUpdate::Changed {
|
||||||
|
path, size, mtime, ..
|
||||||
|
} = update
|
||||||
|
else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if let Some(target) = self.targets.get_mut(path) {
|
||||||
|
target.size = *size;
|
||||||
|
target.mtime = *mtime;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn send(&self, update: LiveUpdate) {
|
||||||
|
if self.update_tx.send(update).is_ok() {
|
||||||
|
(self.notify)();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The first `limit` bytes of a file, for MIME sniffing. A short read is the
|
||||||
|
/// whole file and is not an error; an unreadable file simply has no MIME.
|
||||||
|
fn read_head(path: &Path, limit: usize) -> Option<Vec<u8>> {
|
||||||
|
let file = std::fs::File::open(path).ok()?;
|
||||||
|
let mut head = Vec::new();
|
||||||
|
file.take(limit as u64).read_to_end(&mut head).ok()?;
|
||||||
|
Some(head)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mtime_of(meta: &std::fs::Metadata) -> i64 {
|
||||||
|
meta.modified()
|
||||||
|
.ok()
|
||||||
|
.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok())
|
||||||
|
.map(|d| d.as_secs() as i64)
|
||||||
|
.unwrap_or(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fold one notify event into the pending decisions for this window.
|
||||||
|
///
|
||||||
|
/// Pure and filesystem-free, which is what makes the platform differences
|
||||||
|
/// testable: every shape below is a real emission from `notify` 6.1 on one
|
||||||
|
/// platform or the other, and the coalescing window is what reconciles them.
|
||||||
|
fn classify(
|
||||||
|
event: &NotifyEvent,
|
||||||
|
targets: &HashMap<String, Target>,
|
||||||
|
pending: &mut HashMap<String, Op>,
|
||||||
|
orphan_to: &mut Vec<PathBuf>,
|
||||||
|
) {
|
||||||
|
use notify::event::{ModifyKind, RenameMode};
|
||||||
|
|
||||||
|
let key = |p: &PathBuf| p.to_string_lossy().into_owned();
|
||||||
|
let is_target = |p: &PathBuf| targets.contains_key(&key(p));
|
||||||
|
|
||||||
|
match event.kind {
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::Both)) => {
|
||||||
|
// Linux pairs the halves and emits this *after* the From/To pair,
|
||||||
|
// so it lands in the same window and overwrites the provisional
|
||||||
|
// Gone recorded below.
|
||||||
|
let (Some(from), Some(to)) = (event.paths.first(), event.paths.get(1)) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if is_target(from) {
|
||||||
|
pending.insert(key(from), Op::Renamed(to.clone()));
|
||||||
|
} else if is_target(to) {
|
||||||
|
// The atomic-save shape: a temporary file renamed over a row
|
||||||
|
// we are watching. The row did not move; its contents changed.
|
||||||
|
pending.insert(key(to), Op::Changed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::To)) => {
|
||||||
|
for path in &event.paths {
|
||||||
|
if is_target(path) {
|
||||||
|
pending.insert(key(path), Op::Changed);
|
||||||
|
} else {
|
||||||
|
orphan_to.push(path.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::From)) => {
|
||||||
|
for path in &event.paths {
|
||||||
|
if is_target(path) {
|
||||||
|
// Provisional; a Both in this same window upgrades it.
|
||||||
|
pending.entry(key(path)).or_insert(Op::Gone);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EventKind::Remove(_) => {
|
||||||
|
for path in &event.paths {
|
||||||
|
if is_target(path) {
|
||||||
|
pending.insert(key(path), Op::Gone);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EventKind::Create(_) | EventKind::Modify(_) => {
|
||||||
|
for path in &event.paths {
|
||||||
|
if is_target(path) {
|
||||||
|
// A Create at a watched path un-deletes the row.
|
||||||
|
pending.insert(key(path), Op::Changed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Access events, and anything for a path we are not showing. Live
|
||||||
|
// results never *add* rows: we have no way to know an unrelated new
|
||||||
|
// file matches the query, and guessing would be a second, unranked
|
||||||
|
// search wearing the first one's clothes.
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
#[path = "live_tests.rs"]
|
||||||
|
mod tests;
|
||||||
655
crates/quicksearch-core/src/live_tests.rs
Normal file
655
crates/quicksearch-core/src/live_tests.rs
Normal file
|
|
@ -0,0 +1,655 @@
|
||||||
|
//! Tests for live result watching.
|
||||||
|
//!
|
||||||
|
//! The [`classify`] tests are pure: they feed the exact event shapes `notify`
|
||||||
|
//! 6.1 emits on each platform and check what one settled window decides. They
|
||||||
|
//! are the ones that pin the design — in particular that a watch on a
|
||||||
|
//! *directory* sees an atomic save, which a watch on the file would not.
|
||||||
|
//!
|
||||||
|
//! The end-to-end tests drive a real filesystem through a real watcher.
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
use notify::event::{CreateKind, ModifyKind, RemoveKind, RenameMode};
|
||||||
|
|
||||||
|
fn target(path: &str) -> (String, Target) {
|
||||||
|
(
|
||||||
|
path.to_string(),
|
||||||
|
Target {
|
||||||
|
path: path.to_string(),
|
||||||
|
text: Some(ContentTier::Exact),
|
||||||
|
size: 0,
|
||||||
|
mtime: 0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn targets(paths: &[&str]) -> HashMap<String, Target> {
|
||||||
|
paths.iter().map(|p| target(p)).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn event(kind: EventKind, paths: &[&str]) -> NotifyEvent {
|
||||||
|
NotifyEvent {
|
||||||
|
kind,
|
||||||
|
paths: paths.iter().map(PathBuf::from).collect(),
|
||||||
|
attrs: Default::default(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Feed a window of events and report what it decided, after the same
|
||||||
|
/// orphan-pairing `flush_settled` applies.
|
||||||
|
fn window(targets: &HashMap<String, Target>, events: Vec<NotifyEvent>) -> HashMap<String, Op> {
|
||||||
|
let mut pending = HashMap::new();
|
||||||
|
let mut orphan_to = Vec::new();
|
||||||
|
for event in &events {
|
||||||
|
classify(event, targets, &mut pending, &mut orphan_to);
|
||||||
|
}
|
||||||
|
let gone: Vec<String> = pending
|
||||||
|
.iter()
|
||||||
|
.filter(|(_, op)| **op == Op::Gone)
|
||||||
|
.map(|(p, _)| p.clone())
|
||||||
|
.collect();
|
||||||
|
if gone.len() == 1 && orphan_to.len() == 1 {
|
||||||
|
pending.insert(gone[0].clone(), Op::Renamed(orphan_to[0].clone()));
|
||||||
|
}
|
||||||
|
pending
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An editor saving a file writes a temporary and renames it over the target.
|
||||||
|
/// The row did not move — its contents changed — and a watch on the file
|
||||||
|
/// itself would have seen none of this, because the inode it was attached to
|
||||||
|
/// is the one that got orphaned. This test is why the watches are on
|
||||||
|
/// directories.
|
||||||
|
#[test]
|
||||||
|
fn an_atomic_save_reads_as_a_content_change() {
|
||||||
|
let t = targets(&["/docs/report.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::From)),
|
||||||
|
&["/docs/.report.txt.swp"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::To)),
|
||||||
|
&["/docs/report.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::Both)),
|
||||||
|
&["/docs/.report.txt.swp", "/docs/report.txt"],
|
||||||
|
),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert_eq!(decided.get("/docs/report.txt"), Some(&Op::Changed));
|
||||||
|
assert_eq!(decided.len(), 1, "nothing else was decided: {decided:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Linux emits From, To and then Both for one in-directory rename. The
|
||||||
|
/// provisional Gone recorded for the From half must not escape the window.
|
||||||
|
#[test]
|
||||||
|
fn a_linux_rename_pairs_without_leaking_a_gone() {
|
||||||
|
let t = targets(&["/docs/old.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::From)),
|
||||||
|
&["/docs/old.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::To)),
|
||||||
|
&["/docs/new.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::Both)),
|
||||||
|
&["/docs/old.txt", "/docs/new.txt"],
|
||||||
|
),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
decided.get("/docs/old.txt"),
|
||||||
|
Some(&Op::Renamed(PathBuf::from("/docs/new.txt")))
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!decided.values().any(|op| *op == Op::Gone),
|
||||||
|
"a provisional Gone escaped: {decided:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Windows never emits `Both` and gives no cookie to pair the halves by, so
|
||||||
|
/// one unclaimed destination beside one departed target is paired by position.
|
||||||
|
#[test]
|
||||||
|
fn a_windows_rename_pairs_by_elimination() {
|
||||||
|
let t = targets(&["/docs/old.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::From)),
|
||||||
|
&["/docs/old.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::To)),
|
||||||
|
&["/docs/new.txt"],
|
||||||
|
),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
decided.get("/docs/old.txt"),
|
||||||
|
Some(&Op::Renamed(PathBuf::from("/docs/new.txt")))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two departures and two arrivals in one window cannot be paired without
|
||||||
|
/// guessing. Guessing wrong renames a row to someone else's file, so the
|
||||||
|
/// ambiguous case resolves to the truthful answer instead.
|
||||||
|
#[test]
|
||||||
|
fn an_ambiguous_windows_window_reports_gone_rather_than_guessing() {
|
||||||
|
let t = targets(&["/docs/a.txt", "/docs/b.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::From)),
|
||||||
|
&["/docs/a.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::From)),
|
||||||
|
&["/docs/b.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::To)),
|
||||||
|
&["/docs/x.txt"],
|
||||||
|
),
|
||||||
|
event(
|
||||||
|
EventKind::Modify(ModifyKind::Name(RenameMode::To)),
|
||||||
|
&["/docs/y.txt"],
|
||||||
|
),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert_eq!(decided.get("/docs/a.txt"), Some(&Op::Gone));
|
||||||
|
assert_eq!(decided.get("/docs/b.txt"), Some(&Op::Gone));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A watched directory is full of files that are not on screen. None of them
|
||||||
|
/// may produce an update — live results never add rows.
|
||||||
|
#[test]
|
||||||
|
fn events_for_paths_that_are_not_shown_decide_nothing() {
|
||||||
|
let t = targets(&["/docs/shown.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![
|
||||||
|
event(EventKind::Create(CreateKind::File), &["/docs/other.txt"]),
|
||||||
|
event(EventKind::Modify(ModifyKind::Any), &["/docs/another.txt"]),
|
||||||
|
event(EventKind::Remove(RemoveKind::File), &["/docs/third.txt"]),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert!(decided.is_empty(), "{decided:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A delete marks the row; a file recreated at the same path un-marks it,
|
||||||
|
/// which is what makes the mark reversible without re-registering anything.
|
||||||
|
#[test]
|
||||||
|
fn a_delete_marks_the_row_and_a_recreate_clears_it() {
|
||||||
|
let t = targets(&["/docs/report.txt"]);
|
||||||
|
let gone = window(
|
||||||
|
&t,
|
||||||
|
vec![event(
|
||||||
|
EventKind::Remove(RemoveKind::File),
|
||||||
|
&["/docs/report.txt"],
|
||||||
|
)],
|
||||||
|
);
|
||||||
|
assert_eq!(gone.get("/docs/report.txt"), Some(&Op::Gone));
|
||||||
|
|
||||||
|
let back = window(
|
||||||
|
&t,
|
||||||
|
vec![event(
|
||||||
|
EventKind::Create(CreateKind::File),
|
||||||
|
&["/docs/report.txt"],
|
||||||
|
)],
|
||||||
|
);
|
||||||
|
assert_eq!(back.get("/docs/report.txt"), Some(&Op::Changed));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Several writes to one file inside a window are one decision, not several.
|
||||||
|
#[test]
|
||||||
|
fn repeated_writes_in_one_window_coalesce() {
|
||||||
|
let t = targets(&["/docs/log.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![
|
||||||
|
event(EventKind::Modify(ModifyKind::Any), &["/docs/log.txt"]),
|
||||||
|
event(EventKind::Modify(ModifyKind::Any), &["/docs/log.txt"]),
|
||||||
|
event(EventKind::Modify(ModifyKind::Any), &["/docs/log.txt"]),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert_eq!(decided.len(), 1);
|
||||||
|
assert_eq!(decided.get("/docs/log.txt"), Some(&Op::Changed));
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- end to end ----------------------------------------------------------
|
||||||
|
|
||||||
|
use crate::testutil::scratch_dir;
|
||||||
|
|
||||||
|
/// Collect updates until `want` of them arrive or the timeout expires.
|
||||||
|
fn collect(rx: &mpsc::Receiver<LiveUpdate>, want: usize, timeout: Duration) -> Vec<LiveUpdate> {
|
||||||
|
let deadline = Instant::now() + timeout;
|
||||||
|
let mut out = Vec::new();
|
||||||
|
while out.len() < want {
|
||||||
|
let left = deadline.saturating_duration_since(Instant::now());
|
||||||
|
if left.is_zero() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
match rx.recv_timeout(left) {
|
||||||
|
Ok(update) => out.push(update),
|
||||||
|
Err(_) => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A target describing `path` exactly as it is on disk right now, so the
|
||||||
|
/// arm-time sweep finds nothing to report and the test sees only what it
|
||||||
|
/// provokes afterwards.
|
||||||
|
fn current_target(path: &str, text: Option<ContentTier>) -> Target {
|
||||||
|
let meta = std::fs::metadata(path).expect("target file exists");
|
||||||
|
Target {
|
||||||
|
path: path.to_string(),
|
||||||
|
text,
|
||||||
|
size: meta.len(),
|
||||||
|
mtime: mtime_of(&meta),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn watch_one(dir: &Path, name: &str) -> (LiveWatcher, mpsc::Receiver<LiveUpdate>, String) {
|
||||||
|
watch_one_matching(dir, name, "hello world", None)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Write `body` at `dir/name` and watch it for the query `hello`.
|
||||||
|
fn watch_one_matching(
|
||||||
|
dir: &Path,
|
||||||
|
name: &str,
|
||||||
|
body: &str,
|
||||||
|
text: Option<ContentTier>,
|
||||||
|
) -> (LiveWatcher, mpsc::Receiver<LiveUpdate>, String) {
|
||||||
|
let path = dir.join(name).to_string_lossy().into_owned();
|
||||||
|
std::fs::write(&path, body).unwrap();
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
watcher.watch(
|
||||||
|
"hello",
|
||||||
|
vec![current_target(&path, text)],
|
||||||
|
&Config::default(),
|
||||||
|
);
|
||||||
|
// Registration happens on the watcher thread.
|
||||||
|
std::thread::sleep(Duration::from_millis(300));
|
||||||
|
(watcher, rx, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stop the watcher, then take the tree with it.
|
||||||
|
///
|
||||||
|
/// The stop is not optional: the watcher holds an inotify registration on the
|
||||||
|
/// directory, and pulling the directory out from under a live one is a race
|
||||||
|
/// worth not having. Unlike most of this crate's tests these do clean up on
|
||||||
|
/// the way out — the trees are a file or two apiece, generated identically
|
||||||
|
/// every run, so they hold no evidence the assertion message does not already
|
||||||
|
/// carry.
|
||||||
|
fn stop_and_clean(mut watcher: LiveWatcher, dir: &Path) {
|
||||||
|
watcher.stop();
|
||||||
|
std::fs::remove_dir_all(dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn e2e_a_rename_surfaces_with_the_new_name() {
|
||||||
|
let dir = scratch_dir("live-rename");
|
||||||
|
let (watcher, rx, path) = watch_one(&dir, "before.txt");
|
||||||
|
let renamed = dir.join("after.txt");
|
||||||
|
std::fs::rename(&path, &renamed).unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let Some(LiveUpdate::Renamed {
|
||||||
|
path: from, name, ..
|
||||||
|
}) = updates.first()
|
||||||
|
else {
|
||||||
|
panic!("expected a rename, got {updates:?}");
|
||||||
|
};
|
||||||
|
assert_eq!(from, &path);
|
||||||
|
assert_eq!(name, "after.txt");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn e2e_a_delete_surfaces_as_gone() {
|
||||||
|
let dir = scratch_dir("live-delete");
|
||||||
|
let (watcher, rx, path) = watch_one(&dir, "doomed.txt");
|
||||||
|
std::fs::remove_file(&path).unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
updates
|
||||||
|
.iter()
|
||||||
|
.any(|u| matches!(u, LiveUpdate::Gone { path: p } if *p == path)),
|
||||||
|
"expected a Gone for {path}, got {updates:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The design test, on a real filesystem: write a temporary and rename it over
|
||||||
|
/// the target, the way an editor saves. It must read as a change to the row,
|
||||||
|
/// not as the row disappearing — which is exactly what a watch on the file
|
||||||
|
/// itself would have reported.
|
||||||
|
#[test]
|
||||||
|
fn e2e_an_atomic_save_does_not_read_as_a_delete() {
|
||||||
|
let dir = scratch_dir("live-atomic");
|
||||||
|
let (watcher, rx, path) = watch_one(&dir, "report.txt");
|
||||||
|
let tmp = dir.join("report.txt.tmp");
|
||||||
|
std::fs::write(&tmp, "hello, replaced").unwrap();
|
||||||
|
std::fs::rename(&tmp, &path).unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
!updates.iter().any(|u| matches!(u, LiveUpdate::Gone { .. })),
|
||||||
|
"an atomic save was reported as a deletion: {updates:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A row whose directory does not exist must not panic, and must not stop the
|
||||||
|
/// watcher from covering the rows whose directories do.
|
||||||
|
#[test]
|
||||||
|
fn a_missing_directory_does_not_stop_the_others() {
|
||||||
|
let dir = scratch_dir("live-missing-dir");
|
||||||
|
let good = dir.join("present.txt");
|
||||||
|
std::fs::write(&good, "hello world").unwrap();
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
watcher.watch(
|
||||||
|
"hello",
|
||||||
|
vec![
|
||||||
|
Target {
|
||||||
|
path: dir
|
||||||
|
.join("nowhere")
|
||||||
|
.join("ghost.txt")
|
||||||
|
.to_string_lossy()
|
||||||
|
.into_owned(),
|
||||||
|
text: None,
|
||||||
|
size: 0,
|
||||||
|
mtime: 0,
|
||||||
|
},
|
||||||
|
current_target(&good.to_string_lossy(), None),
|
||||||
|
],
|
||||||
|
&Config::default(),
|
||||||
|
);
|
||||||
|
std::thread::sleep(Duration::from_millis(300));
|
||||||
|
std::fs::remove_file(&good).unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
updates.iter().any(|u| matches!(u, LiveUpdate::Gone { .. })),
|
||||||
|
"the reachable row stopped working: {updates:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- content, read from the file rather than from the index ---------------
|
||||||
|
//
|
||||||
|
// None of these open a database. That is the assertion they all share: a row
|
||||||
|
// on screen tracks the disk with no indexer involved, which is what the
|
||||||
|
// feature is for.
|
||||||
|
|
||||||
|
/// Pull the one `Changed` out of a batch, failing loudly on anything else.
|
||||||
|
fn one_change(updates: &[LiveUpdate], path: &str) -> (u64, i64, WindowUpdate) {
|
||||||
|
let found = updates.iter().find_map(|u| match u {
|
||||||
|
LiveUpdate::Changed {
|
||||||
|
path: p,
|
||||||
|
size,
|
||||||
|
mtime,
|
||||||
|
window,
|
||||||
|
} if p == path => Some((*size, *mtime, window.clone())),
|
||||||
|
_ => None,
|
||||||
|
});
|
||||||
|
found.unwrap_or_else(|| panic!("expected a Changed for {path}, got {updates:?}"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The test the old index-backed design could not have: edit a watched file
|
||||||
|
/// with no index anywhere, and the row's size, modified time and Content Match
|
||||||
|
/// window all follow it.
|
||||||
|
#[test]
|
||||||
|
fn e2e_a_content_change_re_cuts_the_snippet_with_no_index() {
|
||||||
|
let dir = scratch_dir("live-content");
|
||||||
|
let (watcher, rx, path) =
|
||||||
|
watch_one_matching(&dir, "notes.txt", "hello world", Some(ContentTier::Exact));
|
||||||
|
std::fs::write(&path, "hello there, a much longer world").unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let (size, mtime, window) = one_change(&updates, &path);
|
||||||
|
assert_eq!(size, "hello there, a much longer world".len() as u64);
|
||||||
|
assert!(mtime > 0, "no modified time was read");
|
||||||
|
let WindowUpdate::Cut(snippet) = window else {
|
||||||
|
panic!("the body still matches, so it should carry a window: {window:?}");
|
||||||
|
};
|
||||||
|
assert!(
|
||||||
|
snippet.window.contains("much longer"),
|
||||||
|
"the window is stale: {:?}",
|
||||||
|
snippet.window
|
||||||
|
);
|
||||||
|
assert!(!snippet.ranges.is_empty(), "the match was not marked");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Edited until it no longer matches, the row keeps its place and its metadata
|
||||||
|
/// but loses its window — the Content Match cell falls back to its dash rather
|
||||||
|
/// than showing text that is no longer a hit.
|
||||||
|
#[test]
|
||||||
|
fn e2e_an_edit_that_removes_the_match_clears_the_window() {
|
||||||
|
let dir = scratch_dir("live-unmatch");
|
||||||
|
let (watcher, rx, path) =
|
||||||
|
watch_one_matching(&dir, "notes.txt", "hello world", Some(ContentTier::Exact));
|
||||||
|
std::fs::write(&path, "nothing of interest here").unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let (size, _, window) = one_change(&updates, &path);
|
||||||
|
assert_eq!(size, "nothing of interest here".len() as u64);
|
||||||
|
assert_eq!(
|
||||||
|
window,
|
||||||
|
WindowUpdate::NoMatch,
|
||||||
|
"a window survived the match it no longer has"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A fuzzy content hit is re-cut with the fuzzy matcher, not the literal one.
|
||||||
|
/// The literal is absent from the body by construction — that is what made
|
||||||
|
/// it a fuzzy hit — so re-cutting it as an exact row would read as "no longer
|
||||||
|
/// matches" and blank a cell that still has a hit in it.
|
||||||
|
#[test]
|
||||||
|
fn e2e_a_fuzzy_content_hit_is_re_cut_with_the_fuzzy_matcher() {
|
||||||
|
let dir = scratch_dir("live-fuzzy");
|
||||||
|
let path = dir.join("notes.txt").to_string_lossy().into_owned();
|
||||||
|
// "quartz" within one edit of "quarts": a stage-8 hit for that query.
|
||||||
|
std::fs::write(&path, "sphinx of black quarts judge my vow").unwrap();
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
watcher.watch(
|
||||||
|
"quartz",
|
||||||
|
vec![current_target(&path, Some(ContentTier::Fuzzy))],
|
||||||
|
&Config::default(),
|
||||||
|
);
|
||||||
|
std::thread::sleep(Duration::from_millis(300));
|
||||||
|
std::fs::write(&path, "the sphinx of black quarts judged my vow again").unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let (_, _, window) = one_change(&updates, &path);
|
||||||
|
let WindowUpdate::Cut(snippet) = window else {
|
||||||
|
panic!("a fuzzy hit was re-judged as exact: {window:?}");
|
||||||
|
};
|
||||||
|
assert!(
|
||||||
|
snippet.window.contains("quarts judged"),
|
||||||
|
"the window was not re-cut from the new body: {:?}",
|
||||||
|
snippet.window
|
||||||
|
);
|
||||||
|
assert!(!snippet.ranges.is_empty(), "the fuzzy match was not marked");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A file no extractor claims still reports what `metadata` knows. Size and
|
||||||
|
/// Modified are not the text columns' to withhold.
|
||||||
|
#[test]
|
||||||
|
fn e2e_a_file_with_no_extractable_text_still_reports_its_metadata() {
|
||||||
|
let dir = scratch_dir("live-binary");
|
||||||
|
let path = dir.join("blob.bin").to_string_lossy().into_owned();
|
||||||
|
std::fs::write(&path, [0x00u8, 0x01, 0x02, 0xFF]).unwrap();
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
watcher.watch(
|
||||||
|
"hello",
|
||||||
|
vec![current_target(&path, Some(ContentTier::Exact))],
|
||||||
|
&Config::default(),
|
||||||
|
);
|
||||||
|
std::thread::sleep(Duration::from_millis(300));
|
||||||
|
std::fs::write(&path, [0x00u8, 0x01, 0x02, 0xFF, 0xFE, 0xFD]).unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let (size, _, window) = one_change(&updates, &path);
|
||||||
|
assert_eq!(size, 6);
|
||||||
|
// `Unchanged`, not `NoMatch`: nothing readable came back, so there is no
|
||||||
|
// evidence the row's window is wrong — only that it is unverifiable.
|
||||||
|
assert_eq!(window, WindowUpdate::Unchanged);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Past `maximum_text_file_size` the indexer stores no text, so neither does
|
||||||
|
/// the row — but it still says how big the file got.
|
||||||
|
#[test]
|
||||||
|
fn e2e_a_file_over_the_text_size_limit_reports_size_but_no_window() {
|
||||||
|
let dir = scratch_dir("live-oversize");
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
let path = dir.join("huge.txt").to_string_lossy().into_owned();
|
||||||
|
std::fs::write(&path, "hello world").unwrap();
|
||||||
|
let mut config = Config::default();
|
||||||
|
config.processing.maximum_text_file_size = 16;
|
||||||
|
watcher.watch(
|
||||||
|
"hello",
|
||||||
|
vec![current_target(&path, Some(ContentTier::Exact))],
|
||||||
|
&config,
|
||||||
|
);
|
||||||
|
std::thread::sleep(Duration::from_millis(300));
|
||||||
|
std::fs::write(&path, "hello world, and rather more of it besides").unwrap();
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let (size, _, window) = one_change(&updates, &path);
|
||||||
|
assert_eq!(
|
||||||
|
size,
|
||||||
|
"hello world, and rather more of it besides".len() as u64
|
||||||
|
);
|
||||||
|
// Not read, so not disproved: the window the search found stands.
|
||||||
|
assert_eq!(window, WindowUpdate::Unchanged);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- the arm-time sweep ---------------------------------------------------
|
||||||
|
|
||||||
|
/// A row armed with what the *index* said about a file that has since moved on
|
||||||
|
/// is corrected the moment it is watched. This is the check of the index
|
||||||
|
/// against the disk, and it is also the only thing that reports anything at
|
||||||
|
/// all on a filesystem the platform sends no events for.
|
||||||
|
#[test]
|
||||||
|
fn arming_corrects_a_row_that_went_stale_while_it_was_not_watched() {
|
||||||
|
let dir = scratch_dir("live-sweep-stale");
|
||||||
|
let path = dir.join("drifted.txt").to_string_lossy().into_owned();
|
||||||
|
std::fs::write(&path, "hello, a body the index never saw").unwrap();
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
// What a stale index row would have claimed.
|
||||||
|
watcher.watch(
|
||||||
|
"hello",
|
||||||
|
vec![Target {
|
||||||
|
path: path.clone(),
|
||||||
|
text: Some(ContentTier::Exact),
|
||||||
|
size: 5,
|
||||||
|
mtime: 1,
|
||||||
|
}],
|
||||||
|
&Config::default(),
|
||||||
|
);
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
let (size, mtime, window) = one_change(&updates, &path);
|
||||||
|
assert_eq!(size, "hello, a body the index never saw".len() as u64);
|
||||||
|
assert!(mtime > 1, "the stale modified time survived");
|
||||||
|
assert!(
|
||||||
|
matches!(window, WindowUpdate::Cut(_)),
|
||||||
|
"the window was not re-cut: {window:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Same sweep, for a row whose file is simply not there any more.
|
||||||
|
#[test]
|
||||||
|
fn arming_reports_a_row_whose_file_vanished_while_it_was_not_watched() {
|
||||||
|
let dir = scratch_dir("live-sweep-gone");
|
||||||
|
// A sibling keeps the directory watchable, so the ghost is dropped for
|
||||||
|
// being missing rather than for its directory being missing.
|
||||||
|
let sibling = dir.join("present.txt");
|
||||||
|
std::fs::write(&sibling, "hello world").unwrap();
|
||||||
|
let ghost = dir.join("vanished.txt").to_string_lossy().into_owned();
|
||||||
|
let (watcher, rx) = LiveWatcher::start(Arc::new(|| {}));
|
||||||
|
watcher.watch(
|
||||||
|
"hello",
|
||||||
|
vec![Target {
|
||||||
|
path: ghost.clone(),
|
||||||
|
text: None,
|
||||||
|
size: 11,
|
||||||
|
mtime: 1,
|
||||||
|
}],
|
||||||
|
&Config::default(),
|
||||||
|
);
|
||||||
|
|
||||||
|
let updates = collect(&rx, 1, Duration::from_secs(5));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
updates
|
||||||
|
.iter()
|
||||||
|
.any(|u| matches!(u, LiveUpdate::Gone { path } if *path == ghost)),
|
||||||
|
"expected a Gone for {ghost}, got {updates:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The other half of the sweep, and the one that keeps it quiet: a row that
|
||||||
|
/// already agrees with the disk is not touched. Without this the watcher would
|
||||||
|
/// repaint every visible row on every scroll.
|
||||||
|
#[test]
|
||||||
|
fn arming_says_nothing_about_a_row_that_already_agrees_with_the_disk() {
|
||||||
|
let dir = scratch_dir("live-sweep-quiet");
|
||||||
|
let (watcher, rx, _path) =
|
||||||
|
watch_one_matching(&dir, "steady.txt", "hello world", Some(ContentTier::Exact));
|
||||||
|
|
||||||
|
// `watch_one_matching` already waited out registration; anything the
|
||||||
|
// sweep decided has been sent by now.
|
||||||
|
let updates = collect(&rx, 1, Duration::from_millis(500));
|
||||||
|
stop_and_clean(watcher, &dir);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
updates.is_empty(),
|
||||||
|
"the sweep invented an update: {updates:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Re-arming replaces the set wholesale; an event for a path that is no longer
|
||||||
|
/// shown decides nothing.
|
||||||
|
#[test]
|
||||||
|
fn re_arming_drops_the_previous_targets() {
|
||||||
|
let t = targets(&["/docs/new.txt"]);
|
||||||
|
let decided = window(
|
||||||
|
&t,
|
||||||
|
vec![event(
|
||||||
|
EventKind::Modify(ModifyKind::Any),
|
||||||
|
&["/docs/old.txt"],
|
||||||
|
)],
|
||||||
|
);
|
||||||
|
assert!(decided.is_empty(), "{decided:?}");
|
||||||
|
}
|
||||||
|
|
@ -490,26 +490,6 @@ pub fn release_free_heap() {
|
||||||
// spans to the kernel on free.
|
// spans to the kernel on free.
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Live and free-but-retained heap bytes, as `(in_use, free)`: `free` is
|
|
||||||
/// memory already given back to the allocator that glibc still charges the
|
|
||||||
/// process for. `None` where the platform has no way to answer.
|
|
||||||
pub fn heap_stats() -> Option<(u64, u64)> {
|
|
||||||
#[cfg(all(target_os = "linux", target_env = "gnu"))]
|
|
||||||
{
|
|
||||||
// `mallinfo2`, not `mallinfo`: the older struct is `int`-typed and
|
|
||||||
// silently wraps past 2 GiB, which is exactly the size where the
|
|
||||||
// answer starts to matter.
|
|
||||||
//
|
|
||||||
// SAFETY: no arguments, returns a plain struct by value.
|
|
||||||
let info = unsafe { libc::mallinfo2() };
|
|
||||||
Some((info.uordblks as u64, info.fordblks as u64))
|
|
||||||
}
|
|
||||||
#[cfg(not(all(target_os = "linux", target_env = "gnu")))]
|
|
||||||
{
|
|
||||||
None
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// How long to keep retrying a delete that fails because something else holds
|
/// How long to keep retrying a delete that fails because something else holds
|
||||||
/// the file open.
|
/// the file open.
|
||||||
#[cfg(windows)]
|
#[cfg(windows)]
|
||||||
|
|
|
||||||
|
|
@ -89,15 +89,6 @@ impl Scope {
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The configured root `path` lives under, if any. Containment is
|
|
||||||
/// component-wise, per [`crate::file_handling::UnreadableDirs::covers`].
|
|
||||||
pub fn owning_root(&self, path: &Path) -> Option<&Path> {
|
|
||||||
self.roots
|
|
||||||
.iter()
|
|
||||||
.map(|r| r.path.as_path())
|
|
||||||
.find(|root| path.starts_with(root) && path != *root)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Whether the walker would still emit `path` while walking `root`.
|
/// Whether the walker would still emit `path` while walking `root`.
|
||||||
///
|
///
|
||||||
/// Mirrors `read_directory`'s three `continue`s. Full-path ignore
|
/// Mirrors `read_directory`'s three `continue`s. Full-path ignore
|
||||||
|
|
|
||||||
|
|
@ -31,7 +31,6 @@ fn walked(config: &Config, db: &Path) -> HashSet<PathBuf> {
|
||||||
config.clone(),
|
config.clone(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
2,
|
2,
|
||||||
)
|
)
|
||||||
.filter_map(|e| match e {
|
.filter_map(|e| match e {
|
||||||
|
|
@ -155,28 +154,6 @@ fn a_root_is_never_filtered_but_its_children_still_are() {
|
||||||
std::fs::remove_dir_all(&base).ok();
|
std::fs::remove_dir_all(&base).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Root ownership compares whole components, so a sibling whose name
|
|
||||||
/// merely starts with a root's is not inside it — a prune that got this
|
|
||||||
/// wrong would delete a neighbouring folder's entire index.
|
|
||||||
#[test]
|
|
||||||
fn owning_root_does_not_match_name_prefixes() {
|
|
||||||
let base = tmp_tree("prefix");
|
|
||||||
let root = base.join("data");
|
|
||||||
let sibling = base.join("database");
|
|
||||||
std::fs::create_dir_all(&root).unwrap();
|
|
||||||
std::fs::create_dir_all(&sibling).unwrap();
|
|
||||||
|
|
||||||
let mut config = Config::default();
|
|
||||||
config.paths.indexing_paths = vec![root.to_string_lossy().into_owned()];
|
|
||||||
let scope = Scope::from_config(&config).unwrap();
|
|
||||||
|
|
||||||
assert_eq!(scope.owning_root(&root.join("f.txt")), Some(root.as_path()));
|
|
||||||
assert_eq!(scope.owning_root(&sibling.join("f.txt")), None);
|
|
||||||
// The root itself is a directory, never a row, and owns nothing.
|
|
||||||
assert_eq!(scope.owning_root(&root), None);
|
|
||||||
std::fs::remove_dir_all(&base).ok();
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The counters the status display reads: a scan that reports nothing is
|
/// The counters the status display reads: a scan that reports nothing is
|
||||||
/// indistinguishable from a hang.
|
/// indistinguishable from a hang.
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -323,21 +300,3 @@ fn cancelling_stops_the_scan_without_finishing_it() {
|
||||||
std::fs::remove_dir_all(&root).ok();
|
std::fs::remove_dir_all(&root).ok();
|
||||||
std::fs::remove_dir_all(&db_dir).ok();
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A path under no configured root has no rules to apply — a followed
|
|
||||||
/// symlink's target is the real case; `owning_root` returning `None` is
|
|
||||||
/// what keeps it alive.
|
|
||||||
#[test]
|
|
||||||
fn a_path_outside_every_root_has_no_owner() {
|
|
||||||
let base = tmp_tree("outside");
|
|
||||||
let root = base.join("indexed");
|
|
||||||
std::fs::create_dir_all(&root).unwrap();
|
|
||||||
|
|
||||||
let mut config = Config::default();
|
|
||||||
config.paths.indexing_paths = vec![root.to_string_lossy().into_owned()];
|
|
||||||
config.indexing.ignore_patterns = vec!["*".into()];
|
|
||||||
let scope = Scope::from_config(&config).unwrap();
|
|
||||||
|
|
||||||
assert_eq!(scope.owning_root(Path::new("/elsewhere/target.txt")), None);
|
|
||||||
std::fs::remove_dir_all(&base).ok();
|
|
||||||
}
|
|
||||||
|
|
|
||||||
|
|
@ -80,6 +80,62 @@ const CANCEL_CHECK_ROWS: usize = 256;
|
||||||
/// around the match, and the mouseover shows the rest as extended context.
|
/// around the match, and the mouseover shows the rest as extended context.
|
||||||
const SNIPPET_WINDOW_CHARS: usize = 600;
|
const SNIPPET_WINDOW_CHARS: usize = 600;
|
||||||
|
|
||||||
|
/// The Content Match snippet for one document body, cut exactly as the
|
||||||
|
/// full-text passes cut it.
|
||||||
|
///
|
||||||
|
/// `folded` must be `text` ASCII-lowercased. That fold is byte-length
|
||||||
|
/// preserving, which is the whole reason offsets found in it can slice `text`;
|
||||||
|
/// the passes hold one reusable fold buffer per scan and hand it in here
|
||||||
|
/// rather than paying for a second copy.
|
||||||
|
///
|
||||||
|
/// Shared so that [`crate::live`], re-cutting a snippet for a file that
|
||||||
|
/// changed under a result already on screen, produces the same window the
|
||||||
|
/// search itself would — otherwise a row would visibly re-frame its own match
|
||||||
|
/// the moment the file was touched.
|
||||||
|
pub fn text_snippet(
|
||||||
|
pattern: &crate::query::pattern::TermPattern,
|
||||||
|
text: &str,
|
||||||
|
folded: &str,
|
||||||
|
) -> Option<snippet::Snippet> {
|
||||||
|
let opts = snippet::Options {
|
||||||
|
approx_chars: SNIPPET_WINDOW_CHARS,
|
||||||
|
};
|
||||||
|
match pattern.literal() {
|
||||||
|
// Literal terms keep the richer multi-occurrence extract; a wildcard
|
||||||
|
// match marks its own first range.
|
||||||
|
Some(term) => Some(snippet::extract_folded(text, folded, &[term], &opts)),
|
||||||
|
None => pattern.find_first_folded(folded).map(|r| {
|
||||||
|
// A greedy pattern can match megabytes; clamp before the window.
|
||||||
|
let r = clamp_match_range(text, r, SNIPPET_WINDOW_CHARS);
|
||||||
|
snippet::window_around(text, (r.start, r.end), &opts)
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The fuzzy full-text match in one document body: how many times the term
|
||||||
|
/// occurs within the edit budget, and the Content Match window cut around
|
||||||
|
/// the first occurrence at the cascade's own width. `None` when it does not
|
||||||
|
/// occur at all.
|
||||||
|
///
|
||||||
|
/// Shared with [`crate::live`] for the same reason as [`text_snippet`]: a
|
||||||
|
/// fuzzy row whose file changes has to be re-cut the way it was cut, and
|
||||||
|
/// bitap's range is what it was cut around. `bitap` is built once by the
|
||||||
|
/// caller — per scan in the pass, per arm in the live watcher — since
|
||||||
|
/// building it is the cost, and `folded` must be `text` ASCII-lowercased.
|
||||||
|
pub fn fuzzy_snippet(
|
||||||
|
bitap: &crate::search::fuzzy::Bitap,
|
||||||
|
text: &str,
|
||||||
|
folded: &str,
|
||||||
|
) -> Option<(usize, snippet::Snippet)> {
|
||||||
|
let opts = snippet::Options {
|
||||||
|
approx_chars: SNIPPET_WINDOW_CHARS,
|
||||||
|
};
|
||||||
|
// `first` is `Some` exactly when `count` is non-zero: it *is* the first
|
||||||
|
// of them.
|
||||||
|
let (count, first) = bitap.count_and_first(folded.as_bytes());
|
||||||
|
first.map(|range| (count, snippet::window_around(text, range, &opts)))
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub struct Outcome {
|
pub struct Outcome {
|
||||||
pub total: usize,
|
pub total: usize,
|
||||||
|
|
|
||||||
|
|
@ -10,82 +10,6 @@ enum RowHit {
|
||||||
Defer(SearchHit),
|
Defer(SearchHit),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Reusable decode buffer and decompression context for the passes that read
|
|
||||||
/// document text.
|
|
||||||
///
|
|
||||||
/// `zstd::decode_all` builds and tears down a `ZSTD_DCtx` *and* allocates a
|
|
||||||
/// fresh output `Vec` on every call, and it is called once per candidate row.
|
|
||||||
/// One context and one buffer, reused across a whole scan, make that a
|
|
||||||
/// per-scan cost instead of a per-row one.
|
|
||||||
struct DocDecoder {
|
|
||||||
dctx: zstd::bulk::Decompressor<'static>,
|
|
||||||
buf: Vec<u8>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Where [`DocDecoder::decode`]'s buffer starts before it has seen a document.
|
|
||||||
/// Most extracted text is well under this, so the doubling below rarely runs.
|
|
||||||
const INITIAL_DOC_CAPACITY: usize = 64 * 1024;
|
|
||||||
|
|
||||||
/// Where the doubling stops. Stored text is capped at
|
|
||||||
/// `processing.maximum_text_size` (256 KiB by default), so this is far above
|
|
||||||
/// any legitimate document even if that setting is raised — past it, a failure
|
|
||||||
/// is a corrupt frame rather than a buffer that is too small.
|
|
||||||
const MAX_DOC_CAPACITY: usize = 64 * 1024 * 1024;
|
|
||||||
|
|
||||||
impl DocDecoder {
|
|
||||||
fn new() -> Result<Self, String> {
|
|
||||||
Ok(DocDecoder {
|
|
||||||
dctx: zstd::bulk::Decompressor::new().map_err(|e| e.to_string())?,
|
|
||||||
buf: Vec::new(),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Decompress `blob` and borrow the result as text.
|
|
||||||
///
|
|
||||||
/// Returns `None` for a corrupt frame or non-UTF-8 content. Nothing is
|
|
||||||
/// copied: the indexer stores UTF-8, so the bytes are borrowed in place
|
|
||||||
/// rather than run through `String::from_utf8_lossy(..).into_owned()`,
|
|
||||||
/// which duplicated the whole document even when it was already valid.
|
|
||||||
fn decode(&mut self, blob: &[u8]) -> Option<&str> {
|
|
||||||
self.buf.clear();
|
|
||||||
// `decompress_to_buffer` writes into spare capacity and fails rather
|
|
||||||
// than growing, so the room has to be there first.
|
|
||||||
//
|
|
||||||
// The frame header would say how much is needed, but the indexer
|
|
||||||
// writes with `zstd::encode_all`, which is *stream*-based and so
|
|
||||||
// records no content size — `get_frame_content_size` says `None` for
|
|
||||||
// every row this ever sees. Falling back to `zstd::decode_all` there
|
|
||||||
// looked harmless and was not: it builds a streaming decoder per call,
|
|
||||||
// which measured as one ~2.4 MiB allocation per document and 27 of the
|
|
||||||
// 30 GiB a fuzzy search moved through the allocator.
|
|
||||||
//
|
|
||||||
// So grow this buffer instead and keep reusing it. It settles at the
|
|
||||||
// largest document in the scan within the first few rows, after which
|
|
||||||
// decoding a row allocates nothing at all.
|
|
||||||
if let Ok(Some(size)) = zstd::zstd_safe::get_frame_content_size(blob) {
|
|
||||||
self.buf.reserve(usize::try_from(size).ok()?);
|
|
||||||
}
|
|
||||||
loop {
|
|
||||||
if self.buf.capacity() == 0 {
|
|
||||||
self.buf.reserve(INITIAL_DOC_CAPACITY);
|
|
||||||
}
|
|
||||||
match self.dctx.decompress_to_buffer(blob, &mut self.buf) {
|
|
||||||
Ok(_) => break,
|
|
||||||
// Too small, or corrupt — the bulk API cannot tell us which.
|
|
||||||
// Growing is only worth trying while the buffer is still
|
|
||||||
// smaller than any document could legitimately be.
|
|
||||||
Err(_) if self.buf.capacity() < MAX_DOC_CAPACITY => {
|
|
||||||
let bigger = self.buf.capacity().saturating_mul(2);
|
|
||||||
self.buf.clear();
|
|
||||||
self.buf.reserve(bigger);
|
|
||||||
}
|
|
||||||
Err(_) => return None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
std::str::from_utf8(&self.buf).ok()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Fold `text` into `dst` in place, reusing its allocation.
|
/// Fold `text` into `dst` in place, reusing its allocation.
|
||||||
///
|
///
|
||||||
/// The ASCII fold is byte-length preserving, which is what lets the cascade
|
/// The ASCII fold is byte-length preserving, which is what lets the cascade
|
||||||
|
|
@ -242,16 +166,10 @@ impl<'a> Cx<'a> {
|
||||||
let is_path_tier = rank >= 9.0;
|
let is_path_tier = rank >= 9.0;
|
||||||
// The "snippet" of a name or path hit is that field itself
|
// The "snippet" of a name or path hit is that field itself
|
||||||
// with the matched span marked.
|
// with the matched span marked.
|
||||||
let snip = snippet::Snippet {
|
let snip = snippet::whole_field(
|
||||||
ranges: vec![match_range],
|
if is_path_tier { path } else { name.as_str() },
|
||||||
window: if is_path_tier {
|
match_range,
|
||||||
path.to_string()
|
);
|
||||||
} else {
|
|
||||||
name.clone()
|
|
||||||
},
|
|
||||||
truncated_start: false,
|
|
||||||
truncated_end: false,
|
|
||||||
};
|
|
||||||
let (size, mtime) = size_and_mtime(row)?;
|
let (size, mtime) = size_and_mtime(row)?;
|
||||||
let hit = SearchHit {
|
let hit = SearchHit {
|
||||||
file_id,
|
file_id,
|
||||||
|
|
@ -327,12 +245,9 @@ impl<'a> Cx<'a> {
|
||||||
self.params_with_filters(Vec::new()),
|
self.params_with_filters(Vec::new()),
|
||||||
),
|
),
|
||||||
};
|
};
|
||||||
let snippet_opts = snippet::Options {
|
|
||||||
approx_chars: SNIPPET_WINDOW_CHARS,
|
|
||||||
};
|
|
||||||
// One decoder and one fold buffer for the whole scan; both are reused
|
// One decoder and one fold buffer for the whole scan; both are reused
|
||||||
// per row rather than reallocated.
|
// per row rather than reallocated.
|
||||||
let mut doc = DocDecoder::new()?;
|
let mut doc = crate::db::repo::DocDecoder::new()?;
|
||||||
let mut lower = String::new();
|
let mut lower = String::new();
|
||||||
// Decompression dominates: check cancellation every row.
|
// Decompression dominates: check cancellation every row.
|
||||||
self.scan_pass(&sql, params, 1, None, |cx, row, file_id, path| {
|
self.scan_pass(&sql, params, 1, None, |cx, row, file_id, path| {
|
||||||
|
|
@ -369,20 +284,7 @@ impl<'a> Cx<'a> {
|
||||||
if !folded {
|
if !folded {
|
||||||
fold_into(&mut lower, text);
|
fold_into(&mut lower, text);
|
||||||
}
|
}
|
||||||
// Literal terms keep the richer multi-occurrence
|
let snip = super::text_snippet(pattern, text, &lower);
|
||||||
// extract; a wildcard match marks its own first range.
|
|
||||||
let snip = match pattern.literal() {
|
|
||||||
Some(term) => Some(snippet::extract_folded(
|
|
||||||
text,
|
|
||||||
&lower,
|
|
||||||
&[term],
|
|
||||||
&snippet_opts,
|
|
||||||
)),
|
|
||||||
None => pattern.find_first_folded(&lower).map(|r| {
|
|
||||||
let r = clamp_match_range(text, r, SNIPPET_WINDOW_CHARS);
|
|
||||||
snippet::window_around(text, (r.start, r.end), &snippet_opts)
|
|
||||||
}),
|
|
||||||
};
|
|
||||||
(stage as f64 + count_frac(count), stage as u8, snip)
|
(stage as f64 + count_frac(count), stage as u8, snip)
|
||||||
}
|
}
|
||||||
// No stored text: can't case-verify or count. On the
|
// No stored text: can't case-verify or count. On the
|
||||||
|
|
@ -465,15 +367,14 @@ impl<'a> Cx<'a> {
|
||||||
if !cx.regex_accepts(file_id, path, None)? {
|
if !cx.regex_accepts(file_id, path, None)? {
|
||||||
return Ok(RowHit::Skip);
|
return Ok(RowHit::Skip);
|
||||||
}
|
}
|
||||||
// Mark the approximate matched span in the matched field;
|
// The matched field itself with the fuzzy span marked — the
|
||||||
// window_around clamps and aligns.
|
// same shape pass A emits, and what `SearchHit::snippet`
|
||||||
let snip = Some(snippet::window_around(
|
// documents for the name and path tiers. Windowing it here
|
||||||
field,
|
// used to hand back a *suffix* whenever the match sat past
|
||||||
range,
|
// two thirds of the way through, which broke that contract
|
||||||
&snippet::Options {
|
// and left a frontend unable to line the ranges up against
|
||||||
approx_chars: field.len().saturating_mul(2).max(8),
|
// the field it paints.
|
||||||
},
|
let snip = Some(snippet::whole_field(field, range));
|
||||||
));
|
|
||||||
let is_path_tier = rank >= 11.0;
|
let is_path_tier = rank >= 11.0;
|
||||||
let (size, mtime) = size_and_mtime(row)?;
|
let (size, mtime) = size_and_mtime(row)?;
|
||||||
let hit = SearchHit {
|
let hit = SearchHit {
|
||||||
|
|
@ -483,7 +384,14 @@ impl<'a> Cx<'a> {
|
||||||
size,
|
size,
|
||||||
mtime,
|
mtime,
|
||||||
rank,
|
rank,
|
||||||
stage: rank as u8,
|
// Stamped, not truncated from `rank`: this is the one pass
|
||||||
|
// whose ranks carry a fraction large enough to reach the
|
||||||
|
// next integer. `edit_budget` is only warned about above
|
||||||
|
// 3, so a distance of 10 makes rank 8.0 — and truncating
|
||||||
|
// that would file a *filename* hit under stage 8, the
|
||||||
|
// fuzzy full-text tier, telling every frontend to render
|
||||||
|
// it as a content match.
|
||||||
|
stage: if is_path_tier { 11 } else { 7 },
|
||||||
snippet: snip,
|
snippet: snip,
|
||||||
};
|
};
|
||||||
Ok(if is_path_tier {
|
Ok(if is_path_tier {
|
||||||
|
|
@ -518,11 +426,8 @@ impl<'a> Cx<'a> {
|
||||||
HIT_COLUMNS, self.query.filter_sql
|
HIT_COLUMNS, self.query.filter_sql
|
||||||
);
|
);
|
||||||
let params = self.params_with_filters(Vec::new());
|
let params = self.params_with_filters(Vec::new());
|
||||||
let snippet_opts = snippet::Options {
|
|
||||||
approx_chars: SNIPPET_WINDOW_CHARS,
|
|
||||||
};
|
|
||||||
// One decoder and one fold buffer for the whole scan, reused per row.
|
// One decoder and one fold buffer for the whole scan, reused per row.
|
||||||
let mut doc = DocDecoder::new()?;
|
let mut doc = crate::db::repo::DocDecoder::new()?;
|
||||||
let mut folded = String::new();
|
let mut folded = String::new();
|
||||||
// Decompression dominates: check cancellation every row.
|
// Decompression dominates: check cancellation every row.
|
||||||
self.scan_pass(&sql, params, 1, None, |cx, row, file_id, path| {
|
self.scan_pass(&sql, params, 1, None, |cx, row, file_id, path| {
|
||||||
|
|
@ -537,14 +442,12 @@ impl<'a> Cx<'a> {
|
||||||
// ASCII folding is byte-length preserving, so ranges found in
|
// ASCII folding is byte-length preserving, so ranges found in
|
||||||
// the folded buffer are valid in the original.
|
// the folded buffer are valid in the original.
|
||||||
fold_into(&mut folded, text);
|
fold_into(&mut folded, text);
|
||||||
let (count, first) = bitap.count_and_first(folded.as_bytes());
|
let Some((count, snip)) = super::fuzzy_snippet(&bitap, text, &folded) else {
|
||||||
if count == 0 {
|
|
||||||
return Ok(RowHit::Skip);
|
return Ok(RowHit::Skip);
|
||||||
}
|
};
|
||||||
if !cx.regex_accepts(file_id, path, Some(text))? {
|
if !cx.regex_accepts(file_id, path, Some(text))? {
|
||||||
return Ok(RowHit::Skip);
|
return Ok(RowHit::Skip);
|
||||||
}
|
}
|
||||||
let snip = first.map(|range| snippet::window_around(text, range, &snippet_opts));
|
|
||||||
let (size, mtime) = size_and_mtime(row)?;
|
let (size, mtime) = size_and_mtime(row)?;
|
||||||
Ok(RowHit::Emit(SearchHit {
|
Ok(RowHit::Emit(SearchHit {
|
||||||
file_id,
|
file_id,
|
||||||
|
|
@ -554,7 +457,7 @@ impl<'a> Cx<'a> {
|
||||||
mtime,
|
mtime,
|
||||||
rank: 8.0 + count_frac(count),
|
rank: 8.0 + count_frac(count),
|
||||||
stage: 8,
|
stage: 8,
|
||||||
snippet: snip,
|
snippet: Some(snip),
|
||||||
}))
|
}))
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
@ -587,16 +490,10 @@ impl<'a> Cx<'a> {
|
||||||
None => return Ok(RowHit::Skip),
|
None => return Ok(RowHit::Skip),
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
let snip = snippet::Snippet {
|
let snip = snippet::whole_field(
|
||||||
ranges: vec![match_range],
|
if is_path_tier { path } else { name.as_str() },
|
||||||
window: if is_path_tier {
|
match_range,
|
||||||
path.to_string()
|
);
|
||||||
} else {
|
|
||||||
name.clone()
|
|
||||||
},
|
|
||||||
truncated_start: false,
|
|
||||||
truncated_end: false,
|
|
||||||
};
|
|
||||||
let (size, mtime) = size_and_mtime(row)?;
|
let (size, mtime) = size_and_mtime(row)?;
|
||||||
let hit = SearchHit {
|
let hit = SearchHit {
|
||||||
file_id,
|
file_id,
|
||||||
|
|
@ -631,7 +528,7 @@ impl<'a> Cx<'a> {
|
||||||
approx_chars: SNIPPET_WINDOW_CHARS,
|
approx_chars: SNIPPET_WINDOW_CHARS,
|
||||||
};
|
};
|
||||||
// One decoder for the whole scan, reused per row.
|
// One decoder for the whole scan, reused per row.
|
||||||
let mut doc = DocDecoder::new()?;
|
let mut doc = crate::db::repo::DocDecoder::new()?;
|
||||||
// Decompression dominates: check cancellation every row.
|
// Decompression dominates: check cancellation every row.
|
||||||
self.scan_pass(&sql, params, 1, None, |_cx, row, file_id, path| {
|
self.scan_pass(&sql, params, 1, None, |_cx, row, file_id, path| {
|
||||||
let blob: Option<&[u8]> = row
|
let blob: Option<&[u8]> = row
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,10 @@ const MAX_REGISTERS: usize = 22;
|
||||||
pub struct Bitap {
|
pub struct Bitap {
|
||||||
/// `masks[c]` has bit `i` set iff `pattern[i] == c`.
|
/// `masks[c]` has bit `i` set iff `pattern[i] == c`.
|
||||||
masks: [u64; 256],
|
masks: [u64; 256],
|
||||||
|
/// The same table for the *reversed* pattern, which is what lets
|
||||||
|
/// [`Bitap::match_start`] find where a match began by scanning backwards
|
||||||
|
/// from where it ended.
|
||||||
|
rev_masks: [u64; 256],
|
||||||
/// Pattern length in bytes (1..=64).
|
/// Pattern length in bytes (1..=64).
|
||||||
len: usize,
|
len: usize,
|
||||||
/// Maximum edit distance.
|
/// Maximum edit distance.
|
||||||
|
|
@ -36,11 +40,14 @@ impl Bitap {
|
||||||
return None;
|
return None;
|
||||||
}
|
}
|
||||||
let mut masks = [0u64; 256];
|
let mut masks = [0u64; 256];
|
||||||
|
let mut rev_masks = [0u64; 256];
|
||||||
for (i, &b) in pattern.iter().enumerate() {
|
for (i, &b) in pattern.iter().enumerate() {
|
||||||
masks[b as usize] |= 1u64 << i;
|
masks[b as usize] |= 1u64 << i;
|
||||||
|
rev_masks[b as usize] |= 1u64 << (pattern.len() - 1 - i);
|
||||||
}
|
}
|
||||||
Some(Bitap {
|
Some(Bitap {
|
||||||
masks,
|
masks,
|
||||||
|
rev_masks,
|
||||||
len: pattern.len(),
|
len: pattern.len(),
|
||||||
k,
|
k,
|
||||||
})
|
})
|
||||||
|
|
@ -58,9 +65,14 @@ impl Bitap {
|
||||||
|
|
||||||
/// Advance all registers by one haystack byte. Returns the smallest
|
/// Advance all registers by one haystack byte. Returns the smallest
|
||||||
/// error count d for which the full pattern just matched, if any.
|
/// error count d for which the full pattern just matched, if any.
|
||||||
|
///
|
||||||
|
/// `masks` selects the direction: [`Bitap::masks`] to scan forwards,
|
||||||
|
/// [`Bitap::rev_masks`] to scan backwards. Everything else — `len`, `k`,
|
||||||
|
/// the `done` bit, `reset` — is the same either way, since a reversed
|
||||||
|
/// pattern is still a pattern of the same length.
|
||||||
#[inline]
|
#[inline]
|
||||||
fn step(&self, r: &mut [u64], byte: u8) -> Option<usize> {
|
fn step(&self, masks: &[u64; 256], r: &mut [u64], byte: u8) -> Option<usize> {
|
||||||
let mask = self.masks[byte as usize];
|
let mask = masks[byte as usize];
|
||||||
let done = 1u64 << (self.len - 1);
|
let done = 1u64 << (self.len - 1);
|
||||||
let mut hit = None;
|
let mut hit = None;
|
||||||
let mut prev_old = r[0]; // R_old[d-1] for the d-th iteration
|
let mut prev_old = r[0]; // R_old[d-1] for the d-th iteration
|
||||||
|
|
@ -83,52 +95,127 @@ impl Bitap {
|
||||||
hit
|
hit
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Minimum edit distance (≤ k) of any occurrence of the pattern in
|
/// Where the match that ended at `end` with `errors` edits began.
|
||||||
/// `hay`, or `None` if nothing matches within k edits.
|
///
|
||||||
pub fn best_distance(&self, hay: &[u8]) -> Option<usize> {
|
/// The forward scan knows an occurrence's *end* exactly — that is the bit
|
||||||
self.best_distance_and_first(hay).map(|(d, _)| d)
|
/// it tests — but not its start, and with an insertion or a deletion the
|
||||||
}
|
/// match is not `len` bytes long, so `end - len` is simply the wrong
|
||||||
|
/// offset. Highlighting it put the marks a byte or two off the match and
|
||||||
/// [`best_distance`](Self::best_distance) plus the first match's
|
/// over whatever preceded it: `repot` against `1Reporter` marked `1Repo`.
|
||||||
/// approximate byte range, from one sweep. The range carries the same
|
///
|
||||||
/// caveat as [`count_and_first`](Self::count_and_first): it assumes a
|
/// So the same automaton runs over the *reversed* pattern, backwards from
|
||||||
/// pattern-length match, so edits can shift the true start by up to `k`.
|
/// `end`. The first position it accepts **within `errors` edits** is the
|
||||||
pub fn best_distance_and_first(&self, hay: &[u8]) -> Option<(usize, (usize, usize))> {
|
/// start. That bound is what makes this correct rather than merely
|
||||||
|
/// plausible: the reversed pattern will also accept far shorter spans by
|
||||||
|
/// spending its whole budget on deletions — against `xabc` with a 2-edit
|
||||||
|
/// budget it accepts `c` alone on the very first byte — and taking that
|
||||||
|
/// would mark one letter of an exact three-letter match. The true
|
||||||
|
/// alignment costs the same read either way, so requiring `≤ errors`
|
||||||
|
/// rejects the cheap wrong answers and is still guaranteed to fire at or
|
||||||
|
/// before the real start.
|
||||||
|
///
|
||||||
|
/// Which occurrence gets marked is settled elsewhere — see
|
||||||
|
/// [`Bitap::refine_end`]. The rule both producers land on is *the earliest
|
||||||
|
/// alignment at the smallest edit distance*, so a mark is never longer
|
||||||
|
/// than the term and never shorter by more than the budget.
|
||||||
|
fn match_start(&self, hay: &[u8], end: usize, errors: usize) -> usize {
|
||||||
|
// A ≤k-edit alignment of a len-byte pattern is at most len+k long,
|
||||||
|
// so nothing before this can be the start.
|
||||||
|
let floor = end.saturating_sub(self.len + self.k);
|
||||||
let mut r = [0u64; MAX_REGISTERS];
|
let mut r = [0u64; MAX_REGISTERS];
|
||||||
self.reset(&mut r);
|
self.reset(&mut r);
|
||||||
let mut best: Option<(usize, (usize, usize))> = None;
|
for (back, &b) in hay[floor..end].iter().rev().enumerate() {
|
||||||
for (i, &b) in hay.iter().enumerate() {
|
if self
|
||||||
if let Some(d) = self.step(&mut r, b) {
|
.step(&self.rev_masks, &mut r, b)
|
||||||
let end = i + 1;
|
.is_some_and(|d| d <= errors)
|
||||||
let range = (end.saturating_sub(self.len), end);
|
{
|
||||||
if d == 0 {
|
return end - (back + 1);
|
||||||
return Some((0, range));
|
|
||||||
}
|
}
|
||||||
if best.is_none_or(|(cur, _)| d < cur) {
|
}
|
||||||
best = Some((d, range));
|
// Unreachable: the forward scan proved an alignment ends here, and
|
||||||
|
// reversed it costs the same. Falling back to the floor keeps a
|
||||||
|
// hypothetical miss inside the haystack.
|
||||||
|
floor
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Improve on the *earliest* accepting end by looking a little past it.
|
||||||
|
///
|
||||||
|
/// The automaton accepts as soon as a leading part of the pattern has
|
||||||
|
/// matched, paying for the rest with trailing deletions — so the first
|
||||||
|
/// end it reports is systematically short. Searching `abcdef` over
|
||||||
|
/// `zzabcdefzz` accepts after `abcd`, two deletions, with the whole word
|
||||||
|
/// sitting right there.
|
||||||
|
///
|
||||||
|
/// Each further byte can turn one of those deletions into a match, so a
|
||||||
|
/// better alignment ends at most `errors` bytes later and never more.
|
||||||
|
/// Stepping a *copy* of the registers that far finds it without
|
||||||
|
/// disturbing the caller's scan, or its count.
|
||||||
|
fn refine_end(
|
||||||
|
&self,
|
||||||
|
hay: &[u8],
|
||||||
|
r: &[u64; MAX_REGISTERS],
|
||||||
|
end: usize,
|
||||||
|
errors: usize,
|
||||||
|
) -> (usize, usize) {
|
||||||
|
let mut best = (errors, end);
|
||||||
|
let mut probe = *r;
|
||||||
|
for (ahead, &b) in hay[end..].iter().take(errors).enumerate() {
|
||||||
|
if let Some(d) = self.step(&self.masks, &mut probe, b) {
|
||||||
|
if d < best.0 {
|
||||||
|
best = (d, end + ahead + 1);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
best
|
best
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Count non-overlapping occurrences (at ≤ k edits) and report the
|
/// The smallest edit distance (≤ k) at which the pattern occurs in `hay`,
|
||||||
/// first match's approximate byte range in `hay`. After each hit the
|
/// and that occurrence's byte range — the span a frontend marks.
|
||||||
/// automaton resets, so an exact match followed by trailing bytes
|
///
|
||||||
/// counts once, and overlapping suffix matches don't inflate counts.
|
/// This one sweeps the whole haystack, so it finds the best alignment
|
||||||
/// The reported range assumes pattern-length matches — edits can shift
|
/// without help; [`Bitap::count_and_first`] resets after every hit and
|
||||||
/// the true start by up to k bytes, which is fine for snippet windows.
|
/// needs [`Bitap::refine_end`] instead.
|
||||||
|
pub fn best_distance_and_first(&self, hay: &[u8]) -> Option<(usize, (usize, usize))> {
|
||||||
|
let mut r = [0u64; MAX_REGISTERS];
|
||||||
|
self.reset(&mut r);
|
||||||
|
// (errors, end). The start is resolved once, at the end, rather than
|
||||||
|
// per improvement — `match_start` is a second scan, however short.
|
||||||
|
let mut best: Option<(usize, usize)> = None;
|
||||||
|
for (i, &b) in hay.iter().enumerate() {
|
||||||
|
if let Some(d) = self.step(&self.masks, &mut r, b) {
|
||||||
|
let end = i + 1;
|
||||||
|
if d == 0 {
|
||||||
|
best = Some((0, end));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if best.is_none_or(|(cur, _)| d < cur) {
|
||||||
|
best = Some((d, end));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
best.map(|(d, end)| (d, (self.match_start(hay, end, d), end)))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Count non-overlapping occurrences (at ≤ k edits) and report the first
|
||||||
|
/// one's byte range in `hay`. After each hit the automaton resets, so an
|
||||||
|
/// exact match followed by trailing bytes counts once, and overlapping
|
||||||
|
/// suffix matches don't inflate counts.
|
||||||
|
///
|
||||||
|
/// The range is the occurrence itself: [`Bitap::refine_end`] settles which
|
||||||
|
/// end, then [`Bitap::match_start`] finds where it began. Both run for the
|
||||||
|
/// first hit only, and both are bounded by the edit budget, so the cost is
|
||||||
|
/// O(len + k) per row rather than per byte.
|
||||||
pub fn count_and_first(&self, hay: &[u8]) -> (usize, Option<(usize, usize)>) {
|
pub fn count_and_first(&self, hay: &[u8]) -> (usize, Option<(usize, usize)>) {
|
||||||
let mut r = [0u64; MAX_REGISTERS];
|
let mut r = [0u64; MAX_REGISTERS];
|
||||||
self.reset(&mut r);
|
self.reset(&mut r);
|
||||||
let mut count = 0usize;
|
let mut count = 0usize;
|
||||||
let mut first: Option<(usize, usize)> = None;
|
let mut first: Option<(usize, usize)> = None;
|
||||||
for (i, &b) in hay.iter().enumerate() {
|
for (i, &b) in hay.iter().enumerate() {
|
||||||
if self.step(&mut r, b).is_some() {
|
if let Some(d) = self.step(&self.masks, &mut r, b) {
|
||||||
count += 1;
|
count += 1;
|
||||||
if first.is_none() {
|
if first.is_none() {
|
||||||
let end = i + 1;
|
let (errors, end) = self.refine_end(hay, &r, i + 1, d);
|
||||||
first = Some((end.saturating_sub(self.len), end));
|
first = Some((self.match_start(hay, end, errors), end));
|
||||||
}
|
}
|
||||||
self.reset(&mut r);
|
self.reset(&mut r);
|
||||||
}
|
}
|
||||||
|
|
@ -158,7 +245,28 @@ mod tests {
|
||||||
fn best(pattern: &str, hay: &str, k: usize) -> Option<usize> {
|
fn best(pattern: &str, hay: &str, k: usize) -> Option<usize> {
|
||||||
Bitap::new(pattern.as_bytes(), k)
|
Bitap::new(pattern.as_bytes(), k)
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.best_distance(hay.as_bytes())
|
.best_distance_and_first(hay.as_bytes())
|
||||||
|
.map(|(d, _)| d)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The slice of `hay` that a pattern's first occurrence marks — what a
|
||||||
|
/// frontend highlights.
|
||||||
|
fn marked<'h>(pattern: &str, hay: &'h str, k: usize) -> &'h str {
|
||||||
|
let (_, first) = Bitap::new(pattern.as_bytes(), k)
|
||||||
|
.unwrap()
|
||||||
|
.count_and_first(hay.as_bytes());
|
||||||
|
let (s, e) = first.expect("the pattern occurs");
|
||||||
|
&hay[s..e]
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`marked`] through the other range producer, which shares
|
||||||
|
/// `match_start` but reaches it by a different route.
|
||||||
|
fn marked_best<'h>(pattern: &str, hay: &'h str, k: usize) -> &'h str {
|
||||||
|
let (_, (s, e)) = Bitap::new(pattern.as_bytes(), k)
|
||||||
|
.unwrap()
|
||||||
|
.best_distance_and_first(hay.as_bytes())
|
||||||
|
.expect("the pattern occurs");
|
||||||
|
&hay[s..e]
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -223,20 +331,137 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn count_fuzzy_and_range_sane() {
|
fn count_fuzzy_and_range_is_the_occurrence_itself() {
|
||||||
let b = Bitap::new(b"hello", 1).unwrap();
|
let b = Bitap::new(b"hello", 1).unwrap();
|
||||||
let hay = b"say helo and hxllo again";
|
let hay = b"say helo and hxllo again";
|
||||||
let (count, first) = b.count_and_first(hay);
|
let (count, first) = b.count_and_first(hay);
|
||||||
assert_eq!(count, 2);
|
assert_eq!(count, 2);
|
||||||
let (s, e) = first.unwrap();
|
// The occurrence, not a five-byte window ending where it ends: that
|
||||||
assert!(s < e && e <= hay.len());
|
// reached back over the space and marked " helo".
|
||||||
let window = &hay[s..e];
|
assert_eq!(first, Some((4, 8)));
|
||||||
|
assert_eq!(&hay[4..8], b"helo");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The reported bug, exactly: `repot` marked `1Repo` in `1Reporter`.
|
||||||
|
///
|
||||||
|
/// The match is `repo` — one deletion, dropping the `t` — so it is four
|
||||||
|
/// bytes where the term is five, and a range assumed to be term-length
|
||||||
|
/// reached one byte too far left, over the `1`. Both producers, since
|
||||||
|
/// they share `match_start`.
|
||||||
|
#[test]
|
||||||
|
fn a_match_shorter_than_the_term_is_still_marked_exactly() {
|
||||||
|
assert_eq!(marked("repot", "1reporter", 1), "repo");
|
||||||
|
assert_eq!(marked_best("repot", "1reporter", 1), "repo");
|
||||||
|
|
||||||
|
// Substitution keeps the length, which is the case that always
|
||||||
|
// worked — worth holding, since it is the one the old arithmetic got
|
||||||
|
// right by accident.
|
||||||
|
assert_eq!(marked("hello", "xx hxllo xx", 1), "hxllo");
|
||||||
|
assert_eq!(marked_best("hello", "xx hxllo xx", 1), "hxllo");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Text with a byte inserted into the term marks up to the insertion, not
|
||||||
|
/// across it: `abxc` is a one-edit alignment of `abc`, but so is the `ab`
|
||||||
|
/// that ends two bytes earlier, and the rule is the *earliest* alignment
|
||||||
|
/// at the best distance. Nothing longer than the term can win — spanning
|
||||||
|
/// an inserted byte costs an edit, and deleting instead costs the same and
|
||||||
|
/// ends sooner.
|
||||||
|
#[test]
|
||||||
|
fn an_insertion_marks_up_to_it_rather_than_over_it() {
|
||||||
|
assert_eq!(marked("abc", "abxcd", 1), "ab");
|
||||||
|
assert_eq!(marked_best("abc", "zzabxczz", 1), "ab");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The trap in resolving the start backwards: the reversed pattern will
|
||||||
|
/// happily accept a much shorter span by spending its budget on
|
||||||
|
/// deletions, so taking its *first* acceptance marks one letter of an
|
||||||
|
/// exact match. `abc` occurs verbatim in `xabc`, and a 2-edit budget lets
|
||||||
|
/// the reverse pass accept `c` alone one byte in.
|
||||||
|
///
|
||||||
|
/// Only the whole-haystack producer can reach the trap — it is the one
|
||||||
|
/// that reports an exact match while the budget is still generous, so
|
||||||
|
/// `errors` is 0 where `k` is 2.
|
||||||
|
#[test]
|
||||||
|
fn a_generous_budget_does_not_shrink_an_exact_match() {
|
||||||
|
assert_eq!(marked_best("abc", "xabc", 2), "abc");
|
||||||
|
assert_eq!(marked_best("abcdef", "zzabcdefzz", 2), "abcdef");
|
||||||
|
assert_eq!(marked("hello", "say hello world", 2), "hello");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The automaton accepts as soon as a leading part of the term has
|
||||||
|
/// matched, spending the rest of the budget on trailing deletions — so
|
||||||
|
/// the earliest end is short, and marking it highlighted `abcd` for a
|
||||||
|
/// search for `abcdef` with the whole word right there. `refine_end`
|
||||||
|
/// looks the budget's worth of bytes past the first acceptance.
|
||||||
|
///
|
||||||
|
/// Checked against a brute-force Levenshtein oracle over every span.
|
||||||
|
#[test]
|
||||||
|
fn the_mark_is_not_truncated_to_a_leading_part_of_the_term() {
|
||||||
|
assert_eq!(marked("abcdef", "zzabcdefzz", 2), "abcdef");
|
||||||
|
assert_eq!(marked("abc", "xabc", 1), "abc");
|
||||||
|
assert_eq!(marked("reports", "the report went out", 2), "report");
|
||||||
|
|
||||||
|
// Not every term can be extended: `repot` against `1reporter` stops
|
||||||
|
// at `repo` because the next byte (`r`) costs an edit of its own, so
|
||||||
|
// one is the best it does either way.
|
||||||
|
assert_eq!(marked("repot", "1reporter", 1), "repo");
|
||||||
|
// And an alignment already at zero errors has nothing to improve.
|
||||||
|
assert_eq!(marked("hello", "say hello world", 0), "hello");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// However the span is chosen it is a real alignment at the best distance,
|
||||||
|
/// so it is never longer than the term and never shorter by more than the
|
||||||
|
/// budget. That bound is what keeps a mark recognisable: at the production
|
||||||
|
/// ladder of one edit per three characters, a mark is always at least two
|
||||||
|
/// thirds of the term.
|
||||||
|
#[test]
|
||||||
|
fn the_marked_span_is_within_the_budget_of_the_terms_length() {
|
||||||
|
for (term, hay) in [
|
||||||
|
("repot", "1reporter"),
|
||||||
|
("abcdef", "zzabcdefzz"),
|
||||||
|
("quarterly", "the quartrly budget"),
|
||||||
|
("hello", "say helo and hxllo again"),
|
||||||
|
("reports", "the report went out"),
|
||||||
|
] {
|
||||||
|
let k = edit_budget(term.len(), 2).expect("a real budget");
|
||||||
|
let span = marked(term, hay, k).len();
|
||||||
assert!(
|
assert!(
|
||||||
std::str::from_utf8(window).unwrap().contains("hel"),
|
span <= term.len() && term.len() - span <= k,
|
||||||
"first range should cover the first hit, got {:?}",
|
"{term:?} in {hay:?} (k={k}) marked {span} bytes"
|
||||||
std::str::from_utf8(window)
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_zero_budget_marks_exactly_the_term() {
|
||||||
|
assert_eq!(marked("hello", "say hello world", 0), "hello");
|
||||||
|
assert_eq!(marked("ab", "ab ab", 0), "ab");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A match at the very start, and one whose end is inside the term's own
|
||||||
|
/// length, are where the offset arithmetic can underflow.
|
||||||
|
#[test]
|
||||||
|
fn a_match_at_the_start_of_the_haystack_stays_in_bounds() {
|
||||||
|
assert_eq!(marked("repot", "reporter", 1), "repo");
|
||||||
|
// The haystack is shorter than the term: "ab" matches "abc" with one
|
||||||
|
// deletion, ending at 2.
|
||||||
|
let (_, first) = Bitap::new(b"abc", 1).unwrap().count_and_first(b"ab");
|
||||||
|
assert_eq!(first, Some((0, 2)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Bitap works on bytes over an ASCII-folded copy, so a range can land
|
||||||
|
/// inside a multi-byte character. `snippet::aligned_range` is what widens
|
||||||
|
/// it before anything slices; this only pins that the range stays inside
|
||||||
|
/// the haystack so that alignment has something valid to work from.
|
||||||
|
#[test]
|
||||||
|
fn a_range_over_multibyte_text_stays_within_the_haystack() {
|
||||||
|
let hay = "café notes — le rapport";
|
||||||
|
let (_, first) = Bitap::new(b"raport", 1)
|
||||||
|
.unwrap()
|
||||||
|
.count_and_first(hay.as_bytes());
|
||||||
|
let (s, e) = first.expect("one deletion from 'rapport'");
|
||||||
|
assert!(s < e && e <= hay.len(), "({s}, {e}) outside {}", hay.len());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn edit_budget_default_cap_is_the_historic_ladder() {
|
fn edit_budget_default_cap_is_the_historic_ladder() {
|
||||||
|
|
@ -335,7 +560,10 @@ mod tests {
|
||||||
let pattern: Vec<u8> = (0..plen).map(|_| alphabet[rng() % 4]).collect();
|
let pattern: Vec<u8> = (0..plen).map(|_| alphabet[rng() % 4]).collect();
|
||||||
let hay: Vec<u8> = (0..hlen).map(|_| alphabet[rng() % 4]).collect();
|
let hay: Vec<u8> = (0..hlen).map(|_| alphabet[rng() % 4]).collect();
|
||||||
for k in 0..=4 {
|
for k in 0..=4 {
|
||||||
let got = Bitap::new(&pattern, k).unwrap().best_distance(&hay);
|
let got = Bitap::new(&pattern, k)
|
||||||
|
.unwrap()
|
||||||
|
.best_distance_and_first(&hay)
|
||||||
|
.map(|(d, _)| d);
|
||||||
let want = oracle(&pattern, &hay, k);
|
let want = oracle(&pattern, &hay, k);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
got,
|
got,
|
||||||
|
|
|
||||||
|
|
@ -69,6 +69,55 @@ pub struct SearchHit {
|
||||||
pub snippet: Option<Snippet>,
|
pub snippet: Option<Snippet>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Which field a hit's [`SearchHit::snippet`] excerpts, derived from the
|
||||||
|
/// cascade stage. See the rank table at the top of [`crate::search::cascade`].
|
||||||
|
///
|
||||||
|
/// Frontends branch on this rather than on the raw stage number, so a new tier
|
||||||
|
/// is classified in one place instead of in every renderer.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum MatchField {
|
||||||
|
Name,
|
||||||
|
Contents,
|
||||||
|
Path,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How a [`MatchField::Contents`] hit matched its body — what has to be
|
||||||
|
/// re-run to cut its snippet again from the file as it now stands.
|
||||||
|
///
|
||||||
|
/// The two are not interchangeable: an exact tier's snippet is cut around the
|
||||||
|
/// literal term, and a fuzzy tier's around a bitap match the literal is
|
||||||
|
/// usually *absent* from. Re-cutting a fuzzy hit as if it were exact finds
|
||||||
|
/// nothing and reads as "the file no longer matches".
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum ContentTier {
|
||||||
|
/// Stages 5 and 6: the body contains the term as written.
|
||||||
|
Exact,
|
||||||
|
/// Stage 8: the body contains something within the fuzzy edit budget of
|
||||||
|
/// the term.
|
||||||
|
Fuzzy,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SearchHit {
|
||||||
|
pub fn match_field(&self) -> MatchField {
|
||||||
|
match self.stage {
|
||||||
|
1..=4 | 7 => MatchField::Name,
|
||||||
|
5 | 6 | 8 => MatchField::Contents,
|
||||||
|
// 9..=11, and whatever a later tier adds: the path is the safe
|
||||||
|
// reading, since it is the one field every hit carries in full.
|
||||||
|
_ => MatchField::Path,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `Some` for a hit whose snippet is a window on the file's body.
|
||||||
|
pub fn content_tier(&self) -> Option<ContentTier> {
|
||||||
|
match self.stage {
|
||||||
|
5 | 6 => Some(ContentTier::Exact),
|
||||||
|
8 => Some(ContentTier::Fuzzy),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone)]
|
#[derive(Debug, Clone)]
|
||||||
pub enum SearchUpdate {
|
pub enum SearchUpdate {
|
||||||
Started {
|
Started {
|
||||||
|
|
|
||||||
|
|
@ -52,13 +52,6 @@ impl Snippet {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Extract a snippet from `text` marking every occurrence of any term in
|
|
||||||
/// `terms` (ASCII-case-insensitive). With no terms or no matches, returns
|
|
||||||
/// the head of the text as the window with no ranges.
|
|
||||||
pub fn extract(text: &str, terms: &[&str], opts: &Options) -> Snippet {
|
|
||||||
extract_folded(text, &text.to_ascii_lowercase(), terms, opts)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// [`extract`] against a haystack the caller has already ASCII-folded.
|
/// [`extract`] against a haystack the caller has already ASCII-folded.
|
||||||
/// `folded` must be `text.to_ascii_lowercase()` — the fold is byte-length
|
/// `folded` must be `text.to_ascii_lowercase()` — the fold is byte-length
|
||||||
/// preserving, which is what lets offsets found in it slice the original.
|
/// preserving, which is what lets offsets found in it slice the original.
|
||||||
|
|
@ -139,6 +132,48 @@ pub fn extract_folded(text: &str, folded: &str, terms: &[&str], opts: &Options)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Clamp `range` into `text` and widen it to the nearest char boundaries.
|
||||||
|
///
|
||||||
|
/// Both callers below take ranges from matchers that work on bytes — bitap
|
||||||
|
/// over an ASCII-folded copy — so an endpoint can land inside a multi-byte
|
||||||
|
/// character. Slicing there panics, and `Snippet::ranges` promises boundaries.
|
||||||
|
fn aligned_range(text: &str, range: (usize, usize)) -> (usize, usize) {
|
||||||
|
let (mut start, mut end) = range;
|
||||||
|
start = start.min(text.len());
|
||||||
|
end = end.clamp(start, text.len());
|
||||||
|
while start > 0 && !text.is_char_boundary(start) {
|
||||||
|
start -= 1;
|
||||||
|
}
|
||||||
|
while end < text.len() && !text.is_char_boundary(end) {
|
||||||
|
end += 1;
|
||||||
|
}
|
||||||
|
(start, end)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The whole of `text` as the window, with `range` marked.
|
||||||
|
///
|
||||||
|
/// This is the shape [`crate::search::SearchHit::snippet`] documents for the
|
||||||
|
/// name and path tiers, and what lets a frontend highlight the matched span
|
||||||
|
/// inside its own Name or Path column: `window` is that field verbatim, so the
|
||||||
|
/// ranges index the field the column is already painting. A filename or a path
|
||||||
|
/// is short enough to carry whole, so there is nothing to gain by windowing it.
|
||||||
|
pub fn whole_field(text: &str, range: (usize, usize)) -> Snippet {
|
||||||
|
if text.is_empty() {
|
||||||
|
return Snippet::empty();
|
||||||
|
}
|
||||||
|
let (start, end) = aligned_range(text, range);
|
||||||
|
Snippet {
|
||||||
|
window: text.to_string(),
|
||||||
|
ranges: if end > start {
|
||||||
|
vec![(start, end)]
|
||||||
|
} else {
|
||||||
|
Vec::new()
|
||||||
|
},
|
||||||
|
truncated_start: false,
|
||||||
|
truncated_end: false,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Build a snippet window around one known match range in `text` (byte
|
/// Build a snippet window around one known match range in `text` (byte
|
||||||
/// offsets into `text`). Used by fuzzy full-text search, where the match
|
/// offsets into `text`). Used by fuzzy full-text search, where the match
|
||||||
/// was located by the fuzzy matcher rather than exact term search. The
|
/// was located by the fuzzy matcher rather than exact term search. The
|
||||||
|
|
@ -147,15 +182,7 @@ pub fn window_around(text: &str, range: (usize, usize), opts: &Options) -> Snipp
|
||||||
if text.is_empty() {
|
if text.is_empty() {
|
||||||
return Snippet::empty();
|
return Snippet::empty();
|
||||||
}
|
}
|
||||||
let (mut ms, mut me) = range;
|
let (ms, me) = aligned_range(text, range);
|
||||||
ms = ms.min(text.len());
|
|
||||||
me = me.clamp(ms, text.len());
|
|
||||||
while ms > 0 && !text.is_char_boundary(ms) {
|
|
||||||
ms -= 1;
|
|
||||||
}
|
|
||||||
while me < text.len() && !text.is_char_boundary(me) {
|
|
||||||
me += 1;
|
|
||||||
}
|
|
||||||
|
|
||||||
let pre_pad = opts.approx_chars / 3;
|
let pre_pad = opts.approx_chars / 3;
|
||||||
let mut win_start = ms.saturating_sub(pre_pad);
|
let mut win_start = ms.saturating_sub(pre_pad);
|
||||||
|
|
@ -244,6 +271,13 @@ fn coalesce_overlapping(v: Vec<(usize, usize)>) -> Vec<(usize, usize)> {
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
/// The tests were written against a since-removed `extract` wrapper.
|
||||||
|
/// Production always holds a fold buffer already, so the wrapper earned
|
||||||
|
/// nothing; folding here keeps its coverage of the window logic.
|
||||||
|
fn extract(text: &str, terms: &[&str], opts: &Options) -> Snippet {
|
||||||
|
extract_folded(text, &text.to_ascii_lowercase(), terms, opts)
|
||||||
|
}
|
||||||
|
|
||||||
fn opts_small() -> Options {
|
fn opts_small() -> Options {
|
||||||
Options { approx_chars: 40 }
|
Options { approx_chars: 40 }
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -21,12 +21,78 @@ pub fn zstd_of(text: &str) -> Option<Vec<u8>> {
|
||||||
crate::db::repo::encode_one(text, true).expect("zstd encode")
|
crate::db::repo::encode_one(text, true).expect("zstd encode")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// How old a leftover scratch directory must be before [`sweep_stale`] takes
|
||||||
|
/// it. Far longer than any test run, so a failure investigated the same day —
|
||||||
|
/// or the next morning — still has its tree.
|
||||||
|
const STALE_AFTER: std::time::Duration = std::time::Duration::from_secs(12 * 60 * 60);
|
||||||
|
|
||||||
|
/// Whether `name` is one of [`scratch_dir`]'s own directories.
|
||||||
|
///
|
||||||
|
/// Matched on the *shape* — `quicksearch-{tag}-{pid}-{seq}`, so the last two
|
||||||
|
/// dash-separated components must be numbers — rather than on the
|
||||||
|
/// `quicksearch-` prefix alone. `packaging/capture.sh` keeps its output in
|
||||||
|
/// `quicksearch-capture` in the same directory, and a prefix match would eat a
|
||||||
|
/// capture run's screenshots along with the litter.
|
||||||
|
fn is_scratch_name(name: &str) -> bool {
|
||||||
|
let Some(rest) = name.strip_prefix("quicksearch-") else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
let numeric = |part: Option<&str>| {
|
||||||
|
part.is_some_and(|p| !p.is_empty() && p.bytes().all(|b| b.is_ascii_digit()))
|
||||||
|
};
|
||||||
|
let mut tail = rest.rsplitn(3, '-');
|
||||||
|
// seq, then pid, and a tag must remain in front of them.
|
||||||
|
numeric(tail.next()) && numeric(tail.next()) && tail.next().is_some_and(|tag| !tag.is_empty())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Remove scratch directories left by runs that are long over.
|
||||||
|
///
|
||||||
|
/// Nothing here cleans up on the way *out*: a failed test's tree is most of
|
||||||
|
/// the evidence, which is why [`scratch_dir`] deliberately leaves it. But
|
||||||
|
/// passing tests leave theirs too, and most never remove it — so the temp
|
||||||
|
/// directory grew by roughly three hundred directories per full run and had
|
||||||
|
/// accumulated some nine thousand of them. Where `/tmp` is a tmpfs that is
|
||||||
|
/// gigabytes of RAM, which slows the whole suite and pushes the
|
||||||
|
/// timing-sensitive tests toward their budgets.
|
||||||
|
///
|
||||||
|
/// Sweeping on the way *in* keeps both halves: this run's evidence survives,
|
||||||
|
/// and so does yesterday's, while nothing accumulates without bound. Only
|
||||||
|
/// [`scratch_dir`]'s own naming is touched.
|
||||||
|
fn sweep_stale() {
|
||||||
|
let Ok(entries) = std::fs::read_dir(std::env::temp_dir()) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let now = std::time::SystemTime::now();
|
||||||
|
for entry in entries.flatten() {
|
||||||
|
let name = entry.file_name();
|
||||||
|
if !name.to_str().is_some_and(is_scratch_name) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let stale = entry
|
||||||
|
.metadata()
|
||||||
|
.and_then(|m| m.modified())
|
||||||
|
.ok()
|
||||||
|
.and_then(|t| now.duration_since(t).ok())
|
||||||
|
.is_some_and(|age| age >= STALE_AFTER);
|
||||||
|
if stale {
|
||||||
|
// Best effort throughout: two test binaries starting together race
|
||||||
|
// on the same directory and one of them loses, which is fine.
|
||||||
|
std::fs::remove_dir_all(entry.path()).ok();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// A fresh, empty directory under the system temp dir, named for `tag`.
|
/// A fresh, empty directory under the system temp dir, named for `tag`.
|
||||||
|
///
|
||||||
/// Not cleaned up on drop: when a test fails, the tree it built is most of
|
/// Not cleaned up on drop: when a test fails, the tree it built is most of
|
||||||
/// the evidence. Panics — a test that cannot create a directory has nothing
|
/// the evidence. Long-dead runs' trees are swept once per process instead —
|
||||||
/// left to assert.
|
/// see [`sweep_stale`]. Panics — a test that cannot create a directory has
|
||||||
|
/// nothing left to assert.
|
||||||
#[doc(hidden)]
|
#[doc(hidden)]
|
||||||
pub fn scratch_dir(tag: &str) -> PathBuf {
|
pub fn scratch_dir(tag: &str) -> PathBuf {
|
||||||
|
static SWEPT: std::sync::Once = std::sync::Once::new();
|
||||||
|
SWEPT.call_once(sweep_stale);
|
||||||
|
|
||||||
let mut p = std::env::temp_dir();
|
let mut p = std::env::temp_dir();
|
||||||
p.push(format!(
|
p.push(format!(
|
||||||
"quicksearch-{}-{}-{}",
|
"quicksearch-{}-{}-{}",
|
||||||
|
|
@ -95,4 +161,75 @@ mod tests {
|
||||||
touch(&deep, b"hi");
|
touch(&deep, b"hi");
|
||||||
assert_eq!(std::fs::read(&deep).unwrap(), b"hi");
|
assert_eq!(std::fs::read(&deep).unwrap(), b"hi");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The sweep runs against a shared temp directory, so what it matches is
|
||||||
|
/// the whole safety argument. `quicksearch-capture` is the one that would
|
||||||
|
/// hurt: `packaging/capture.sh` puts a run's screenshots and screencasts
|
||||||
|
/// there, and a prefix match would delete them mid-capture.
|
||||||
|
#[test]
|
||||||
|
fn only_scratch_directories_are_swept() {
|
||||||
|
for ours in [
|
||||||
|
"quicksearch-coord-1234-0",
|
||||||
|
"quicksearch-stall-heavy-1001402-7",
|
||||||
|
"quicksearch-a-0-0",
|
||||||
|
// Tags contain dashes of their own; only the last two components
|
||||||
|
// are read as numbers.
|
||||||
|
"quicksearch-sniff-binary-db-2621744-1",
|
||||||
|
] {
|
||||||
|
assert!(is_scratch_name(ours), "{ours} should be swept");
|
||||||
|
}
|
||||||
|
|
||||||
|
for theirs in [
|
||||||
|
// The capture output directory, the reason this is a shape match.
|
||||||
|
"quicksearch-capture",
|
||||||
|
"quicksearch",
|
||||||
|
"quicksearch-",
|
||||||
|
// A tag but no pid/seq pair.
|
||||||
|
"quicksearch-coord",
|
||||||
|
"quicksearch-coord-1234",
|
||||||
|
// Numbers, but nothing in front of them to be a tag.
|
||||||
|
"quicksearch-1234-0",
|
||||||
|
// Not ours at all.
|
||||||
|
"cargo-install-abc-1-2",
|
||||||
|
"tmp-quicksearch-coord-1-2",
|
||||||
|
] {
|
||||||
|
assert!(!is_scratch_name(theirs), "{theirs} must not be swept");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fresh directories survive; only long-dead runs are collected. Uses a
|
||||||
|
/// hand-built name rather than `scratch_dir` so the assertion is about the
|
||||||
|
/// age gate and not about whatever else the suite has left lying around.
|
||||||
|
#[test]
|
||||||
|
fn the_sweep_keeps_recent_trees_and_takes_old_ones() {
|
||||||
|
let fresh = scratch_dir("sweep-fresh");
|
||||||
|
touch(&fresh.join("evidence.txt"), b"kept");
|
||||||
|
|
||||||
|
// Same shape, but back-dated past the threshold. `set_times` is the
|
||||||
|
// only way to age a directory without waiting twelve hours for it.
|
||||||
|
let old = std::env::temp_dir().join(format!(
|
||||||
|
"quicksearch-sweep-old-{}-{}",
|
||||||
|
std::process::id(),
|
||||||
|
NEXT.fetch_add(1, Ordering::Relaxed)
|
||||||
|
));
|
||||||
|
std::fs::create_dir_all(&old).expect("create the aged directory");
|
||||||
|
let long_ago =
|
||||||
|
std::time::SystemTime::now() - STALE_AFTER - std::time::Duration::from_secs(60);
|
||||||
|
std::fs::File::open(&old)
|
||||||
|
.and_then(|d| {
|
||||||
|
d.set_times(
|
||||||
|
std::fs::FileTimes::new()
|
||||||
|
.set_accessed(long_ago)
|
||||||
|
.set_modified(long_ago),
|
||||||
|
)
|
||||||
|
})
|
||||||
|
.expect("back-date the aged directory");
|
||||||
|
|
||||||
|
sweep_stale();
|
||||||
|
|
||||||
|
assert!(fresh.exists(), "a fresh scratch tree was swept away");
|
||||||
|
assert!(!old.exists(), "a long-dead scratch tree survived the sweep");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&fresh).ok();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
299
crates/quicksearch-core/src/verify.rs
Normal file
299
crates/quicksearch-core/src/verify.rs
Normal file
|
|
@ -0,0 +1,299 @@
|
||||||
|
//! Byte-for-byte verification that a set of files really is identical.
|
||||||
|
//!
|
||||||
|
//! The index groups duplicates by `sha256(size ‖ first hash_length bytes)`
|
||||||
|
//! (see [`crate::file_handling`]), which reads a file's head and nothing else
|
||||||
|
//! — a deliberate trade, since hashing every byte on a disk is most of the
|
||||||
|
//! cost of indexing it. Files of the same size whose heads agree are therefore
|
||||||
|
//! listed as duplicates whether or not they are: a fixed-size VHD keeps what
|
||||||
|
//! makes it unique in a footer, and a freshly pre-allocated disk image is
|
||||||
|
//! zeroes as far as the head can see. This turns that advisory grouping into
|
||||||
|
//! an answer, for the moment before someone deletes something.
|
||||||
|
//!
|
||||||
|
//! No hashing here, by policy. A digest per file would be shorter code and the
|
||||||
|
//! same answer nearly always — but "nearly always" is what the head hash
|
||||||
|
//! already offers, and the whole point of asking a second time is that this
|
||||||
|
//! time the bytes are compared.
|
||||||
|
|
||||||
|
use std::fs::File;
|
||||||
|
use std::io::Read;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
/// Total read-buffer memory, split across the files being compared. A group is
|
||||||
|
/// usually two files and can be thousands — a hardlink farm, which is exactly
|
||||||
|
/// what `[indexing] ignore_patterns` warns about — so a per-file buffer of any
|
||||||
|
/// fixed size would become the largest allocation the process ever makes.
|
||||||
|
const CHUNK_BUDGET: usize = 8 * 1024 * 1024;
|
||||||
|
const MIN_CHUNK: usize = 16 * 1024;
|
||||||
|
const MAX_CHUNK: usize = 256 * 1024;
|
||||||
|
|
||||||
|
/// How often progress is emitted. Each one repaints the UI, and a chunk off a
|
||||||
|
/// warm page cache takes microseconds.
|
||||||
|
const PROGRESS_INTERVAL: Duration = Duration::from_millis(100);
|
||||||
|
|
||||||
|
/// What one member turned out to be, against the reference.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub enum MemberVerdict {
|
||||||
|
/// Same length, every byte agreed. The reference itself reads this.
|
||||||
|
Identical,
|
||||||
|
/// Offset of the first byte that disagreed.
|
||||||
|
DiffersAt(u64),
|
||||||
|
/// Lengths disagree, so nothing was read. Within a duplicate group this
|
||||||
|
/// can only mean a stale index — the hash covers the size.
|
||||||
|
LengthDiffers { len: u64, reference_len: u64 },
|
||||||
|
/// Could not be opened, or stopped being readable part way through.
|
||||||
|
Unreadable(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MemberVerdict {
|
||||||
|
pub fn is_identical(&self) -> bool {
|
||||||
|
matches!(self, MemberVerdict::Identical)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct VerifyReport {
|
||||||
|
/// Index into the input paths of the file everything else was compared
|
||||||
|
/// against: the first one that opened. `None` when none of them did.
|
||||||
|
pub reference: Option<usize>,
|
||||||
|
/// One verdict per input path, in the input order.
|
||||||
|
pub verdicts: Vec<MemberVerdict>,
|
||||||
|
/// Bytes actually read from disk, across every file.
|
||||||
|
pub bytes_read: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl VerifyReport {
|
||||||
|
/// Whether every member was read and matched. An empty or single-file set
|
||||||
|
/// is vacuously identical.
|
||||||
|
pub fn all_identical(&self) -> bool {
|
||||||
|
self.verdicts.iter().all(MemberVerdict::is_identical)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn differing(&self) -> usize {
|
||||||
|
self.verdicts.iter().filter(|v| !v.is_identical()).count()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub enum VerifyUpdate {
|
||||||
|
Progress { bytes_read: u64, bytes_total: u64 },
|
||||||
|
Done(VerifyReport),
|
||||||
|
Cancelled,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A file still in the running, with its own read buffer.
|
||||||
|
struct Live {
|
||||||
|
index: usize,
|
||||||
|
file: File,
|
||||||
|
buf: Vec<u8>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Compare every path against the first one that opens, byte for byte, and
|
||||||
|
/// report what each turned out to be.
|
||||||
|
///
|
||||||
|
/// Emits `Progress` while it works and exactly one terminal update — `Done`,
|
||||||
|
/// or `Cancelled` if `cancel` went up before the comparison finished.
|
||||||
|
pub fn verify_identical(paths: &[PathBuf], cancel: &AtomicBool, on: &mut dyn FnMut(VerifyUpdate)) {
|
||||||
|
// Checked before the files are even opened, so a run cancelled before it
|
||||||
|
// starts reports the cancellation rather than a verdict nobody waited for.
|
||||||
|
if cancel.load(Ordering::Relaxed) {
|
||||||
|
on(VerifyUpdate::Cancelled);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let mut verdicts = vec![MemberVerdict::Identical; paths.len()];
|
||||||
|
|
||||||
|
// The reference is the first path that both opens *and* stats, not simply
|
||||||
|
// the first path: one unreadable member must not cost the answer about all
|
||||||
|
// the others.
|
||||||
|
let mut reference: Option<(usize, File, u64)> = None;
|
||||||
|
let mut rest: Vec<(usize, File)> = Vec::new();
|
||||||
|
for (i, path) in paths.iter().enumerate() {
|
||||||
|
let file = match File::open(path) {
|
||||||
|
Ok(f) => f,
|
||||||
|
Err(e) => {
|
||||||
|
verdicts[i] = MemberVerdict::Unreadable(describe(path, &e));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if reference.is_some() {
|
||||||
|
rest.push((i, file));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
match file.metadata() {
|
||||||
|
Ok(m) => reference = Some((i, file, m.len())),
|
||||||
|
Err(e) => verdicts[i] = MemberVerdict::Unreadable(describe(path, &e)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let Some((reference, mut reference_file, reference_len)) = reference else {
|
||||||
|
on(VerifyUpdate::Done(VerifyReport {
|
||||||
|
reference: None,
|
||||||
|
verdicts,
|
||||||
|
bytes_read: 0,
|
||||||
|
}));
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
|
||||||
|
// A length mismatch is decided from the handles, before a byte is read.
|
||||||
|
let mut live: Vec<Live> = Vec::with_capacity(rest.len());
|
||||||
|
for (i, file) in rest {
|
||||||
|
match file.metadata() {
|
||||||
|
Ok(m) if m.len() != reference_len => {
|
||||||
|
verdicts[i] = MemberVerdict::LengthDiffers {
|
||||||
|
len: m.len(),
|
||||||
|
reference_len,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
Ok(_) => live.push(Live {
|
||||||
|
index: i,
|
||||||
|
file,
|
||||||
|
buf: Vec::new(),
|
||||||
|
}),
|
||||||
|
Err(e) => verdicts[i] = MemberVerdict::Unreadable(describe(&paths[i], &e)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let chunk = (CHUNK_BUDGET / (live.len() + 1)).clamp(MIN_CHUNK, MAX_CHUNK);
|
||||||
|
let mut reference_buf = vec![0u8; chunk];
|
||||||
|
for l in live.iter_mut() {
|
||||||
|
l.buf = vec![0u8; chunk];
|
||||||
|
}
|
||||||
|
|
||||||
|
let bytes_total = match live.len() {
|
||||||
|
0 => 0,
|
||||||
|
n => reference_len.saturating_mul(n as u64 + 1),
|
||||||
|
};
|
||||||
|
let mut bytes_read = 0u64;
|
||||||
|
let mut offset = 0u64;
|
||||||
|
// Backdated so the first chunk reports: a progress bar that only appears
|
||||||
|
// after the first interval reads as a frozen window on a slow disk, which
|
||||||
|
// is the case this is for.
|
||||||
|
let mut last_progress = Instant::now()
|
||||||
|
.checked_sub(PROGRESS_INTERVAL)
|
||||||
|
.unwrap_or_else(Instant::now);
|
||||||
|
|
||||||
|
while !live.is_empty() {
|
||||||
|
if cancel.load(Ordering::Relaxed) {
|
||||||
|
on(VerifyUpdate::Cancelled);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Termination is driven by what the reference actually reads rather
|
||||||
|
// than by the length it claimed, so a file truncated underneath us
|
||||||
|
// degrades to a short comparison instead of a hang or a false match.
|
||||||
|
let n = match read_chunk(&mut reference_file, &mut reference_buf) {
|
||||||
|
Ok(0) => break, // EOF: everything still live matched all the way.
|
||||||
|
Ok(n) => n,
|
||||||
|
Err(e) => {
|
||||||
|
verdicts[reference] = MemberVerdict::Unreadable(describe(&paths[reference], &e));
|
||||||
|
// Survivors agreed up to here but cannot be finished. Saying
|
||||||
|
// so is the only honest answer; "identical" would not be.
|
||||||
|
for l in live.iter() {
|
||||||
|
verdicts[l.index] = MemberVerdict::Unreadable(format!(
|
||||||
|
"compared only to byte {offset}: {} could not be read to the end",
|
||||||
|
paths[reference].display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
bytes_read += n as u64;
|
||||||
|
|
||||||
|
let mut i = 0;
|
||||||
|
while i < live.len() {
|
||||||
|
let (got, verdict) = {
|
||||||
|
let l = &mut live[i];
|
||||||
|
match read_chunk(&mut l.file, &mut l.buf[..n]) {
|
||||||
|
Ok(m) => {
|
||||||
|
let common = n.min(m);
|
||||||
|
if let Some(k) =
|
||||||
|
first_difference(&reference_buf[..common], &l.buf[..common])
|
||||||
|
{
|
||||||
|
(m, Some(MemberVerdict::DiffersAt(offset + k as u64)))
|
||||||
|
} else if m < n {
|
||||||
|
// Same length a moment ago, shorter now.
|
||||||
|
(
|
||||||
|
m,
|
||||||
|
Some(MemberVerdict::Unreadable(format!(
|
||||||
|
"{}: ended at byte {} while the file it was compared \
|
||||||
|
against had more",
|
||||||
|
paths[l.index].display(),
|
||||||
|
offset + m as u64
|
||||||
|
))),
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
(m, None)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(e) => (
|
||||||
|
0,
|
||||||
|
Some(MemberVerdict::Unreadable(describe(&paths[l.index], &e))),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
};
|
||||||
|
bytes_read += got as u64;
|
||||||
|
match verdict {
|
||||||
|
Some(v) => {
|
||||||
|
verdicts[live[i].index] = v;
|
||||||
|
live.swap_remove(i);
|
||||||
|
}
|
||||||
|
None => i += 1,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
offset += n as u64;
|
||||||
|
|
||||||
|
if last_progress.elapsed() >= PROGRESS_INTERVAL {
|
||||||
|
last_progress = Instant::now();
|
||||||
|
on(VerifyUpdate::Progress {
|
||||||
|
bytes_read,
|
||||||
|
bytes_total,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
on(VerifyUpdate::Done(VerifyReport {
|
||||||
|
reference: Some(reference),
|
||||||
|
verdicts,
|
||||||
|
bytes_read,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fill `buf` as far as the file allows, returning how much. Short reads are
|
||||||
|
/// resumed and `Interrupted` retried, the way `extract::plaintext` does, so a
|
||||||
|
/// short return really does mean end of file.
|
||||||
|
fn read_chunk(f: &mut File, buf: &mut [u8]) -> std::io::Result<usize> {
|
||||||
|
let mut filled = 0;
|
||||||
|
while filled < buf.len() {
|
||||||
|
match f.read(&mut buf[filled..]) {
|
||||||
|
Ok(0) => break,
|
||||||
|
Ok(n) => filled += n,
|
||||||
|
Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => {}
|
||||||
|
Err(e) => return Err(e),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(filled)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Offset of the first byte that differs. The equality test comes first
|
||||||
|
/// because it is a `memcmp`; the byte walk only ever runs on the one chunk
|
||||||
|
/// that turned out to differ.
|
||||||
|
fn first_difference(a: &[u8], b: &[u8]) -> Option<usize> {
|
||||||
|
if a == b {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some(
|
||||||
|
a.iter()
|
||||||
|
.zip(b.iter())
|
||||||
|
.position(|(x, y)| x != y)
|
||||||
|
.unwrap_or(a.len().min(b.len())),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn describe(path: &Path, e: &std::io::Error) -> String {
|
||||||
|
format!("{}: {}", path.display(), e)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
#[path = "verify_tests.rs"]
|
||||||
|
mod tests;
|
||||||
281
crates/quicksearch-core/src/verify_tests.rs
Normal file
281
crates/quicksearch-core/src/verify_tests.rs
Normal file
|
|
@ -0,0 +1,281 @@
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
use crate::testutil::{scratch_dir, touch};
|
||||||
|
|
||||||
|
/// Run to completion with cancellation switched off, returning every update.
|
||||||
|
fn run(paths: &[PathBuf]) -> Vec<VerifyUpdate> {
|
||||||
|
let cancel = AtomicBool::new(false);
|
||||||
|
let mut seen = Vec::new();
|
||||||
|
verify_identical(paths, &cancel, &mut |u| seen.push(u));
|
||||||
|
seen
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The report from a run, asserting it produced exactly one terminal update
|
||||||
|
/// and that the update was `Done`.
|
||||||
|
fn report(paths: &[PathBuf]) -> VerifyReport {
|
||||||
|
let seen = run(paths);
|
||||||
|
let terminal: Vec<&VerifyUpdate> = seen
|
||||||
|
.iter()
|
||||||
|
.filter(|u| !matches!(u, VerifyUpdate::Progress { .. }))
|
||||||
|
.collect();
|
||||||
|
assert_eq!(terminal.len(), 1, "expected one terminal update: {seen:?}");
|
||||||
|
match terminal[0] {
|
||||||
|
VerifyUpdate::Done(r) => r.clone(),
|
||||||
|
other => panic!("expected Done, got {other:?}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `n` files in a fresh directory, each with the body it is given.
|
||||||
|
fn files(tag: &str, bodies: &[&[u8]]) -> Vec<PathBuf> {
|
||||||
|
let dir = scratch_dir(tag);
|
||||||
|
bodies
|
||||||
|
.iter()
|
||||||
|
.enumerate()
|
||||||
|
.map(|(i, body)| {
|
||||||
|
let p = dir.join(format!("copy{i}.bin"));
|
||||||
|
touch(&p, body);
|
||||||
|
p
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn identical_files_all_report_identical() {
|
||||||
|
for count in [2, 3] {
|
||||||
|
let bodies = vec![&b"the same bytes in every copy"[..]; count];
|
||||||
|
let paths = files("verify-same", &bodies);
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.reference, Some(0));
|
||||||
|
assert!(r.all_identical(), "{r:?}");
|
||||||
|
assert_eq!(r.differing(), 0);
|
||||||
|
// Every file was read through: the head hash alone would not do.
|
||||||
|
assert_eq!(r.bytes_read, 28 * count as u64);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_difference_is_reported_at_its_offset() {
|
||||||
|
// First byte, mid-file, and the very last byte: the last is the one a
|
||||||
|
// head-only hash can never see, and the reason this module exists.
|
||||||
|
for (label, a, b, at) in [
|
||||||
|
("first", &b"Xbcdefgh"[..], &b"abcdefgh"[..], 0),
|
||||||
|
("middle", &b"abcdefgh"[..], &b"abcXefgh"[..], 3),
|
||||||
|
("last", &b"abcdefgh"[..], &b"abcdefgX"[..], 7),
|
||||||
|
] {
|
||||||
|
let paths = files("verify-diff", &[a, b]);
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.verdicts[0], MemberVerdict::Identical, "{label}");
|
||||||
|
assert_eq!(r.verdicts[1], MemberVerdict::DiffersAt(at), "{label}");
|
||||||
|
assert!(!r.all_identical(), "{label}");
|
||||||
|
assert_eq!(r.differing(), 1, "{label}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The realistic false positive: same size, same head, different tail — a
|
||||||
|
/// pre-allocated disk image, which is what `[processing] hash_length`
|
||||||
|
/// documents as the known limitation.
|
||||||
|
#[test]
|
||||||
|
fn a_shared_head_with_a_different_tail_is_caught() {
|
||||||
|
let head = vec![0u8; 64 * 1024];
|
||||||
|
let mut a = head.clone();
|
||||||
|
let mut b = head;
|
||||||
|
a.extend_from_slice(b"footer-a");
|
||||||
|
b.extend_from_slice(b"footer-b");
|
||||||
|
let paths = files("verify-tail", &[a.as_slice(), b.as_slice()]);
|
||||||
|
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.verdicts[1], MemberVerdict::DiffersAt(64 * 1024 + 7));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Bigger than one chunk, so the multi-chunk path and the running offset are
|
||||||
|
/// both exercised rather than assumed.
|
||||||
|
#[test]
|
||||||
|
fn a_difference_past_the_first_chunk_is_found() {
|
||||||
|
// Two files, so the chunk is the 256 KiB ceiling and the difference sits
|
||||||
|
// in the second one.
|
||||||
|
let size = MAX_CHUNK * 2 + 1234;
|
||||||
|
let a = vec![7u8; size];
|
||||||
|
let mut b = a.clone();
|
||||||
|
let at = MAX_CHUNK + 500;
|
||||||
|
b[at] = 8;
|
||||||
|
let paths = files("verify-chunks", &[a.as_slice(), b.as_slice()]);
|
||||||
|
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.verdicts[1], MemberVerdict::DiffersAt(at as u64));
|
||||||
|
// Two chunks out of each file and then it stops: with nothing left to
|
||||||
|
// compare against, reading the remainder of the reference would be work
|
||||||
|
// that cannot change the answer.
|
||||||
|
assert_eq!(
|
||||||
|
r.bytes_read,
|
||||||
|
4 * MAX_CHUNK as u64,
|
||||||
|
"did not stop once the last live file dropped out"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn different_lengths_are_decided_without_reading() {
|
||||||
|
let paths = files("verify-len", &[b"abcdefgh", b"abcdefghij"]);
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(
|
||||||
|
r.verdicts[1],
|
||||||
|
MemberVerdict::LengthDiffers {
|
||||||
|
len: 10,
|
||||||
|
reference_len: 8
|
||||||
|
}
|
||||||
|
);
|
||||||
|
assert_eq!(r.bytes_read, 0, "a length mismatch reads nothing");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn empty_files_are_identical() {
|
||||||
|
let paths = files("verify-empty", &[b"", b""]);
|
||||||
|
let r = report(&paths);
|
||||||
|
assert!(r.all_identical(), "{r:?}");
|
||||||
|
assert_eq!(r.bytes_read, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_missing_member_is_unreadable_and_the_rest_still_compare() {
|
||||||
|
let mut paths = files("verify-missing", &[b"same", b"same"]);
|
||||||
|
paths.insert(1, PathBuf::from("/nonexistent/quicksearch-verify-missing"));
|
||||||
|
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.reference, Some(0));
|
||||||
|
assert!(matches!(r.verdicts[1], MemberVerdict::Unreadable(_)));
|
||||||
|
assert_eq!(
|
||||||
|
r.verdicts[2],
|
||||||
|
MemberVerdict::Identical,
|
||||||
|
"an unreadable member stopped the others being compared"
|
||||||
|
);
|
||||||
|
assert_eq!(r.differing(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The first path is the obvious reference, but not a required one: an
|
||||||
|
/// unreadable first member must not sink the whole run.
|
||||||
|
#[test]
|
||||||
|
fn the_reference_falls_through_to_the_first_readable_member() {
|
||||||
|
let mut paths = files("verify-refmissing", &[b"same", b"same"]);
|
||||||
|
paths.insert(0, PathBuf::from("/nonexistent/quicksearch-verify-ref"));
|
||||||
|
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.reference, Some(1));
|
||||||
|
assert!(matches!(r.verdicts[0], MemberVerdict::Unreadable(_)));
|
||||||
|
assert_eq!(r.verdicts[1], MemberVerdict::Identical);
|
||||||
|
assert_eq!(r.verdicts[2], MemberVerdict::Identical);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn nothing_readable_reports_no_reference() {
|
||||||
|
let paths = vec![
|
||||||
|
PathBuf::from("/nonexistent/quicksearch-verify-a"),
|
||||||
|
PathBuf::from("/nonexistent/quicksearch-verify-b"),
|
||||||
|
];
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.reference, None);
|
||||||
|
assert_eq!(r.verdicts.len(), 2);
|
||||||
|
assert!(r.verdicts.iter().all(|v| !v.is_identical()));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_single_file_and_an_empty_set_are_vacuously_identical() {
|
||||||
|
let paths = files("verify-one", &[b"alone"]);
|
||||||
|
let r = report(&paths);
|
||||||
|
assert_eq!(r.reference, Some(0));
|
||||||
|
assert!(r.all_identical());
|
||||||
|
assert_eq!(r.bytes_read, 0, "nothing to compare it against");
|
||||||
|
|
||||||
|
let r = report(&[]);
|
||||||
|
assert_eq!(r.reference, None);
|
||||||
|
assert!(r.verdicts.is_empty());
|
||||||
|
assert!(r.all_identical());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_run_cancelled_before_it_starts_reports_only_that() {
|
||||||
|
let paths = files("verify-cancel", &[b"same", b"same"]);
|
||||||
|
let cancel = AtomicBool::new(true);
|
||||||
|
let mut seen = Vec::new();
|
||||||
|
verify_identical(&paths, &cancel, &mut |u| seen.push(u));
|
||||||
|
assert_eq!(seen, vec![VerifyUpdate::Cancelled]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Cancelling part way through ends the run there, with no `Done` claiming a
|
||||||
|
/// verdict it never reached. The first chunk always reports progress, so the
|
||||||
|
/// flag goes up between two chunks rather than at a time the test has to race
|
||||||
|
/// for.
|
||||||
|
#[test]
|
||||||
|
fn cancelling_mid_run_ends_it_without_a_verdict() {
|
||||||
|
let body = vec![3u8; MAX_CHUNK * 4];
|
||||||
|
let paths = files("verify-cancel-mid", &[body.as_slice(), body.as_slice()]);
|
||||||
|
|
||||||
|
let cancel = AtomicBool::new(false);
|
||||||
|
let mut seen = Vec::new();
|
||||||
|
verify_identical(&paths, &cancel, &mut |u| {
|
||||||
|
cancel.store(true, Ordering::Relaxed);
|
||||||
|
seen.push(u);
|
||||||
|
});
|
||||||
|
assert!(
|
||||||
|
matches!(seen.first(), Some(VerifyUpdate::Progress { .. })),
|
||||||
|
"the first chunk did not report progress: {seen:?}"
|
||||||
|
);
|
||||||
|
assert_eq!(seen.last(), Some(&VerifyUpdate::Cancelled), "{seen:?}");
|
||||||
|
assert!(
|
||||||
|
!seen.iter().any(|u| matches!(u, VerifyUpdate::Done(_))),
|
||||||
|
"a cancelled run still reported a verdict: {seen:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whatever progress reports, it is a fraction that makes sense: monotonic,
|
||||||
|
/// and never past its own denominator.
|
||||||
|
#[test]
|
||||||
|
fn progress_climbs_and_never_overruns_its_denominator() {
|
||||||
|
let body = vec![9u8; MAX_CHUNK * 6];
|
||||||
|
let paths = files(
|
||||||
|
"verify-progress",
|
||||||
|
&[body.as_slice(), body.as_slice(), body.as_slice()],
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut last = 0;
|
||||||
|
let mut count = 0;
|
||||||
|
for update in run(&paths) {
|
||||||
|
if let VerifyUpdate::Progress {
|
||||||
|
bytes_read,
|
||||||
|
bytes_total,
|
||||||
|
} = update
|
||||||
|
{
|
||||||
|
assert!(
|
||||||
|
bytes_read <= bytes_total,
|
||||||
|
"{bytes_read} read of {bytes_total}"
|
||||||
|
);
|
||||||
|
assert!(bytes_read >= last, "progress went backwards");
|
||||||
|
last = bytes_read;
|
||||||
|
count += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert!(count > 0, "a six-chunk comparison reported no progress");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A directory is not a file this can compare, however the platform refuses
|
||||||
|
/// it — `File::open` fails outright on Windows, while on Linux it opens and
|
||||||
|
/// then refuses to be read. Either way it is that member's problem, not the
|
||||||
|
/// run's.
|
||||||
|
#[test]
|
||||||
|
fn an_unreadable_member_does_not_stop_the_run() {
|
||||||
|
let dir = scratch_dir("verify-dir");
|
||||||
|
let a = dir.join("a.bin");
|
||||||
|
let b = dir.join("b.bin");
|
||||||
|
touch(&a, b"identical bytes");
|
||||||
|
touch(&b, b"identical bytes");
|
||||||
|
let sub = dir.join("subdir");
|
||||||
|
std::fs::create_dir_all(&sub).unwrap();
|
||||||
|
|
||||||
|
let r = report(&[a, sub, b]);
|
||||||
|
assert_eq!(r.reference, Some(0));
|
||||||
|
assert!(
|
||||||
|
!r.verdicts[1].is_identical(),
|
||||||
|
"a directory was called an identical file"
|
||||||
|
);
|
||||||
|
assert_eq!(r.verdicts[2], MemberVerdict::Identical);
|
||||||
|
}
|
||||||
|
|
@ -29,7 +29,6 @@ use crate::file_handling::{
|
||||||
classify_by_mtime, classify_for_indexing, path_to_db_string, prepare_file_record,
|
classify_by_mtime, classify_for_indexing, path_to_db_string, prepare_file_record,
|
||||||
warn_if_unrepresentable, DirRows, FileIndexAction, OwnedNewFile, UnreadableDirs,
|
warn_if_unrepresentable, DirRows, FileIndexAction, OwnedNewFile, UnreadableDirs,
|
||||||
};
|
};
|
||||||
use crate::indexing::should_abort;
|
|
||||||
|
|
||||||
mod pool;
|
mod pool;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|
@ -170,7 +169,6 @@ struct Ctx {
|
||||||
registry: Arc<Registry>,
|
registry: Arc<Registry>,
|
||||||
unreadable: UnreadableDirs,
|
unreadable: UnreadableDirs,
|
||||||
stop_flag: Arc<AtomicBool>,
|
stop_flag: Arc<AtomicBool>,
|
||||||
suspend_flag: Arc<AtomicBool>,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Individual unreadable-directory warnings allowed per run before only the
|
/// Individual unreadable-directory warnings allowed per run before only the
|
||||||
|
|
@ -422,7 +420,7 @@ fn prepare(file: PendingFile, known: Known<'_>, ctx: &Ctx) -> WalkedFile {
|
||||||
fn worker(shared: &Shared, ctx: &Ctx, tx: &mpsc::SyncSender<WalkEvent>) {
|
fn worker(shared: &Shared, ctx: &Ctx, tx: &mpsc::SyncSender<WalkEvent>) {
|
||||||
while let Some((job, slot)) = shared.take() {
|
while let Some((job, slot)) = shared.take() {
|
||||||
let _busy = shared.stats.enter();
|
let _busy = shared.stats.enter();
|
||||||
if should_abort(&ctx.stop_flag, &ctx.suspend_flag) {
|
if ctx.stop_flag.load(Ordering::Relaxed) {
|
||||||
shared.shutdown();
|
shared.shutdown();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -459,7 +457,7 @@ fn worker(shared: &Shared, ctx: &Ctx, tx: &mpsc::SyncSender<WalkEvent>) {
|
||||||
}
|
}
|
||||||
|
|
||||||
for file in files {
|
for file in files {
|
||||||
if should_abort(&ctx.stop_flag, &ctx.suspend_flag) {
|
if ctx.stop_flag.load(Ordering::Relaxed) {
|
||||||
shared.shutdown();
|
shared.shutdown();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -703,7 +701,6 @@ pub fn walk_indexable_files(
|
||||||
config: Config,
|
config: Config,
|
||||||
registry: Arc<Registry>,
|
registry: Arc<Registry>,
|
||||||
stop_flag: Arc<AtomicBool>,
|
stop_flag: Arc<AtomicBool>,
|
||||||
suspend_flag: Arc<AtomicBool>,
|
|
||||||
workers: usize,
|
workers: usize,
|
||||||
) -> ParallelWalk {
|
) -> ParallelWalk {
|
||||||
let mut queue = Queue::default();
|
let mut queue = Queue::default();
|
||||||
|
|
@ -746,7 +743,6 @@ pub fn walk_indexable_files(
|
||||||
registry,
|
registry,
|
||||||
unreadable: UnreadableDirs::default(),
|
unreadable: UnreadableDirs::default(),
|
||||||
stop_flag,
|
stop_flag,
|
||||||
suspend_flag,
|
|
||||||
});
|
});
|
||||||
|
|
||||||
for root in unresolvable {
|
for root in unresolvable {
|
||||||
|
|
|
||||||
|
|
@ -54,7 +54,6 @@ fn walk_with(
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
@ -304,7 +303,6 @@ fn unreadable_directory_is_recorded_not_silently_empty() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
);
|
);
|
||||||
let files: Vec<WalkedFile> = w
|
let files: Vec<WalkedFile> = w
|
||||||
|
|
@ -462,7 +460,6 @@ fn hidden_and_ignored_entries_are_pruned() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
));
|
));
|
||||||
assert_eq!(names(&files), vec!["keep.txt", "keep2.txt"]);
|
assert_eq!(names(&files), vec!["keep.txt", "keep2.txt"]);
|
||||||
|
|
@ -508,7 +505,6 @@ fn pruned_entries_are_counted_by_reason() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
);
|
);
|
||||||
let files: Vec<WalkedFile> = (&mut walk)
|
let files: Vec<WalkedFile> = (&mut walk)
|
||||||
|
|
@ -557,7 +553,6 @@ fn a_tree_with_nothing_pruned_reports_no_summary() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
);
|
);
|
||||||
let files: Vec<WalkedFile> = (&mut walk)
|
let files: Vec<WalkedFile> = (&mut walk)
|
||||||
|
|
@ -603,7 +598,6 @@ fn a_directory_reports_rows_with_no_file_behind_them() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
));
|
));
|
||||||
stale.sort();
|
stale.sort();
|
||||||
|
|
@ -643,7 +637,6 @@ fn an_unreadable_directory_reports_nothing_stale() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
));
|
));
|
||||||
fs::set_permissions(&locked, fs::Permissions::from_mode(0o755)).ok();
|
fs::set_permissions(&locked, fs::Permissions::from_mode(0o755)).ok();
|
||||||
|
|
@ -684,7 +677,6 @@ fn stop_flag_ends_the_walk_without_hanging() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
stop,
|
stop,
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
));
|
));
|
||||||
|
|
||||||
|
|
@ -713,7 +705,6 @@ fn dropping_the_walk_early_does_not_hang() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
);
|
);
|
||||||
assert!(w.next().is_some());
|
assert!(w.next().is_some());
|
||||||
|
|
@ -739,7 +730,6 @@ fn overlapping_roots_yield_each_file_once() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
));
|
));
|
||||||
|
|
||||||
|
|
@ -787,7 +777,6 @@ fn finish_reports_a_clean_walk_and_is_idempotent() {
|
||||||
Config::default(),
|
Config::default(),
|
||||||
Arc::new(Registry::default_set()),
|
Arc::new(Registry::default_set()),
|
||||||
Arc::new(AtomicBool::new(false)),
|
Arc::new(AtomicBool::new(false)),
|
||||||
Arc::new(AtomicBool::new(false)),
|
|
||||||
4,
|
4,
|
||||||
);
|
);
|
||||||
let files: Vec<WalkedFile> = w
|
let files: Vec<WalkedFile> = w
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,9 @@ use quicksearch_core::db::open_or_recreate;
|
||||||
use quicksearch_core::db::repo::{insert_file, set_content_done, NewFile};
|
use quicksearch_core::db::repo::{insert_file, set_content_done, NewFile};
|
||||||
use quicksearch_core::mime::FileType;
|
use quicksearch_core::mime::FileType;
|
||||||
use quicksearch_core::query::split::split_for_cascade;
|
use quicksearch_core::query::split::split_for_cascade;
|
||||||
use quicksearch_core::search::{cascade, SearchHit, SearchOptions, SearchService, SearchUpdate};
|
use quicksearch_core::search::{
|
||||||
|
cascade, MatchField, SearchHit, SearchOptions, SearchService, SearchUpdate,
|
||||||
|
};
|
||||||
use quicksearch_core::testutil::zstd_of;
|
use quicksearch_core::testutil::zstd_of;
|
||||||
|
|
||||||
mod common;
|
mod common;
|
||||||
|
|
@ -355,6 +357,39 @@ fn fuzzy_max_edits_widens_and_narrows_the_budget() {
|
||||||
std::fs::remove_file(&p).ok();
|
std::fs::remove_file(&p).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Regression: the fuzzy filename tier stamps stage 7 rather than truncating
|
||||||
|
/// its own rank. `7.0 + 0.1 * distance` reaches 8.0 at ten edits — the fuzzy
|
||||||
|
/// *full-text* tier — and every frontend reads `match_field()`, so a filename
|
||||||
|
/// hit would have been rendered as a match on the file's contents.
|
||||||
|
#[test]
|
||||||
|
fn a_distant_fuzzy_filename_hit_stays_a_name_hit() {
|
||||||
|
let p = tmp_db("fuzzystage");
|
||||||
|
let mut s = Seeder::new(&p, true);
|
||||||
|
// Ten substitutions against a 30-character term, whose budget is ten.
|
||||||
|
let far = s.add("abcdefghijklmnopqrst##########", "/d", 1, None);
|
||||||
|
let conn = s.done();
|
||||||
|
|
||||||
|
let (hits, _) = run_collect(
|
||||||
|
&conn,
|
||||||
|
"abcdefghijklmnopqrstuvwxyz0123",
|
||||||
|
&fuzzy_options_with_edits(10),
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
hits.iter().map(|h| h.file_id).collect::<Vec<_>>(),
|
||||||
|
vec![far]
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
(hits[0].rank - 8.0).abs() < 1e-9,
|
||||||
|
"rank {} is not the 8.0 that used to truncate into the next stage",
|
||||||
|
hits[0].rank
|
||||||
|
);
|
||||||
|
assert_eq!(hits[0].stage, 7, "the name tier is stage 7 at any distance");
|
||||||
|
assert_eq!(hits[0].match_field(), MatchField::Name);
|
||||||
|
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn dedup_keeps_best_rank() {
|
fn dedup_keeps_best_rank() {
|
||||||
let p = tmp_db("dedup");
|
let p = tmp_db("dedup");
|
||||||
|
|
@ -1400,3 +1435,138 @@ fn streaming_does_not_change_the_result_set() {
|
||||||
assert_eq!(outcome.total, 3, "batch size {}", batch);
|
assert_eq!(outcome.total, 3, "batch size {}", batch);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `SearchHit::snippet` is documented as "the filename for name stages, the
|
||||||
|
/// full path for path stages", and a frontend relies on it to highlight the
|
||||||
|
/// match inside the Name or Path column it is already painting: the ranges
|
||||||
|
/// index that field, so they only line up if the window *is* that field.
|
||||||
|
///
|
||||||
|
/// The fuzzy tiers used to window it instead, which silently broke the
|
||||||
|
/// contract whenever the match landed past two thirds of the way through a
|
||||||
|
/// long name — the ranges then indexed a suffix, and a column that trusted
|
||||||
|
/// them would mark the wrong glyphs.
|
||||||
|
#[test]
|
||||||
|
fn fuzzy_name_and_path_snippets_carry_the_whole_field() {
|
||||||
|
let p = tmp_db("fuzzy-whole-field");
|
||||||
|
let mut s = Seeder::new(&p, true);
|
||||||
|
// The match sits in the last third of the name, which is what used to
|
||||||
|
// push the window's start off zero.
|
||||||
|
s.add(
|
||||||
|
"a_long_and_deliberately_padded_out_quarterly_repot.txt",
|
||||||
|
"/home/me/documents/archive",
|
||||||
|
1,
|
||||||
|
None,
|
||||||
|
);
|
||||||
|
let conn = s.done();
|
||||||
|
|
||||||
|
let (hits, _) = run_collect(&conn, "report", &fuzzy_options());
|
||||||
|
let hit = hits
|
||||||
|
.iter()
|
||||||
|
.find(|h| h.stage == 7)
|
||||||
|
.expect("a fuzzy filename hit");
|
||||||
|
|
||||||
|
let snip = hit.snippet.as_ref().expect("a name hit carries a snippet");
|
||||||
|
assert_eq!(snip.window, hit.name, "the window is not the whole name");
|
||||||
|
assert!(!snip.truncated_start, "the name was windowed");
|
||||||
|
assert!(!snip.truncated_end, "the name was windowed");
|
||||||
|
for &(a, b) in &snip.ranges {
|
||||||
|
assert!(b <= hit.name.len(), "range {a}..{b} runs past the name");
|
||||||
|
assert!(
|
||||||
|
hit.name.is_char_boundary(a) && hit.name.is_char_boundary(b),
|
||||||
|
"range {a}..{b} is not on char boundaries"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A fuzzy tier's marks have to cover the *matched* text and nothing else.
|
||||||
|
///
|
||||||
|
/// The bug this pins: bitap reports where a match ends, and the range took
|
||||||
|
/// its start to be `end - term.len()`, which is only right when the match
|
||||||
|
/// happens to be as long as the term. Searching `repot` marked `1Repo` inside
|
||||||
|
/// `1Reporter` — one byte too far left, over a character that matched nothing.
|
||||||
|
///
|
||||||
|
/// Asserted on the *text* rather than on offsets, so it reads as the symptom
|
||||||
|
/// and survives the fixture being reworded.
|
||||||
|
#[test]
|
||||||
|
fn a_fuzzy_mark_covers_the_matched_text_and_nothing_else() {
|
||||||
|
let p = tmp_db("fuzzy-mark-span");
|
||||||
|
let mut s = Seeder::new(&p, true);
|
||||||
|
// The leading digit is the point: it is what the old range reached back
|
||||||
|
// over. In the body too, for the full-text tier.
|
||||||
|
s.add("1Reporter.txt", "/home/me/docs", 1, None);
|
||||||
|
s.add(
|
||||||
|
"body.txt",
|
||||||
|
"/home/me/docs",
|
||||||
|
2,
|
||||||
|
Some("filed under 1Reporter last week"),
|
||||||
|
);
|
||||||
|
let conn = s.done();
|
||||||
|
|
||||||
|
let (hits, _) = run_collect(&conn, "repot", &fuzzy_options());
|
||||||
|
|
||||||
|
// Stage 7 — fuzzy filename, marked inside the whole name.
|
||||||
|
let name_hit = hits
|
||||||
|
.iter()
|
||||||
|
.find(|h| h.stage == 7)
|
||||||
|
.expect("a fuzzy filename hit");
|
||||||
|
let snip = name_hit.snippet.as_ref().expect("name tiers carry one");
|
||||||
|
let (a, b) = snip.ranges[0];
|
||||||
|
assert_eq!(
|
||||||
|
&snip.window[a..b],
|
||||||
|
"Repo",
|
||||||
|
"the mark is {:?}; it must cover the match and not the leading digit",
|
||||||
|
&snip.window[a..b]
|
||||||
|
);
|
||||||
|
|
||||||
|
// Stage 8 — fuzzy full text, marked inside the snippet window.
|
||||||
|
let body_hit = hits
|
||||||
|
.iter()
|
||||||
|
.find(|h| h.stage == 8)
|
||||||
|
.expect("a fuzzy full-text hit");
|
||||||
|
let snip = body_hit.snippet.as_ref().expect("content tiers carry one");
|
||||||
|
let (a, b) = snip.ranges[0];
|
||||||
|
assert_eq!(
|
||||||
|
&snip.window[a..b],
|
||||||
|
"Repo",
|
||||||
|
"the mark is {:?}; it must cover the match and not the leading digit",
|
||||||
|
&snip.window[a..b]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The other half of the same defect: the automaton accepts the moment a
|
||||||
|
/// leading part of the term has matched, paying for the term's tail with
|
||||||
|
/// deletions — so the mark stopped short of the text that actually matched.
|
||||||
|
///
|
||||||
|
/// `quarterly` against a body holding `quartrly` accepts after `quartrl`,
|
||||||
|
/// spending both edits on the missing `y` and the dropped `e`. One byte
|
||||||
|
/// further is a *better* alignment (one edit) covering the whole word, which
|
||||||
|
/// is what a reader expects to see lit up.
|
||||||
|
#[test]
|
||||||
|
fn a_fuzzy_mark_is_not_truncated_to_a_leading_part_of_the_term() {
|
||||||
|
let p = tmp_db("fuzzy-mark-full");
|
||||||
|
let mut s = Seeder::new(&p, true);
|
||||||
|
// The name must not match at all: a row the filename tier claims never
|
||||||
|
// reaches the full-text tier. And the body must hold a fuzzy *variant* —
|
||||||
|
// the term verbatim would be an exact content match, stage 5 or 6.
|
||||||
|
s.add(
|
||||||
|
"notes.txt",
|
||||||
|
"/home/me/docs",
|
||||||
|
1,
|
||||||
|
Some("the quartrly budget was revised"),
|
||||||
|
);
|
||||||
|
let conn = s.done();
|
||||||
|
|
||||||
|
let (hits, _) = run_collect(&conn, "quarterly", &fuzzy_options());
|
||||||
|
let hit = hits
|
||||||
|
.iter()
|
||||||
|
.find(|h| h.stage == 8)
|
||||||
|
.expect("a fuzzy full-text hit");
|
||||||
|
let snip = hit.snippet.as_ref().expect("content tiers carry one");
|
||||||
|
let (a, b) = snip.ranges[0];
|
||||||
|
assert_eq!(
|
||||||
|
&snip.window[a..b],
|
||||||
|
"quartrly",
|
||||||
|
"the mark is {:?}, a leading part of what matched",
|
||||||
|
&snip.window[a..b]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -11,12 +11,26 @@ use std::sync::{Arc, Mutex};
|
||||||
use std::time::{Duration, Instant, SystemTime};
|
use std::time::{Duration, Instant, SystemTime};
|
||||||
|
|
||||||
use quicksearch_core::config::Config;
|
use quicksearch_core::config::Config;
|
||||||
use quicksearch_core::file_handling::{extract_scope_prepare, ExtractCursor};
|
use quicksearch_core::file_handling::{
|
||||||
|
count_extract_scope, mark_oversize_pending_na, ExtractCursor, ExtractScope,
|
||||||
|
};
|
||||||
use quicksearch_core::indexing::{IndexingService, IndexingStatus, RootPhase};
|
use quicksearch_core::indexing::{IndexingService, IndexingStatus, RootPhase};
|
||||||
|
|
||||||
mod common;
|
mod common;
|
||||||
use common::{scratch_dir as tmp_dir, touch};
|
use common::{scratch_dir as tmp_dir, touch};
|
||||||
|
|
||||||
|
/// The removed `extract_scope_prepare`: the oversize sweep the writer still
|
||||||
|
/// does, then the count the content pass now does on its own connection.
|
||||||
|
fn extract_scope_prepare(
|
||||||
|
conn_mutex: &Arc<Mutex<rusqlite::Connection>>,
|
||||||
|
cursor: &ExtractCursor,
|
||||||
|
config: &Config,
|
||||||
|
) -> Result<ExtractScope, String> {
|
||||||
|
let conn = conn_mutex.lock().unwrap();
|
||||||
|
mark_oversize_pending_na(&conn, cursor, config).unwrap();
|
||||||
|
count_extract_scope(&conn, cursor, config)
|
||||||
|
}
|
||||||
|
|
||||||
/// Run one full index over `root` and wait for it to finish.
|
/// Run one full index over `root` and wait for it to finish.
|
||||||
fn index_once(root: &Path, db: &Path, config: &Config) {
|
fn index_once(root: &Path, db: &Path, config: &Config) {
|
||||||
common::IndexOnce {
|
common::IndexOnce {
|
||||||
|
|
@ -1230,6 +1244,80 @@ fn contentless_mode_still_indexes_inlined_files_without_storing_bodies() {
|
||||||
std::fs::remove_dir_all(&db_dir).ok();
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// What one watch of a heavy/light overlap saw; see [`observe_overlap`].
|
||||||
|
struct Overlap {
|
||||||
|
/// Light files drained and heavy rows stored, across the window in which
|
||||||
|
/// the heavy root extracted while the light root walked.
|
||||||
|
light_drained: usize,
|
||||||
|
heavy_stored: usize,
|
||||||
|
/// The heavy root's `extract_total` and pool size, for the fixture guards.
|
||||||
|
heavy_pending: usize,
|
||||||
|
heavy_pool: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Watch a two-root run until the heavy root has finished extracting and
|
||||||
|
/// report how the two counters moved while both were in flight, then stop
|
||||||
|
/// the run. Removing the fixture is the caller's.
|
||||||
|
///
|
||||||
|
/// Deltas across the overlap, never durations. Sparse samples cost only the
|
||||||
|
/// window's edges, and they trim both counters together. Panics if the window
|
||||||
|
/// never opened — a fixture that does not exercise the case proves nothing.
|
||||||
|
fn observe_overlap(service: &IndexingService, heavy_tag: &str, light_tag: &str) -> Overlap {
|
||||||
|
let mut opened: Option<(usize, usize)> = None; // (light.walked, heavy.extracted)
|
||||||
|
let mut last = (0usize, 0usize);
|
||||||
|
let mut heavy_pending = 0usize;
|
||||||
|
let mut heavy_pool = 0usize;
|
||||||
|
let deadline = Instant::now() + Duration::from_secs(120);
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
let mut in_window = false;
|
||||||
|
match service.get_status() {
|
||||||
|
IndexingStatus::Running { roots, .. } => {
|
||||||
|
let heavy_p = roots.iter().find(|r| r.root.contains(heavy_tag));
|
||||||
|
let light_p = roots.iter().find(|r| r.root.contains(light_tag));
|
||||||
|
if let (Some(h), Some(l)) = (heavy_p, light_p) {
|
||||||
|
// The light root's *walk* is what used to be starved, so the
|
||||||
|
// window closes with it — past that there is no drain left
|
||||||
|
// to observe, and `walked` is the only counter in play.
|
||||||
|
in_window = h.phase == RootPhase::Extracting && l.phase == RootPhase::Walking;
|
||||||
|
if in_window {
|
||||||
|
last = (l.walked, h.extracted);
|
||||||
|
opened.get_or_insert(last);
|
||||||
|
if let Some(total) = h.extract_total {
|
||||||
|
heavy_pending = total;
|
||||||
|
}
|
||||||
|
heavy_pool = h.total_workers;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The run is claimed but has not reached its walk yet; there is
|
||||||
|
// nothing to sample, and breaking here would end the watch before
|
||||||
|
// the run it is watching had started.
|
||||||
|
IndexingStatus::Preparing { .. } => {}
|
||||||
|
IndexingStatus::Error(e) => panic!("indexing failed: {}", e),
|
||||||
|
_ => break,
|
||||||
|
}
|
||||||
|
// Both phases are monotone, so a closed window will not reopen.
|
||||||
|
if opened.is_some() && !in_window {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(2));
|
||||||
|
}
|
||||||
|
service.stop_indexing().unwrap();
|
||||||
|
|
||||||
|
let Some((light_open, heavy_open)) = opened else {
|
||||||
|
panic!(
|
||||||
|
"never observed the heavy root extracting while the light root walked; \
|
||||||
|
the fixture is not exercising the case"
|
||||||
|
);
|
||||||
|
};
|
||||||
|
Overlap {
|
||||||
|
light_drained: last.0 - light_open,
|
||||||
|
heavy_stored: last.1 - heavy_open,
|
||||||
|
heavy_pending,
|
||||||
|
heavy_pool,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// A slow root must not stall the others.
|
/// A slow root must not stall the others.
|
||||||
///
|
///
|
||||||
/// This is the complaint stated directly: one root doing heavy extraction used
|
/// This is the complaint stated directly: one root doing heavy extraction used
|
||||||
|
|
@ -1241,120 +1329,252 @@ fn contentless_mode_still_indexes_inlined_files_without_storing_bodies() {
|
||||||
/// wrong measure: writing is serial by construction (one SQLite connection),
|
/// wrong measure: writing is serial by construction (one SQLite connection),
|
||||||
/// so on a local disk the writer, not extraction, is the bottleneck and a
|
/// so on a local disk the writer, not extraction, is the bottleneck and a
|
||||||
/// wall-clock comparison would mostly measure the machine.
|
/// wall-clock comparison would mostly measure the machine.
|
||||||
|
///
|
||||||
|
/// A stall is therefore counted in *work*, not in milliseconds: while the heavy
|
||||||
|
/// root extracts, how many files the light root's walk was drained of, against
|
||||||
|
/// how many rows the heavy root's extraction landed. Both counters are advanced
|
||||||
|
/// by the same writer loop, each root's turn bounded by one slice
|
||||||
|
/// (`service_walking`, `service_extracting` in `indexing/pipeline.rs`), so their
|
||||||
|
/// ratio *is* the interleaving.
|
||||||
|
///
|
||||||
|
/// - Serialised — the regression — the writer reads the heavy batch itself and
|
||||||
|
/// drains nobody meanwhile. Whatever shape that takes it obeys
|
||||||
|
/// `light < heavy + quantum`: one quantum of each per round is the most a
|
||||||
|
/// single thread taking turns can manage. Its own time budget says the same
|
||||||
|
/// from the other side, since time spent reading is time not spent inserting.
|
||||||
|
/// - As built, extraction is off on the root's own pool and the writer's turn
|
||||||
|
/// for the heavy root is a store and nothing more, so the light root is
|
||||||
|
/// drained at the writer's full rate throughout — on this fixture several
|
||||||
|
/// times the bound.
|
||||||
|
///
|
||||||
|
/// Counting rather than timing is what makes the verdict the same on a loaded
|
||||||
|
/// CI runner and an idle workstation. Every way a host can be slow — a
|
||||||
|
/// preempted writer, a checkpoint, a long round — freezes *both* counters, and
|
||||||
|
/// cancels. The wall-clock figure this replaced did not cancel: the same
|
||||||
|
/// correct behaviour measured ~20 ms here and 188 ms on the CI runner, which is
|
||||||
|
/// *more* than the 130 ms the broken design measured here. At that point CI was
|
||||||
|
/// overriding the budget six-fold and the check had stopped telling the two
|
||||||
|
/// designs apart. A bound that has to be calibrated per host is not an
|
||||||
|
/// assertion.
|
||||||
#[test]
|
#[test]
|
||||||
fn a_heavy_root_does_not_stall_a_light_one() {
|
fn a_heavy_root_does_not_stall_a_light_one() {
|
||||||
// HEAVY: few files, each big enough that reading it is real work, with a
|
// The writer's round-robin quantum. Set here rather than inherited from the
|
||||||
|
// default 500 because the bound below is arithmetic in it, and because a
|
||||||
|
// 500-file round is a coarse enough publish interval to look like a stall
|
||||||
|
// on a slow host all by itself.
|
||||||
|
const QUANTUM: usize = 16;
|
||||||
|
// HEAVY: few files, each big enough that reading one is real work, with a
|
||||||
// small `maximum_text_size` so the cost lands in extraction rather than in
|
// small `maximum_text_size` so the cost lands in extraction rather than in
|
||||||
// the writer's tokenising.
|
// the writer's tokenising. Few and large rather than many and small: the
|
||||||
|
// margin is set by the cost of *one* heavy file against one light one,
|
||||||
|
// while the fixture's size on disk is their product — so for a given number
|
||||||
|
// of bytes, bigger files discriminate better. If a host ever comes in
|
||||||
|
// short, raise this size; raising the count only lengthens the window.
|
||||||
|
const HEAVY_FILES: usize = 32;
|
||||||
|
// LIGHT: a wide tree of tiny files, so its walk outlasts the heavy root's
|
||||||
|
// extraction and its counter moves finely. Each is inlined by its walk
|
||||||
|
// worker, so this root has no extraction phase of its own to confuse the
|
||||||
|
// window with. Sized for a walk the writer never holds up: at 6000 it was
|
||||||
|
// over before half the heavy rows had landed.
|
||||||
|
const LIGHT_FILES: usize = 16_000;
|
||||||
|
// Light files drained per (heavy row + quantum). Three times a bound the
|
||||||
|
// serialised design provably cannot reach, and roughly a ninth of what the
|
||||||
|
// built one reaches here.
|
||||||
|
const MIN_INTERLEAVE: usize = 3;
|
||||||
|
|
||||||
let heavy = tmp_dir("stall-heavy");
|
let heavy = tmp_dir("stall-heavy");
|
||||||
let body: Vec<u8> = "sphinx of black quartz judge my vow "
|
let body: Vec<u8> = "sphinx of black quartz judge my vow "
|
||||||
.repeat(40_000)
|
.repeat(80_000)
|
||||||
.into_bytes();
|
.into_bytes();
|
||||||
for i in 0..200 {
|
for i in 0..HEAVY_FILES {
|
||||||
touch(&heavy.join(format!("d{}/big{:04}.txt", i % 8, i)), &body);
|
touch(&heavy.join(format!("d{}/big{:04}.txt", i % 8, i)), &body);
|
||||||
}
|
}
|
||||||
// LIGHT: a wide tree of tiny files, so its walk runs long enough to sample
|
|
||||||
// and its progress counter moves finely.
|
|
||||||
let light = tmp_dir("stall-light");
|
let light = tmp_dir("stall-light");
|
||||||
for i in 0..6000 {
|
for i in 0..LIGHT_FILES {
|
||||||
touch(&light.join(format!("d{}/f{:05}.txt", i % 60, i)), b"x");
|
touch(&light.join(format!("d{}/f{:05}.txt", i % 60, i)), b"x");
|
||||||
}
|
}
|
||||||
|
|
||||||
let db_dir = tmp_dir("stall-db");
|
let db_dir = tmp_dir("stall-db");
|
||||||
let db = db_dir.join("index.sqlite");
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let roots = vec![
|
||||||
|
heavy.to_string_lossy().into_owned(),
|
||||||
|
light.to_string_lossy().into_owned(),
|
||||||
|
];
|
||||||
|
|
||||||
let mut config = test_config();
|
let mut config = test_config();
|
||||||
config.processing.maximum_text_size = 1024;
|
config.processing.maximum_text_size = 1024;
|
||||||
config.processing.maximum_text_file_size = 8 * 1024 * 1024;
|
config.processing.maximum_text_file_size = 8 * 1024 * 1024;
|
||||||
|
config.processing.batch_size = QUANTUM;
|
||||||
|
// One extraction thread for the heavy root, so its pass costs about what
|
||||||
|
// the broken design's inline read would and the two differ only in *which*
|
||||||
|
// thread pays for it. `root_workers` is keyed by the `indexing_paths`
|
||||||
|
// spelling; both sides canonicalize before matching.
|
||||||
|
config.paths.indexing_paths = roots.clone();
|
||||||
|
config.indexing.root_workers.insert(roots[0].clone(), 1);
|
||||||
|
// The default WAL cap is far above anything this run writes, so no forced
|
||||||
|
// checkpoint lands inside the window. That stops being true if the fixture
|
||||||
|
// ever grows by an order of magnitude.
|
||||||
|
|
||||||
let service = IndexingService::new();
|
let service = IndexingService::new();
|
||||||
service
|
service
|
||||||
.start_indexing(
|
.start_indexing(roots, db.to_string_lossy().into_owned(), config.clone())
|
||||||
vec![
|
|
||||||
heavy.to_string_lossy().into_owned(),
|
|
||||||
light.to_string_lossy().into_owned(),
|
|
||||||
],
|
|
||||||
db.to_string_lossy().into_owned(),
|
|
||||||
config.clone(),
|
|
||||||
)
|
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
let seen = observe_overlap(&service, "stall-heavy", "stall-light");
|
||||||
// Sample the light root's progress while the heavy one is extracting, and
|
|
||||||
// keep the longest interval over which it did not move.
|
|
||||||
let mut worst = Duration::ZERO;
|
|
||||||
let mut last_change = Instant::now();
|
|
||||||
let mut last_seen = 0usize;
|
|
||||||
let mut sampled_together = false;
|
|
||||||
let deadline = Instant::now() + Duration::from_secs(120);
|
|
||||||
while Instant::now() < deadline {
|
|
||||||
match service.get_status() {
|
|
||||||
IndexingStatus::Running { roots, .. } => {
|
|
||||||
let heavy_p = roots.iter().find(|r| r.root.contains("stall-heavy"));
|
|
||||||
let light_p = roots.iter().find(|r| r.root.contains("stall-light"));
|
|
||||||
if let (Some(h), Some(l)) = (heavy_p, light_p) {
|
|
||||||
let light_busy = l.phase != RootPhase::Done;
|
|
||||||
if h.phase == RootPhase::Extracting && light_busy {
|
|
||||||
sampled_together = true;
|
|
||||||
let now = l.walked + l.extracted;
|
|
||||||
if now != last_seen {
|
|
||||||
last_seen = now;
|
|
||||||
last_change = Instant::now();
|
|
||||||
} else {
|
|
||||||
worst = worst.max(last_change.elapsed());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// The run is claimed but has not reached its walk yet; there is
|
|
||||||
// nothing to sample, and breaking here would end the watch before
|
|
||||||
// the run it is watching had started.
|
|
||||||
IndexingStatus::Preparing { .. } => {}
|
|
||||||
IndexingStatus::Error(e) => panic!("indexing failed: {}", e),
|
|
||||||
_ => break,
|
|
||||||
}
|
|
||||||
std::thread::sleep(Duration::from_millis(2));
|
|
||||||
}
|
|
||||||
service.stop_indexing().unwrap();
|
|
||||||
drop(service);
|
drop(service);
|
||||||
|
|
||||||
|
// Before the assertions, unlike the rest of this file: those tests keep
|
||||||
|
// their trees because a failing test's tree is the evidence, but this
|
||||||
|
// fixture is generated and identical every run, and its evidence is the two
|
||||||
|
// counters printed below. Leaving 92 MB of it in a RAM-backed /tmp behind a
|
||||||
|
// failure is itself a reason for the next run to fail.
|
||||||
|
std::fs::remove_dir_all(&heavy).ok();
|
||||||
|
std::fs::remove_dir_all(&light).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
|
||||||
|
// The fixture is as configured. Each of these silently costs a factor of
|
||||||
|
// the margin below if it stops holding, so they are checked before the
|
||||||
|
// ratio is read as a verdict on the design.
|
||||||
|
assert_eq!(
|
||||||
|
seen.heavy_pool, 1,
|
||||||
|
"the heavy root must extract on the single worker root_workers asked for; \
|
||||||
|
with the default four its pass is four times shorter and so is the margin"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
seen.heavy_pending, HEAVY_FILES,
|
||||||
|
"every heavy file must reach the content pass; one inlined by its walk \
|
||||||
|
worker never produces an extraction phase to overlap with"
|
||||||
|
);
|
||||||
assert!(
|
assert!(
|
||||||
sampled_together,
|
seen.heavy_stored * 2 >= HEAVY_FILES,
|
||||||
"never observed the two roots overlapping; the fixture is not exercising the case"
|
"only {} of {} heavy rows landed inside the observed window; the sample \
|
||||||
);
|
did not cover the pass",
|
||||||
// Measured on this fixture: ~20 ms with the extraction pools, ~130 ms when
|
seen.heavy_stored,
|
||||||
// the file reading is forced back onto the writer thread (and unbounded in
|
HEAVY_FILES
|
||||||
// the real failure, where the heavy root is on a network share). The bound
|
|
||||||
// sits between, with several times the observed headroom.
|
|
||||||
//
|
|
||||||
// The fixed design's stall does not grow with the heavy root's cost — it is
|
|
||||||
// one round-robin pass plus one commit — so making that root heavier only
|
|
||||||
// widens the margin.
|
|
||||||
// The figures above are wall-clock, so they scale with the host: a small CI
|
|
||||||
// VM running the other tests in this binary alongside this one measures
|
|
||||||
// several times the developer-machine number without the design having
|
|
||||||
// changed at all. QSB_STALL_BUDGET_MS lets that environment say so out loud
|
|
||||||
// instead of the bound being quietly loosened for everyone. Keep any override
|
|
||||||
// well under the broken design's figure scaled by the same factor, or the
|
|
||||||
// test stops discriminating between the two.
|
|
||||||
let budget = Duration::from_millis(
|
|
||||||
std::env::var("QSB_STALL_BUDGET_MS")
|
|
||||||
.ok()
|
|
||||||
.and_then(|v| v.parse().ok())
|
|
||||||
.unwrap_or(100),
|
|
||||||
);
|
);
|
||||||
|
|
||||||
eprintln!(
|
eprintln!(
|
||||||
"longest light-root stall while heavy extracted: {:?} (budget {:?})",
|
"light files drained while the heavy root extracted: {} against {} heavy \
|
||||||
worst, budget
|
rows (quantum {}) — {}x the {}x required; the serialised design cannot \
|
||||||
|
exceed 1x",
|
||||||
|
seen.light_drained,
|
||||||
|
seen.heavy_stored,
|
||||||
|
QUANTUM,
|
||||||
|
seen.light_drained / (seen.heavy_stored + QUANTUM),
|
||||||
|
MIN_INTERLEAVE
|
||||||
);
|
);
|
||||||
assert!(
|
assert!(
|
||||||
worst < budget,
|
seen.light_drained >= MIN_INTERLEAVE * (seen.heavy_stored + QUANTUM),
|
||||||
"the light root stalled for {:?} while the heavy root extracted (budget {:?})",
|
"the light root was drained of only {} files while the heavy root landed \
|
||||||
worst,
|
{} rows; one quantum of each per round is all a writer that extracts \
|
||||||
budget
|
inline can manage, so anything near {} means the extraction is back on \
|
||||||
|
the writer thread",
|
||||||
|
seen.light_drained,
|
||||||
|
seen.heavy_stored,
|
||||||
|
seen.heavy_stored + QUANTUM
|
||||||
);
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The sibling of [`a_heavy_root_does_not_stall_a_light_one`] for the cost that
|
||||||
|
/// test deliberately keeps small: the writer's own tokenising.
|
||||||
|
///
|
||||||
|
/// There the heavy files are expensive to *read* and cheap to *write*
|
||||||
|
/// (`maximum_text_size = 1024`), so it never exercised the writer. Here each
|
||||||
|
/// heavy row carries the default 256 KiB of text and its FTS5 trigram insert is
|
||||||
|
/// the expensive step — and it runs on the writer thread, inside the
|
||||||
|
/// transaction, where nothing can take it off. Four workers keep the ready
|
||||||
|
/// channel full, so what one turn finds waiting is a whole channel of them.
|
||||||
|
///
|
||||||
|
/// Before turns had a slice, an extraction turn wrote everything it found —
|
||||||
|
/// half a second to two seconds of tokenising — and the light root's walk got
|
||||||
|
/// one quantum in between: the ratio below came in under one. With turns
|
||||||
|
/// bounded by `TURN_SLICE` and walks served first, the light root drains at
|
||||||
|
/// its own rate while the heavy root lands a row or two per round.
|
||||||
|
#[test]
|
||||||
|
fn a_heavy_root_does_not_stall_a_light_one_at_the_writer() {
|
||||||
|
const QUANTUM: usize = 16;
|
||||||
|
// Over the walk's inline threshold, and enough that the stored text is the
|
||||||
|
// full `maximum_text_size` (256 KiB) — the tokenising is what is measured.
|
||||||
|
const HEAVY_FILES: usize = 32;
|
||||||
|
// Wider than the sibling's: with the walk no longer waiting on the writer
|
||||||
|
// it drains so fast that 6000 files were gone before half the heavy rows
|
||||||
|
// had landed, and the window closed on a sample too short to trust.
|
||||||
|
const LIGHT_FILES: usize = 16_000;
|
||||||
|
// As in the sibling: three times a bound the unsliced writer cannot reach.
|
||||||
|
const MIN_INTERLEAVE: usize = 3;
|
||||||
|
|
||||||
|
let heavy = tmp_dir("stall-writer-heavy");
|
||||||
|
let body: Vec<u8> = "sphinx of black quartz judge my vow "
|
||||||
|
.repeat(9_000)
|
||||||
|
.into_bytes();
|
||||||
|
for i in 0..HEAVY_FILES {
|
||||||
|
touch(&heavy.join(format!("d{}/big{:04}.txt", i % 8, i)), &body);
|
||||||
|
}
|
||||||
|
let light = tmp_dir("stall-writer-light");
|
||||||
|
for i in 0..LIGHT_FILES {
|
||||||
|
touch(&light.join(format!("d{}/f{:05}.txt", i % 60, i)), b"x");
|
||||||
|
}
|
||||||
|
|
||||||
|
let db_dir = tmp_dir("stall-writer-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let roots = vec![
|
||||||
|
heavy.to_string_lossy().into_owned(),
|
||||||
|
light.to_string_lossy().into_owned(),
|
||||||
|
];
|
||||||
|
|
||||||
|
let mut config = test_config();
|
||||||
|
config.processing.batch_size = QUANTUM;
|
||||||
|
config.paths.indexing_paths = roots.clone();
|
||||||
|
// Four readers, so the heavy rows reach the writer faster than it can
|
||||||
|
// tokenise them and the ready channel is full when its turn comes.
|
||||||
|
config.indexing.root_workers.insert(roots[0].clone(), 4);
|
||||||
|
|
||||||
|
let service = IndexingService::new();
|
||||||
|
service
|
||||||
|
.start_indexing(roots, db.to_string_lossy().into_owned(), config.clone())
|
||||||
|
.unwrap();
|
||||||
|
let seen = observe_overlap(&service, "stall-writer-heavy", "stall-writer-light");
|
||||||
|
drop(service);
|
||||||
|
|
||||||
std::fs::remove_dir_all(&heavy).ok();
|
std::fs::remove_dir_all(&heavy).ok();
|
||||||
std::fs::remove_dir_all(&light).ok();
|
std::fs::remove_dir_all(&light).ok();
|
||||||
std::fs::remove_dir_all(&db_dir).ok();
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
seen.heavy_pool, 4,
|
||||||
|
"the heavy root must extract on four workers"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
seen.heavy_pending, HEAVY_FILES,
|
||||||
|
"every heavy file must reach the content pass"
|
||||||
|
);
|
||||||
|
// A quarter, not the sibling's half: the light walk now outruns the heavy
|
||||||
|
// pass by design, and a window over eight 256 KiB rows is evidence enough
|
||||||
|
// that the writer yielded between them.
|
||||||
|
assert!(
|
||||||
|
seen.heavy_stored * 4 >= HEAVY_FILES,
|
||||||
|
"only {} of {} heavy rows landed inside the observed window; the sample \
|
||||||
|
did not cover the pass",
|
||||||
|
seen.heavy_stored,
|
||||||
|
HEAVY_FILES
|
||||||
|
);
|
||||||
|
|
||||||
|
eprintln!(
|
||||||
|
"light files drained while the heavy root tokenised: {} against {} heavy \
|
||||||
|
rows (quantum {}) — {}x the {}x required",
|
||||||
|
seen.light_drained,
|
||||||
|
seen.heavy_stored,
|
||||||
|
QUANTUM,
|
||||||
|
seen.light_drained / (seen.heavy_stored + QUANTUM),
|
||||||
|
MIN_INTERLEAVE
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
seen.light_drained >= MIN_INTERLEAVE * (seen.heavy_stored + QUANTUM),
|
||||||
|
"the light root was drained of only {} files while the heavy root landed \
|
||||||
|
{} rows; an extraction turn is writing to the end of its batch again \
|
||||||
|
instead of yielding at its slice",
|
||||||
|
seen.light_drained,
|
||||||
|
seen.heavy_stored
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The write-ahead log must not grow for the length of a run.
|
/// The write-ahead log must not grow for the length of a run.
|
||||||
|
|
|
||||||
|
|
@ -289,7 +289,8 @@ fn snippet_paths_perf_comparison() {
|
||||||
}
|
}
|
||||||
None => String::new(),
|
None => String::new(),
|
||||||
};
|
};
|
||||||
let _snip = snippet::extract(&text, &[q], &opts);
|
let folded = text.to_ascii_lowercase();
|
||||||
|
let _snip = snippet::extract_folded(&text, &folded, &[q], &opts);
|
||||||
rows_b_total += 1;
|
rows_b_total += 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
//! Application shell: tab strip, per-frame event drains, debounce,
|
//! Application shell: tab strip, per-frame event drains, debounce,
|
||||||
//! status bar, and config-change routing.
|
//! status bar, and config-change routing.
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
use std::sync::mpsc;
|
use std::sync::mpsc;
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
|
|
@ -22,8 +23,8 @@ use crate::format::{fmt_interval, group_thousands};
|
||||||
use crate::keychain;
|
use crate::keychain;
|
||||||
use crate::logs_tab::LogsTab;
|
use crate::logs_tab::LogsTab;
|
||||||
use crate::manage_tab::ManageTab;
|
use crate::manage_tab::ManageTab;
|
||||||
use crate::options::{OptionsWindow, SecurityAction};
|
|
||||||
use crate::search_tab::SearchTab;
|
use crate::search_tab::SearchTab;
|
||||||
|
use crate::settings_tab::{SecurityAction, SettingsTab};
|
||||||
use crate::unlock::KeySource;
|
use crate::unlock::KeySource;
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
|
@ -33,6 +34,17 @@ pub(crate) enum Tab {
|
||||||
Duplicates,
|
Duplicates,
|
||||||
Logs,
|
Logs,
|
||||||
Help,
|
Help,
|
||||||
|
Settings,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The editor a tab holds, if it stages its edits on a draft rather than
|
||||||
|
/// saving them the moment they change.
|
||||||
|
fn tab_editor(tab: Tab) -> Option<UnsavedSource> {
|
||||||
|
match tab {
|
||||||
|
Tab::Manage => Some(UnsavedSource::Manage),
|
||||||
|
Tab::Settings => Some(UnsavedSource::Settings),
|
||||||
|
Tab::Search | Tab::Duplicates | Tab::Logs | Tab::Help => None,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A navigation the unsaved-changes guard put on hold; once nothing relevant
|
/// A navigation the unsaved-changes guard put on hold; once nothing relevant
|
||||||
|
|
@ -40,7 +52,6 @@ pub(crate) enum Tab {
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
enum NavIntent {
|
enum NavIntent {
|
||||||
SwitchTab(Tab),
|
SwitchTab(Tab),
|
||||||
CloseOptions,
|
|
||||||
Quit,
|
Quit,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -48,21 +59,30 @@ enum NavIntent {
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
enum UnsavedSource {
|
enum UnsavedSource {
|
||||||
Manage,
|
Manage,
|
||||||
Options,
|
Settings,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Which editor the guard must ask about for `intent`, if any. Quit asks
|
/// The editor `from` holds, if it has one and it is holding unapplied edits.
|
||||||
/// about Options before Manage, one prompt at a time: each draft is a full
|
fn dirty_editor(from: Tab, manage_dirty: bool, settings_dirty: bool) -> Option<UnsavedSource> {
|
||||||
/// `Config` snapshot, so applying both at once would revert the first.
|
match tab_editor(from)? {
|
||||||
|
UnsavedSource::Manage => manage_dirty.then_some(UnsavedSource::Manage),
|
||||||
|
UnsavedSource::Settings => settings_dirty.then_some(UnsavedSource::Settings),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Which editor the guard must ask about for `intent` while sitting on
|
||||||
|
/// `from`, if any. A tab switch asks only about the tab being left; Quit asks
|
||||||
|
/// about Settings before Manage, one prompt at a time, because each draft is
|
||||||
|
/// a full `Config` snapshot and applying both at once would revert the first.
|
||||||
fn guard_source(
|
fn guard_source(
|
||||||
intent: NavIntent,
|
intent: NavIntent,
|
||||||
|
from: Tab,
|
||||||
manage_dirty: bool,
|
manage_dirty: bool,
|
||||||
options_dirty: bool,
|
settings_dirty: bool,
|
||||||
) -> Option<UnsavedSource> {
|
) -> Option<UnsavedSource> {
|
||||||
match intent {
|
match intent {
|
||||||
NavIntent::SwitchTab(_) => manage_dirty.then_some(UnsavedSource::Manage),
|
NavIntent::SwitchTab(_) => dirty_editor(from, manage_dirty, settings_dirty),
|
||||||
NavIntent::CloseOptions => options_dirty.then_some(UnsavedSource::Options),
|
NavIntent::Quit if settings_dirty => Some(UnsavedSource::Settings),
|
||||||
NavIntent::Quit if options_dirty => Some(UnsavedSource::Options),
|
|
||||||
NavIntent::Quit if manage_dirty => Some(UnsavedSource::Manage),
|
NavIntent::Quit if manage_dirty => Some(UnsavedSource::Manage),
|
||||||
NavIntent::Quit => None,
|
NavIntent::Quit => None,
|
||||||
}
|
}
|
||||||
|
|
@ -78,8 +98,13 @@ fn quit_needs_reconcile_warning(intent: NavIntent, reconciling: bool) -> bool {
|
||||||
/// Whether leaving the current tab has to go through the unsaved-changes
|
/// Whether leaving the current tab has to go through the unsaved-changes
|
||||||
/// guard. A navigation already on hold wins: a second intent would replace
|
/// guard. A navigation already on hold wins: a second intent would replace
|
||||||
/// the answer the guard is waiting for.
|
/// the answer the guard is waiting for.
|
||||||
fn switch_needs_guard(from: Tab, manage_dirty: bool, nav_pending: bool) -> bool {
|
fn switch_needs_guard(
|
||||||
from == Tab::Manage && manage_dirty && !nav_pending
|
from: Tab,
|
||||||
|
manage_dirty: bool,
|
||||||
|
settings_dirty: bool,
|
||||||
|
nav_pending: bool,
|
||||||
|
) -> bool {
|
||||||
|
!nav_pending && dirty_editor(from, manage_dirty, settings_dirty).is_some()
|
||||||
}
|
}
|
||||||
|
|
||||||
pub struct QuickSearchApp {
|
pub struct QuickSearchApp {
|
||||||
|
|
@ -90,9 +115,12 @@ pub struct QuickSearchApp {
|
||||||
manage: ManageTab,
|
manage: ManageTab,
|
||||||
dups: DuplicatesTab,
|
dups: DuplicatesTab,
|
||||||
logs: LogsTab,
|
logs: LogsTab,
|
||||||
options: OptionsWindow,
|
settings: SettingsTab,
|
||||||
/// Set when applying a config that invalidates the stored index.
|
/// Set when applying a config that invalidates the stored index.
|
||||||
rebuild_prompt: Option<Vec<ConfigChange>>,
|
rebuild_prompt: Option<Vec<ConfigChange>>,
|
||||||
|
/// The first-start tour, while it is open. Only ever `Some` for a config
|
||||||
|
/// file this version created — see [`crate::tutorial`].
|
||||||
|
tutorial: Option<crate::tutorial::Tutorial>,
|
||||||
/// Set while the "delete the index?" confirmation is open.
|
/// Set while the "delete the index?" confirmation is open.
|
||||||
clear_prompt: bool,
|
clear_prompt: bool,
|
||||||
/// Nested roots found in the loaded config; shown as a modal over the
|
/// Nested roots found in the loaded config; shown as a modal over the
|
||||||
|
|
@ -112,8 +140,12 @@ pub struct QuickSearchApp {
|
||||||
/// Set when the watcher gave up on the directory budget and live
|
/// Set when the watcher gave up on the directory budget and live
|
||||||
/// updates are off.
|
/// updates are off.
|
||||||
watch_cap_prompt: Option<WatchError>,
|
watch_cap_prompt: Option<WatchError>,
|
||||||
|
/// The byte-for-byte check of one duplicate group, while its modal is up.
|
||||||
|
verify: Option<VerifyModal>,
|
||||||
/// In-flight security flow (enable/disable/change password).
|
/// In-flight security flow (enable/disable/change password).
|
||||||
security_prompt: Option<SecurityPrompt>,
|
security_prompt: Option<SecurityPrompt>,
|
||||||
|
/// In-flight show-key flow (confirm password, then reveal).
|
||||||
|
key_prompt: Option<KeyPrompt>,
|
||||||
/// A navigation held by the unsaved-changes guard; see [`NavIntent`].
|
/// A navigation held by the unsaved-changes guard; see [`NavIntent`].
|
||||||
pending_nav: Option<NavIntent>,
|
pending_nav: Option<NavIntent>,
|
||||||
/// The guard resolved a Quit: let the next close request through.
|
/// The guard resolved a Quit: let the next close request through.
|
||||||
|
|
@ -129,8 +161,10 @@ mod security;
|
||||||
mod status_bar;
|
mod status_bar;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests;
|
mod tests;
|
||||||
|
mod verify;
|
||||||
|
|
||||||
use security::SecurityPrompt;
|
use security::{KeyPrompt, SecurityPrompt};
|
||||||
|
use verify::VerifyModal;
|
||||||
|
|
||||||
impl QuickSearchApp {
|
impl QuickSearchApp {
|
||||||
/// `initial_query` pre-fills the search box and fires a search on the
|
/// `initial_query` pre-fills the search box and fires a search on the
|
||||||
|
|
@ -178,7 +212,12 @@ impl QuickSearchApp {
|
||||||
} else {
|
} else {
|
||||||
(Tab::Manage, Some(nested))
|
(Tab::Manage, Some(nested))
|
||||||
};
|
};
|
||||||
let mut search = SearchTab::new(fuzzy);
|
// `Some(false)` means a config file *this version wrote*, which is
|
||||||
|
// the only thing that counts as a first start. A key that is absent
|
||||||
|
// (`None`) belongs to an installation that upgraded into this version
|
||||||
|
// and has already found its way around.
|
||||||
|
let tutorial = (cfg.ui.tutorial_seen == Some(false)).then(crate::tutorial::Tutorial::new);
|
||||||
|
let mut search = SearchTab::new(fuzzy, cfg.search.columns.clone(), cfg.search.live_results);
|
||||||
if let Some(query) = initial_query {
|
if let Some(query) = initial_query {
|
||||||
search.seed(query);
|
search.seed(query);
|
||||||
}
|
}
|
||||||
|
|
@ -190,8 +229,9 @@ impl QuickSearchApp {
|
||||||
manage: ManageTab::new(),
|
manage: ManageTab::new(),
|
||||||
dups: DuplicatesTab::new(),
|
dups: DuplicatesTab::new(),
|
||||||
logs: LogsTab::new(),
|
logs: LogsTab::new(),
|
||||||
options: OptionsWindow::new(),
|
settings: SettingsTab::new(),
|
||||||
rebuild_prompt: None,
|
rebuild_prompt: None,
|
||||||
|
tutorial,
|
||||||
clear_prompt: false,
|
clear_prompt: false,
|
||||||
nested_prompt,
|
nested_prompt,
|
||||||
key_source,
|
key_source,
|
||||||
|
|
@ -199,7 +239,9 @@ impl QuickSearchApp {
|
||||||
reconcile_owed,
|
reconcile_owed,
|
||||||
reconcile_owed_since,
|
reconcile_owed_since,
|
||||||
watch_cap_prompt: None,
|
watch_cap_prompt: None,
|
||||||
|
verify: None,
|
||||||
security_prompt: None,
|
security_prompt: None,
|
||||||
|
key_prompt: None,
|
||||||
pending_nav: None,
|
pending_nav: None,
|
||||||
quit_confirmed: false,
|
quit_confirmed: false,
|
||||||
config_error,
|
config_error,
|
||||||
|
|
@ -222,6 +264,10 @@ impl QuickSearchApp {
|
||||||
let Some(search) = self.backend.search() else {
|
let Some(search) = self.backend.search() else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
|
// The single funnel every search goes through — the debounce, `seed`,
|
||||||
|
// and every `actions.rerun` producer — so it is the one place the old
|
||||||
|
// results' watches have to be dropped.
|
||||||
|
self.backend.clear_live();
|
||||||
let generation = search.search(&self.search.query, self.search_options());
|
let generation = search.search(&self.search.query, self.search_options());
|
||||||
self.search.on_search_started(generation);
|
self.search.on_search_started(generation);
|
||||||
}
|
}
|
||||||
|
|
@ -232,6 +278,32 @@ impl QuickSearchApp {
|
||||||
self.backend.start_duplicates(&cfg, ctx.clone());
|
self.backend.start_duplicates(&cfg, ctx.clone());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Move to another tab, running what leaving one tab and arriving at the
|
||||||
|
/// other owe. Every switch goes through here — including the ones the
|
||||||
|
/// unsaved-changes guard completes a frame later, which is why this is a
|
||||||
|
/// funnel rather than a comparison against the previous frame's tab.
|
||||||
|
fn switch_tab(&mut self, ctx: &egui::Context, to: Tab) {
|
||||||
|
if self.tab == to {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
match self.tab {
|
||||||
|
// Watching rows nobody is looking at costs descriptors for
|
||||||
|
// nothing.
|
||||||
|
Tab::Search => {
|
||||||
|
self.backend.clear_live();
|
||||||
|
self.search.reset_live();
|
||||||
|
}
|
||||||
|
// A draft kept while the config is edited elsewhere would go
|
||||||
|
// stale, and applying it later would revert those edits.
|
||||||
|
Tab::Settings => self.settings.discard(),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
self.tab = to;
|
||||||
|
if to == Tab::Duplicates {
|
||||||
|
self.start_duplicates_scan(ctx);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Save + route an edited config to the running services. Reports
|
/// Save + route an edited config to the running services. Reports
|
||||||
/// whether the config was accepted — a `false` means nothing was saved
|
/// whether the config was accepted — a `false` means nothing was saved
|
||||||
/// and the caller must keep any staged edits alive.
|
/// and the caller must keep any staged edits alive.
|
||||||
|
|
@ -290,6 +362,16 @@ impl QuickSearchApp {
|
||||||
self.rebuild_prompt = Some(changes);
|
self.rebuild_prompt = Some(changes);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
self.search.live_enabled = new.search.live_results;
|
||||||
|
if new.search.live_results {
|
||||||
|
// The watcher holds a copy of the config for its extraction
|
||||||
|
// limits and filters, so a config edit has to re-arm; dropping
|
||||||
|
// the tab-side state is what makes the next frame do it.
|
||||||
|
self.search.reset_live();
|
||||||
|
} else {
|
||||||
|
self.backend.clear_live();
|
||||||
|
self.search.reset_live();
|
||||||
|
}
|
||||||
self.cfg = new;
|
self.cfg = new;
|
||||||
true
|
true
|
||||||
}
|
}
|
||||||
|
|
@ -297,18 +379,23 @@ impl QuickSearchApp {
|
||||||
/// What the system-wide search shortcut does once the window is up:
|
/// What the system-wide search shortcut does once the window is up:
|
||||||
/// show the Search tab with the caret in the query box and any existing
|
/// show the Search tab with the caret in the query box and any existing
|
||||||
/// text selected.
|
/// text selected.
|
||||||
pub(crate) fn activate_search(&mut self) {
|
pub(crate) fn activate_search(&mut self, ctx: &egui::Context) {
|
||||||
if switch_needs_guard(self.tab, self.manage.is_dirty(), self.pending_nav.is_some()) {
|
if switch_needs_guard(
|
||||||
|
self.tab,
|
||||||
|
self.manage.is_dirty(),
|
||||||
|
self.settings.is_dirty(&self.cfg),
|
||||||
|
self.pending_nav.is_some(),
|
||||||
|
) {
|
||||||
self.pending_nav = Some(NavIntent::SwitchTab(Tab::Search));
|
self.pending_nav = Some(NavIntent::SwitchTab(Tab::Search));
|
||||||
} else {
|
} else {
|
||||||
self.tab = Tab::Search;
|
self.switch_tab(ctx, Tab::Search);
|
||||||
}
|
}
|
||||||
self.search.request_focus();
|
self.search.request_focus();
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether the Options window is currently reading a key press to bind.
|
/// Whether the Settings tab is currently reading a key press to bind.
|
||||||
pub(crate) fn capturing_hotkey(&self) -> bool {
|
pub(crate) fn capturing_hotkey(&self) -> bool {
|
||||||
self.options.capturing_hotkey()
|
self.tab == Tab::Settings && self.settings.capturing_hotkey()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Switch the indexing mode and write it to the config immediately: a
|
/// Switch the indexing mode and write it to the config immediately: a
|
||||||
|
|
@ -334,6 +421,21 @@ impl QuickSearchApp {
|
||||||
self.search
|
self.search
|
||||||
.apply_update(update, self.cfg.search.display_limit);
|
.apply_update(update, self.cfg.search.display_limit);
|
||||||
}
|
}
|
||||||
|
// Every live update is something the watcher read off the disk that
|
||||||
|
// the index has not been told about. Handing the paths back keeps the
|
||||||
|
// index from drifting away from the rows on screen — and it is the
|
||||||
|
// only thing that does so while indexing is stopped.
|
||||||
|
let mut touched: Vec<PathBuf> = Vec::new();
|
||||||
|
while let Ok(update) = self.backend.live_rx.try_recv() {
|
||||||
|
touched.push(PathBuf::from(update.path()));
|
||||||
|
// A rename has two sides: the old path leaves the index and the
|
||||||
|
// new one enters it.
|
||||||
|
if let quicksearch_core::live::LiveUpdate::Renamed { to, .. } = &update {
|
||||||
|
touched.push(PathBuf::from(to));
|
||||||
|
}
|
||||||
|
self.search.apply_live(update);
|
||||||
|
}
|
||||||
|
self.backend.reindex_live_paths(touched);
|
||||||
// Duplicates worker.
|
// Duplicates worker.
|
||||||
if let Some(rx) = &self.backend.dup_job {
|
if let Some(rx) = &self.backend.dup_job {
|
||||||
use std::sync::mpsc::TryRecvError;
|
use std::sync::mpsc::TryRecvError;
|
||||||
|
|
@ -350,6 +452,7 @@ impl QuickSearchApp {
|
||||||
self.backend.dup_job = None;
|
self.backend.dup_job = None;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
self.drain_verify();
|
||||||
}
|
}
|
||||||
|
|
||||||
fn tick_debounce(&mut self, ctx: &egui::Context) {
|
fn tick_debounce(&mut self, ctx: &egui::Context) {
|
||||||
|
|
@ -389,7 +492,7 @@ impl QuickSearchApp {
|
||||||
}
|
}
|
||||||
|
|
||||||
pub(crate) fn capture_search_settled(&self) -> bool {
|
pub(crate) fn capture_search_settled(&self) -> bool {
|
||||||
self.search.capture_settled()
|
self.search.settled()
|
||||||
}
|
}
|
||||||
|
|
||||||
pub(crate) fn capture_dups_done(&self) -> bool {
|
pub(crate) fn capture_dups_done(&self) -> bool {
|
||||||
|
|
@ -419,6 +522,12 @@ impl QuickSearchApp {
|
||||||
pub(crate) fn pin_live_fields(new: &mut Config, live: &Config) {
|
pub(crate) fn pin_live_fields(new: &mut Config, live: &Config) {
|
||||||
new.security = live.security.clone();
|
new.security = live.security.clone();
|
||||||
new.indexing.auto_index = live.indexing.auto_index;
|
new.indexing.auto_index = live.indexing.auto_index;
|
||||||
|
// The column picker writes straight to the live config the moment a
|
||||||
|
// checkbox moves — from the table header *or* from the Settings tab,
|
||||||
|
// which is why the Settings controls for it are not draft-backed. Pinning
|
||||||
|
// here is what stops a draft taken before a header-menu change from
|
||||||
|
// undoing it on Apply.
|
||||||
|
new.search.columns = live.search.columns.clone();
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Keep the configured UI scale within sane, recoverable bounds.
|
/// Keep the configured UI scale within sane, recoverable bounds.
|
||||||
|
|
@ -448,7 +557,8 @@ pub(crate) fn apply_theme(ctx: &egui::Context, setting: &str) {
|
||||||
|
|
||||||
impl eframe::App for QuickSearchApp {
|
impl eframe::App for QuickSearchApp {
|
||||||
fn update(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
|
fn update(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
|
||||||
// First, so `previous_tab` below sees pre-navigation state.
|
// First, so a scripted navigation is held before the tab strip reads
|
||||||
|
// this frame's state.
|
||||||
#[cfg(feature = "capture")]
|
#[cfg(feature = "capture")]
|
||||||
self.capture_tick(ctx);
|
self.capture_tick(ctx);
|
||||||
|
|
||||||
|
|
@ -461,7 +571,7 @@ impl eframe::App for QuickSearchApp {
|
||||||
if ctx.input(|i| i.viewport().close_requested())
|
if ctx.input(|i| i.viewport().close_requested())
|
||||||
&& !self.quit_confirmed
|
&& !self.quit_confirmed
|
||||||
&& (self.manage.is_dirty()
|
&& (self.manage.is_dirty()
|
||||||
|| self.options.is_dirty(&self.cfg)
|
|| self.settings.is_dirty(&self.cfg)
|
||||||
|| self.backend.coordinator.reconciling())
|
|| self.backend.coordinator.reconciling())
|
||||||
{
|
{
|
||||||
ctx.send_viewport_cmd(egui::ViewportCommand::CancelClose);
|
ctx.send_viewport_cmd(egui::ViewportCommand::CancelClose);
|
||||||
|
|
@ -471,7 +581,6 @@ impl eframe::App for QuickSearchApp {
|
||||||
|
|
||||||
self.status_bar(ctx);
|
self.status_bar(ctx);
|
||||||
|
|
||||||
let previous_tab = self.tab;
|
|
||||||
// Tab clicks land on a local first so the unsaved-changes guard can
|
// Tab clicks land on a local first so the unsaved-changes guard can
|
||||||
// hold them.
|
// hold them.
|
||||||
let mut requested = self.tab;
|
let mut requested = self.tab;
|
||||||
|
|
@ -482,31 +591,21 @@ impl eframe::App for QuickSearchApp {
|
||||||
ui.selectable_value(&mut requested, Tab::Duplicates, "Duplicates");
|
ui.selectable_value(&mut requested, Tab::Duplicates, "Duplicates");
|
||||||
ui.selectable_value(&mut requested, Tab::Logs, "Logs");
|
ui.selectable_value(&mut requested, Tab::Logs, "Logs");
|
||||||
ui.selectable_value(&mut requested, Tab::Help, "Help");
|
ui.selectable_value(&mut requested, Tab::Help, "Help");
|
||||||
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
ui.selectable_value(&mut requested, Tab::Settings, "Settings");
|
||||||
if ui.button("⚙").on_hover_text("Options").clicked() {
|
|
||||||
if !self.options.open {
|
|
||||||
self.options.open_with(&self.cfg);
|
|
||||||
} else if self.options.is_dirty(&self.cfg) {
|
|
||||||
if self.pending_nav.is_none() {
|
|
||||||
self.pending_nav = Some(NavIntent::CloseOptions);
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
self.options.close_discard();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
if requested != self.tab {
|
if requested != self.tab {
|
||||||
if switch_needs_guard(self.tab, self.manage.is_dirty(), self.pending_nav.is_some()) {
|
if switch_needs_guard(
|
||||||
|
self.tab,
|
||||||
|
self.manage.is_dirty(),
|
||||||
|
self.settings.is_dirty(&self.cfg),
|
||||||
|
self.pending_nav.is_some(),
|
||||||
|
) {
|
||||||
self.pending_nav = Some(NavIntent::SwitchTab(requested));
|
self.pending_nav = Some(NavIntent::SwitchTab(requested));
|
||||||
} else {
|
} else {
|
||||||
self.tab = requested;
|
self.switch_tab(ctx, requested);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if self.tab == Tab::Duplicates && previous_tab != Tab::Duplicates {
|
|
||||||
self.start_duplicates_scan(ctx);
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(err) = &self.config_error {
|
if let Some(err) = &self.config_error {
|
||||||
let err = err.clone();
|
let err = err.clone();
|
||||||
|
|
@ -533,6 +632,17 @@ impl eframe::App for QuickSearchApp {
|
||||||
self.config_error = Some(e);
|
self.config_error = Some(e);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Live state, saved the moment it changes — like the fuzzy
|
||||||
|
// default above, and unlike anything edited through the
|
||||||
|
// Settings draft. The Settings tab's own column controls take
|
||||||
|
// this same path, so the two editors cannot disagree and a
|
||||||
|
// stale draft cannot revert either of them.
|
||||||
|
if let Some(columns) = actions.save_columns {
|
||||||
|
self.cfg.search.columns = columns;
|
||||||
|
if let Err(e) = self.cfg.save() {
|
||||||
|
self.config_error = Some(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
if let Some(pattern) = actions.persist_ignore {
|
if let Some(pattern) = actions.persist_ignore {
|
||||||
let mut new_cfg = self.cfg.clone();
|
let mut new_cfg = self.cfg.clone();
|
||||||
if !new_cfg.indexing.ignore_patterns.contains(&pattern) {
|
if !new_cfg.indexing.ignore_patterns.contains(&pattern) {
|
||||||
|
|
@ -540,6 +650,10 @@ impl eframe::App for QuickSearchApp {
|
||||||
self.apply_new_config(ctx, new_cfg);
|
self.apply_new_config(ctx, new_cfg);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if let Some(targets) = actions.live_targets {
|
||||||
|
self.backend
|
||||||
|
.watch_live(&self.search.query, targets, &self.cfg);
|
||||||
|
}
|
||||||
if actions.rerun {
|
if actions.rerun {
|
||||||
self.start_search();
|
self.start_search();
|
||||||
}
|
}
|
||||||
|
|
@ -571,32 +685,54 @@ impl eframe::App for QuickSearchApp {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Tab::Duplicates => {
|
Tab::Duplicates => {
|
||||||
let actions = self.dups.ui(ui);
|
let actions = self.dups.ui(ui, self.verify.is_some());
|
||||||
if actions.refresh {
|
if actions.refresh {
|
||||||
self.start_duplicates_scan(ctx);
|
self.start_duplicates_scan(ctx);
|
||||||
}
|
}
|
||||||
|
if let Some(paths) = actions.verify {
|
||||||
|
let paths: Vec<std::path::PathBuf> =
|
||||||
|
paths.into_iter().map(std::path::PathBuf::from).collect();
|
||||||
|
self.backend.start_verify(paths.clone(), ctx.clone());
|
||||||
|
self.verify = Some(VerifyModal::new(paths));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
Tab::Logs => self.logs.ui(ui),
|
Tab::Logs => self.logs.ui(ui),
|
||||||
Tab::Help => crate::help_tab::ui(ui),
|
Tab::Help => {
|
||||||
});
|
if crate::help_tab::ui(ui) {
|
||||||
|
self.show_tutorial();
|
||||||
let options_out = self.options.ui(ctx, &self.cfg);
|
}
|
||||||
if let Some(new_cfg) = options_out.applied {
|
}
|
||||||
|
Tab::Settings => {
|
||||||
|
let out = self.settings.ui(ui, &self.cfg);
|
||||||
|
if let Some(new_cfg) = out.applied {
|
||||||
self.apply_new_config(ctx, new_cfg);
|
self.apply_new_config(ctx, new_cfg);
|
||||||
}
|
}
|
||||||
if let Some(action) = options_out.security {
|
if let Some(action) = out.security {
|
||||||
self.handle_security_action(action);
|
self.handle_security_action(action);
|
||||||
}
|
}
|
||||||
if options_out.close_requested && self.pending_nav.is_none() {
|
// Same live path the table header's picker takes, so the two
|
||||||
self.pending_nav = Some(NavIntent::CloseOptions);
|
// controls stay in step and neither needs an Apply.
|
||||||
|
if let Some(columns) = out.columns {
|
||||||
|
self.cfg.search.columns = columns.clone();
|
||||||
|
self.search.columns = columns;
|
||||||
|
self.search.mark_sort_dirty();
|
||||||
|
if let Err(e) = self.cfg.save() {
|
||||||
|
self.config_error = Some(e);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
self.rebuild_prompt_ui(ctx);
|
self.rebuild_prompt_ui(ctx);
|
||||||
self.security_prompt_ui(ctx);
|
self.security_prompt_ui(ctx);
|
||||||
|
self.key_prompt_ui(ctx);
|
||||||
self.clear_prompt_ui(ctx);
|
self.clear_prompt_ui(ctx);
|
||||||
self.nested_prompt_ui(ctx);
|
self.nested_prompt_ui(ctx);
|
||||||
// Ahead of the watch-cap warning: on a fresh upgrade both can be true.
|
// Ahead of the watch-cap warning: on a fresh upgrade both can be true.
|
||||||
self.stale_index_prompt_ui(ctx);
|
self.stale_index_prompt_ui(ctx);
|
||||||
self.watch_cap_prompt_ui(ctx);
|
self.watch_cap_prompt_ui(ctx);
|
||||||
|
self.verify_modal_ui(ctx);
|
||||||
|
self.tutorial_ui(ctx);
|
||||||
// Last: the guard must sit above everything else on screen.
|
// Last: the guard must sit above everything else on screen.
|
||||||
self.unsaved_prompt_ui(ctx);
|
self.unsaved_prompt_ui(ctx);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -84,7 +84,7 @@ impl QuickSearchApp {
|
||||||
});
|
});
|
||||||
if close == Some(true) {
|
if close == Some(true) {
|
||||||
self.nested_prompt = None;
|
self.nested_prompt = None;
|
||||||
self.tab = Tab::Manage;
|
self.switch_tab(ctx, Tab::Manage);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -187,6 +187,29 @@ impl QuickSearchApp {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The first-start tour. Dismissal is written straight to the config, the
|
||||||
|
/// way the fuzzy default is — not through the Settings draft, which this
|
||||||
|
/// has nothing to do with.
|
||||||
|
pub(super) fn tutorial_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(tour) = &mut self.tutorial else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let roots = self.cfg.paths.indexing_paths.clone();
|
||||||
|
if tour.ui(ctx, &roots) {
|
||||||
|
self.tutorial = None;
|
||||||
|
self.cfg.ui.tutorial_seen = Some(true);
|
||||||
|
if let Err(e) = self.cfg.save() {
|
||||||
|
self.config_error = Some(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Re-open the tour from the Help tab. Nothing is written until it is
|
||||||
|
/// dismissed again, so a re-read costs the config nothing.
|
||||||
|
pub(crate) fn show_tutorial(&mut self) {
|
||||||
|
self.tutorial = Some(crate::tutorial::Tutorial::new());
|
||||||
|
}
|
||||||
|
|
||||||
pub(super) fn clear_prompt_ui(&mut self, ctx: &egui::Context) {
|
pub(super) fn clear_prompt_ui(&mut self, ctx: &egui::Context) {
|
||||||
if !self.clear_prompt {
|
if !self.clear_prompt {
|
||||||
return;
|
return;
|
||||||
|
|
@ -225,8 +248,10 @@ impl QuickSearchApp {
|
||||||
let Some(intent) = self.pending_nav else {
|
let Some(intent) = self.pending_nav else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
let dirty = (self.manage.is_dirty(), self.options.is_dirty(&self.cfg));
|
let dirty = (self.manage.is_dirty(), self.settings.is_dirty(&self.cfg));
|
||||||
let Some(source) = guard_source(intent, dirty.0, dirty.1) else {
|
// `self.tab` is the tab being left: the switch itself is what the
|
||||||
|
// intent is holding back.
|
||||||
|
let Some(source) = guard_source(intent, self.tab, dirty.0, dirty.1) else {
|
||||||
// Inside the guard: the Discard-then-quit path sets
|
// Inside the guard: the Discard-then-quit path sets
|
||||||
// `quit_confirmed` and never returns to the close-request check,
|
// `quit_confirmed` and never returns to the close-request check,
|
||||||
// so a warning living only there would be skipped.
|
// so a warning living only there would be skipped.
|
||||||
|
|
@ -249,7 +274,7 @@ impl QuickSearchApp {
|
||||||
Some(UnsavedChoice::Cancel) => self.pending_nav = None,
|
Some(UnsavedChoice::Cancel) => self.pending_nav = None,
|
||||||
Some(UnsavedChoice::Discard) => match source {
|
Some(UnsavedChoice::Discard) => match source {
|
||||||
UnsavedSource::Manage => self.manage.discard(),
|
UnsavedSource::Manage => self.manage.discard(),
|
||||||
UnsavedSource::Options => self.options.close_discard(),
|
UnsavedSource::Settings => self.settings.discard(),
|
||||||
},
|
},
|
||||||
Some(UnsavedChoice::Apply) => {
|
Some(UnsavedChoice::Apply) => {
|
||||||
let ok = match source {
|
let ok = match source {
|
||||||
|
|
@ -263,11 +288,11 @@ impl QuickSearchApp {
|
||||||
}
|
}
|
||||||
None => true,
|
None => true,
|
||||||
},
|
},
|
||||||
UnsavedSource::Options => match self.options.draft_config() {
|
UnsavedSource::Settings => match self.settings.draft_config() {
|
||||||
Some(cfg) => {
|
Some(cfg) => {
|
||||||
let ok = self.apply_new_config(ctx, cfg);
|
let ok = self.apply_new_config(ctx, cfg);
|
||||||
if ok {
|
if ok {
|
||||||
self.options.close_discard();
|
self.settings.discard();
|
||||||
}
|
}
|
||||||
ok
|
ok
|
||||||
}
|
}
|
||||||
|
|
@ -287,14 +312,7 @@ impl QuickSearchApp {
|
||||||
pub(super) fn complete_nav(&mut self, ctx: &egui::Context, intent: NavIntent) {
|
pub(super) fn complete_nav(&mut self, ctx: &egui::Context, intent: NavIntent) {
|
||||||
self.pending_nav = None;
|
self.pending_nav = None;
|
||||||
match intent {
|
match intent {
|
||||||
NavIntent::SwitchTab(tab) => {
|
NavIntent::SwitchTab(tab) => self.switch_tab(ctx, tab),
|
||||||
let was = self.tab;
|
|
||||||
self.tab = tab;
|
|
||||||
if tab == Tab::Duplicates && was != Tab::Duplicates {
|
|
||||||
self.start_duplicates_scan(ctx);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
NavIntent::CloseOptions => self.options.close_discard(),
|
|
||||||
NavIntent::Quit => {
|
NavIntent::Quit => {
|
||||||
self.quit_confirmed = true;
|
self.quit_confirmed = true;
|
||||||
ctx.send_viewport_cmd(egui::ViewportCommand::Close);
|
ctx.send_viewport_cmd(egui::ViewportCommand::Close);
|
||||||
|
|
@ -308,7 +326,7 @@ impl QuickSearchApp {
|
||||||
///
|
///
|
||||||
/// Unlike the centered `egui::Window` the other prompts use, `egui::Modal`'s
|
/// Unlike the centered `egui::Window` the other prompts use, `egui::Modal`'s
|
||||||
/// backdrop blocks input to everything behind it — a click landing on the
|
/// backdrop blocks input to everything behind it — a click landing on the
|
||||||
/// tab strip or the Options ✕ would re-trigger or bypass the guard.
|
/// tab strip would re-trigger or bypass the guard.
|
||||||
fn unsaved_changes_modal(ctx: &egui::Context, source: UnsavedSource) -> Option<UnsavedChoice> {
|
fn unsaved_changes_modal(ctx: &egui::Context, source: UnsavedSource) -> Option<UnsavedChoice> {
|
||||||
let mut choice = None;
|
let mut choice = None;
|
||||||
let modal = egui::Modal::new(egui::Id::new("unsaved-guard")).show(ctx, |ui| {
|
let modal = egui::Modal::new(egui::Id::new("unsaved-guard")).show(ctx, |ui| {
|
||||||
|
|
@ -316,7 +334,7 @@ fn unsaved_changes_modal(ctx: &egui::Context, source: UnsavedSource) -> Option<U
|
||||||
ui.heading("Unsaved changes");
|
ui.heading("Unsaved changes");
|
||||||
ui.label(match source {
|
ui.label(match source {
|
||||||
UnsavedSource::Manage => "The Manage Index tab has edits that have not been applied.",
|
UnsavedSource::Manage => "The Manage Index tab has edits that have not been applied.",
|
||||||
UnsavedSource::Options => "The Options window has edits that have not been applied.",
|
UnsavedSource::Settings => "The Settings tab has edits that have not been applied.",
|
||||||
});
|
});
|
||||||
ui.add_space(6.0);
|
ui.add_space(6.0);
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
|
|
@ -527,7 +545,7 @@ mod tests {
|
||||||
/// buttons fire, Esc cancels, and an untouched frame decides nothing.
|
/// buttons fire, Esc cancels, and an untouched frame decides nothing.
|
||||||
#[test]
|
#[test]
|
||||||
fn the_unsaved_modal_reports_each_choice() {
|
fn the_unsaved_modal_reports_each_choice() {
|
||||||
for source in [UnsavedSource::Manage, UnsavedSource::Options] {
|
for source in [UnsavedSource::Manage, UnsavedSource::Settings] {
|
||||||
let ctx = egui::Context::default();
|
let ctx = egui::Context::default();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
modal_frame(&ctx, source, Vec::new()),
|
modal_frame(&ctx, source, Vec::new()),
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,9 @@
|
||||||
|
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
use crate::ui_util::centered_modal;
|
use quicksearch_core::security::SALT_LEN;
|
||||||
|
|
||||||
|
use crate::ui_util::{centered_modal, hint};
|
||||||
|
|
||||||
/// The two-step security flow: collect a password (enable/change), derive
|
/// The two-step security flow: collect a password (enable/change), derive
|
||||||
/// its key off the UI thread, then confirm the mandatory index rebuild.
|
/// its key off the UI thread, then confirm the mandatory index rebuild.
|
||||||
|
|
@ -16,7 +18,10 @@ pub(super) enum SecurityPrompt {
|
||||||
change: bool,
|
change: bool,
|
||||||
},
|
},
|
||||||
Deriving {
|
Deriving {
|
||||||
rx: mpsc::Receiver<(SecurityConfig, IndexKey)>,
|
rx: mpsc::Receiver<IndexKey>,
|
||||||
|
/// Built with the salt the pending key is being derived from, so the
|
||||||
|
/// two always describe each other.
|
||||||
|
new_security: SecurityConfig,
|
||||||
},
|
},
|
||||||
ConfirmRebuild {
|
ConfirmRebuild {
|
||||||
new_security: SecurityConfig,
|
new_security: SecurityConfig,
|
||||||
|
|
@ -33,8 +38,69 @@ impl Drop for SecurityPrompt {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The show-key flow: confirm the password, re-derive from it, then reveal
|
||||||
|
/// the installed key. Nothing here can change the key or the config.
|
||||||
|
pub(super) enum KeyPrompt {
|
||||||
|
Confirm {
|
||||||
|
pw: String,
|
||||||
|
wrong: bool,
|
||||||
|
},
|
||||||
|
Deriving {
|
||||||
|
rx: mpsc::Receiver<IndexKey>,
|
||||||
|
},
|
||||||
|
/// The key as displayed: `0x` followed by 64 hex digits.
|
||||||
|
Reveal {
|
||||||
|
display: String,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for KeyPrompt {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
match self {
|
||||||
|
KeyPrompt::Confirm { pw, .. } => pw.zeroize(),
|
||||||
|
KeyPrompt::Reveal { display } => display.zeroize(),
|
||||||
|
KeyPrompt::Deriving { .. } => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Derive a key off the UI thread. The password is consumed and dropped
|
||||||
|
/// there, so it never outlives the derivation.
|
||||||
|
fn spawn_derive(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
password: Zeroizing<String>,
|
||||||
|
salt: [u8; SALT_LEN],
|
||||||
|
) -> mpsc::Receiver<IndexKey> {
|
||||||
|
let (tx, rx) = mpsc::channel();
|
||||||
|
let repaint = ctx.clone();
|
||||||
|
std::thread::spawn(move || {
|
||||||
|
let key = derive_key(&password, &salt);
|
||||||
|
drop(password);
|
||||||
|
let _ = tx.send(key);
|
||||||
|
repaint.request_repaint();
|
||||||
|
});
|
||||||
|
rx
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Paint the spinner shown while a derivation runs. Not `centered_modal`:
|
||||||
|
/// this one hides its title bar.
|
||||||
|
fn deriving_window(ctx: &egui::Context) {
|
||||||
|
egui::Window::new("Deriving key")
|
||||||
|
.collapsible(false)
|
||||||
|
.resizable(false)
|
||||||
|
.title_bar(false)
|
||||||
|
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.spinner();
|
||||||
|
ui.label("Deriving key…");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
ctx.request_repaint_after(Duration::from_millis(100));
|
||||||
|
}
|
||||||
|
|
||||||
impl QuickSearchApp {
|
impl QuickSearchApp {
|
||||||
/// Route a click in the Options window's Security block. Keychain
|
/// Route a click in the Settings tab's Security block. Keychain
|
||||||
/// toggles act immediately; everything else opens the two-step flow.
|
/// toggles act immediately; everything else opens the two-step flow.
|
||||||
pub(super) fn handle_security_action(&mut self, action: SecurityAction) {
|
pub(super) fn handle_security_action(&mut self, action: SecurityAction) {
|
||||||
match action {
|
match action {
|
||||||
|
|
@ -52,6 +118,12 @@ impl QuickSearchApp {
|
||||||
new_key: None,
|
new_key: None,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
SecurityAction::ShowKey => {
|
||||||
|
self.key_prompt = Some(KeyPrompt::Confirm {
|
||||||
|
pw: String::new(),
|
||||||
|
wrong: false,
|
||||||
|
});
|
||||||
|
}
|
||||||
SecurityAction::SetKeychain(remember) => {
|
SecurityAction::SetKeychain(remember) => {
|
||||||
let db_path = self.cfg.resolved_database_path();
|
let db_path = self.cfg.resolved_database_path();
|
||||||
if remember {
|
if remember {
|
||||||
|
|
@ -139,47 +211,25 @@ impl QuickSearchApp {
|
||||||
} else if submit {
|
} else if submit {
|
||||||
let password = Zeroizing::new(std::mem::take(pw1));
|
let password = Zeroizing::new(std::mem::take(pw1));
|
||||||
pw2.zeroize();
|
pw2.zeroize();
|
||||||
let remember = *remember;
|
|
||||||
purge_security_field_state(ctx);
|
|
||||||
let (tx, rx) = mpsc::channel();
|
|
||||||
let repaint = ctx.clone();
|
|
||||||
std::thread::spawn(move || {
|
|
||||||
let salt = generate_salt();
|
let salt = generate_salt();
|
||||||
let key = derive_key(&password, &salt);
|
|
||||||
drop(password);
|
|
||||||
let new_security = SecurityConfig {
|
let new_security = SecurityConfig {
|
||||||
password_protected: true,
|
password_protected: true,
|
||||||
salt: Some(salt_to_hex(&salt)),
|
salt: Some(salt_to_hex(&salt)),
|
||||||
use_keychain: remember,
|
use_keychain: *remember,
|
||||||
};
|
};
|
||||||
let _ = tx.send((new_security, key));
|
purge_security_field_state(ctx);
|
||||||
repaint.request_repaint();
|
let rx = spawn_derive(ctx, password, salt);
|
||||||
});
|
self.security_prompt = Some(SecurityPrompt::Deriving { rx, new_security });
|
||||||
self.security_prompt = Some(SecurityPrompt::Deriving { rx });
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
SecurityPrompt::Deriving { rx } => match rx.try_recv() {
|
SecurityPrompt::Deriving { rx, new_security } => match rx.try_recv() {
|
||||||
Ok((new_security, key)) => {
|
Ok(key) => {
|
||||||
self.security_prompt = Some(SecurityPrompt::ConfirmRebuild {
|
self.security_prompt = Some(SecurityPrompt::ConfirmRebuild {
|
||||||
new_security,
|
new_security: new_security.clone(),
|
||||||
new_key: Some(key),
|
new_key: Some(key),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
Err(mpsc::TryRecvError::Empty) => {
|
Err(mpsc::TryRecvError::Empty) => deriving_window(ctx),
|
||||||
// Not `centered_modal`: this one hides its title bar.
|
|
||||||
egui::Window::new("Deriving key")
|
|
||||||
.collapsible(false)
|
|
||||||
.resizable(false)
|
|
||||||
.title_bar(false)
|
|
||||||
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
|
||||||
.show(ctx, |ui| {
|
|
||||||
ui.horizontal(|ui| {
|
|
||||||
ui.spinner();
|
|
||||||
ui.label("Deriving key…");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
ctx.request_repaint_after(Duration::from_millis(100));
|
|
||||||
}
|
|
||||||
Err(mpsc::TryRecvError::Disconnected) => {
|
Err(mpsc::TryRecvError::Disconnected) => {
|
||||||
self.config_error = Some("key derivation thread died".to_string());
|
self.config_error = Some("key derivation thread died".to_string());
|
||||||
self.security_prompt = None;
|
self.security_prompt = None;
|
||||||
|
|
@ -222,6 +272,74 @@ impl QuickSearchApp {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Render the show-key flow (drawn with the other modals). Only ever
|
||||||
|
/// open while protection is on, so a salt and a process key both exist.
|
||||||
|
pub(super) fn key_prompt_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(prompt) = &mut self.key_prompt else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
match prompt {
|
||||||
|
KeyPrompt::Confirm { pw, wrong } => {
|
||||||
|
let (submit, cancel) = confirm_key_modal(ctx, pw, *wrong);
|
||||||
|
if cancel {
|
||||||
|
self.key_prompt = None; // Drop impl zeroizes
|
||||||
|
purge_security_field_state(ctx);
|
||||||
|
} else if submit {
|
||||||
|
let password = Zeroizing::new(std::mem::take(pw));
|
||||||
|
purge_security_field_state(ctx);
|
||||||
|
match self.cfg.security.salt_bytes() {
|
||||||
|
Ok(salt) => {
|
||||||
|
let rx = spawn_derive(ctx, password, salt);
|
||||||
|
self.key_prompt = Some(KeyPrompt::Deriving { rx });
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
self.config_error = Some(e);
|
||||||
|
self.key_prompt = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
KeyPrompt::Deriving { rx } => match rx.try_recv() {
|
||||||
|
Ok(key) => match db::process_key_hex() {
|
||||||
|
// What is shown is the installed key, not the derived
|
||||||
|
// one: it is the key that actually opens the index.
|
||||||
|
Some(installed) => match reveal_display(&installed, &key.to_hex()) {
|
||||||
|
Some(display) => {
|
||||||
|
self.key_prompt = Some(KeyPrompt::Reveal { display });
|
||||||
|
}
|
||||||
|
None => {
|
||||||
|
self.key_prompt = Some(KeyPrompt::Confirm {
|
||||||
|
pw: String::new(),
|
||||||
|
wrong: true,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
},
|
||||||
|
None => {
|
||||||
|
// Unreachable while protected — the gate always
|
||||||
|
// installs a key before the app starts.
|
||||||
|
self.config_error =
|
||||||
|
Some("no key installed; restart and unlock first".to_string());
|
||||||
|
self.key_prompt = None;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
Err(mpsc::TryRecvError::Empty) => deriving_window(ctx),
|
||||||
|
Err(mpsc::TryRecvError::Disconnected) => {
|
||||||
|
self.config_error = Some("key derivation thread died".to_string());
|
||||||
|
self.key_prompt = None;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
KeyPrompt::Reveal { display } => {
|
||||||
|
let (copy, close) = reveal_key_modal(ctx, display);
|
||||||
|
if copy {
|
||||||
|
ctx.copy_text(display.clone());
|
||||||
|
}
|
||||||
|
if close {
|
||||||
|
self.key_prompt = None; // Drop impl zeroizes
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Commit a confirmed security change: config, keychain, process key —
|
/// Commit a confirmed security change: config, keychain, process key —
|
||||||
/// in that order, before the rebuild so the fresh index is created
|
/// in that order, before the rebuild so the fresh index is created
|
||||||
/// under the new key (or none).
|
/// under the new key (or none).
|
||||||
|
|
@ -255,11 +373,92 @@ impl QuickSearchApp {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Id of the show-key confirmation field, shared by the widget and the
|
||||||
|
/// purge below.
|
||||||
|
const SHOW_KEY_FIELD: &str = "show-key-pw";
|
||||||
|
|
||||||
/// Drop egui's retained text-field state (buffer + undo history) for the
|
/// Drop egui's retained text-field state (buffer + undo history) for the
|
||||||
/// password dialog fields.
|
/// password dialog fields.
|
||||||
fn purge_security_field_state(ctx: &egui::Context) {
|
fn purge_security_field_state(ctx: &egui::Context) {
|
||||||
ctx.data_mut(|d| {
|
ctx.data_mut(|d| {
|
||||||
d.remove::<egui::text_edit::TextEditState>(egui::Id::new("security-pw1"));
|
d.remove::<egui::text_edit::TextEditState>(egui::Id::new("security-pw1"));
|
||||||
d.remove::<egui::text_edit::TextEditState>(egui::Id::new("security-pw2"));
|
d.remove::<egui::text_edit::TextEditState>(egui::Id::new("security-pw2"));
|
||||||
|
d.remove::<egui::text_edit::TextEditState>(egui::Id::new(SHOW_KEY_FIELD));
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The display form of the installed key, or `None` when the password the
|
||||||
|
/// user typed does not derive it. Both arguments come from
|
||||||
|
/// [`IndexKey::to_hex`], which is always lowercase, so a plain comparison is
|
||||||
|
/// exact; nothing secret is learned from its timing, since the caller
|
||||||
|
/// already holds the guess.
|
||||||
|
fn reveal_display(installed_hex: &str, derived_hex: &str) -> Option<String> {
|
||||||
|
(installed_hex == derived_hex).then(|| format!("0x{}", installed_hex))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Paint the password confirmation; `(submit, cancel)` from its buttons.
|
||||||
|
/// Free, like the reveal below, so both halves of the flow can be rendered
|
||||||
|
/// against a bare context.
|
||||||
|
fn confirm_key_modal(ctx: &egui::Context, pw: &mut String, wrong: bool) -> (bool, bool) {
|
||||||
|
centered_modal(ctx, "Show database key", |ui| {
|
||||||
|
ui.set_max_width(360.0);
|
||||||
|
ui.label(
|
||||||
|
"Confirm your password to show the raw key the index is \
|
||||||
|
encrypted with.",
|
||||||
|
);
|
||||||
|
let field = ui.add(
|
||||||
|
egui::TextEdit::singleline(pw)
|
||||||
|
.id(egui::Id::new(SHOW_KEY_FIELD))
|
||||||
|
.password(true)
|
||||||
|
.hint_text("Password")
|
||||||
|
.desired_width(240.0),
|
||||||
|
);
|
||||||
|
// On open, and again after a wrong attempt. Never steals focus from
|
||||||
|
// something the user moved to themselves.
|
||||||
|
if ui.memory(|m| m.focused().is_none()) {
|
||||||
|
field.request_focus();
|
||||||
|
}
|
||||||
|
if wrong {
|
||||||
|
ui.colored_label(ui.visuals().error_fg_color, "That password is not correct.");
|
||||||
|
}
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
let ok = !pw.is_empty();
|
||||||
|
// Enter in the field submits, like the unlock screen.
|
||||||
|
let entered = field.lost_focus() && ui.input(|i| i.key_pressed(egui::Key::Enter));
|
||||||
|
let submit =
|
||||||
|
ui.add_enabled(ok, egui::Button::new("Show key")).clicked() || (ok && entered);
|
||||||
|
(submit, ui.button("Cancel").clicked())
|
||||||
|
})
|
||||||
|
.inner
|
||||||
|
})
|
||||||
|
.unwrap_or((false, false))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Paint the revealed key; `(copy, close)` from its buttons. A free function
|
||||||
|
/// rather than a method so it can be rendered against a bare context.
|
||||||
|
fn reveal_key_modal(ctx: &egui::Context, display: &str) -> (bool, bool) {
|
||||||
|
centered_modal(ctx, "Database key", |ui| {
|
||||||
|
ui.set_max_width(420.0);
|
||||||
|
ui.label(
|
||||||
|
"This is the SQLCipher raw key for the index. Anyone holding it can \
|
||||||
|
read the index without the password.",
|
||||||
|
);
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.horizontal_wrapped(|ui| {
|
||||||
|
ui.label(egui::RichText::new(display).monospace());
|
||||||
|
});
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.label(hint(
|
||||||
|
"Other SQLCipher tools take the key in this form. A copy stays on the \
|
||||||
|
clipboard until something else replaces it.",
|
||||||
|
));
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.horizontal(|ui| (ui.button("Copy").clicked(), ui.button("Close").clicked()))
|
||||||
|
.inner
|
||||||
|
})
|
||||||
|
.unwrap_or((false, false))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
#[path = "security_tests.rs"]
|
||||||
|
mod tests;
|
||||||
|
|
|
||||||
169
crates/quicksearch-gui/src/app/security_tests.rs
Normal file
169
crates/quicksearch-gui/src/app/security_tests.rs
Normal file
|
|
@ -0,0 +1,169 @@
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
use crate::test_ui::{click_at, painted_text, painted_text_center, raw_input};
|
||||||
|
|
||||||
|
const SCREEN: egui::Vec2 = egui::vec2(1000.0, 700.0);
|
||||||
|
|
||||||
|
/// Two 64-digit keys that differ, in the lowercase form [`IndexKey::to_hex`]
|
||||||
|
/// produces.
|
||||||
|
const KEY: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
|
||||||
|
const OTHER: &str = "fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210";
|
||||||
|
|
||||||
|
/// Two passes: an `egui::Window` is measured on its first frame and placed on
|
||||||
|
/// the next, so a single pass paints nothing to read back. Same shape as the
|
||||||
|
/// verify modal's test frame.
|
||||||
|
fn frame(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
display: &str,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (egui::FullOutput, (bool, bool)) {
|
||||||
|
let _ = ctx.run(raw_input(SCREEN, Vec::new()), |ctx| {
|
||||||
|
reveal_key_modal(ctx, display);
|
||||||
|
});
|
||||||
|
let mut buttons = (false, false);
|
||||||
|
let out = ctx.run(raw_input(SCREEN, events), |ctx| {
|
||||||
|
buttons = reveal_key_modal(ctx, display);
|
||||||
|
});
|
||||||
|
(out, buttons)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The confirmation half, in the same two passes.
|
||||||
|
fn confirm_frame(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
pw: &mut String,
|
||||||
|
wrong: bool,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (egui::FullOutput, (bool, bool)) {
|
||||||
|
let _ = ctx.run(raw_input(SCREEN, Vec::new()), |ctx| {
|
||||||
|
confirm_key_modal(ctx, pw, wrong);
|
||||||
|
});
|
||||||
|
let mut buttons = (false, false);
|
||||||
|
let out = ctx.run(raw_input(SCREEN, events), |ctx| {
|
||||||
|
buttons = confirm_key_modal(ctx, pw, wrong);
|
||||||
|
});
|
||||||
|
(out, buttons)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An empty field cannot submit: there is nothing to derive from, and a
|
||||||
|
/// dead button says so more clearly than a rejected attempt would.
|
||||||
|
#[test]
|
||||||
|
fn the_confirmation_will_not_submit_an_empty_password() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut pw = String::new();
|
||||||
|
let (out, buttons) = confirm_frame(&ctx, &mut pw, false, Vec::new());
|
||||||
|
assert_eq!(buttons, (false, false));
|
||||||
|
|
||||||
|
let pos = painted_text_center(&out, "Show key").expect("no submit button painted");
|
||||||
|
let (_, buttons) = confirm_frame(&ctx, &mut pw, false, click_at(pos));
|
||||||
|
assert!(!buttons.0, "an empty password was submitted");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_confirmation_submits_a_typed_password_and_cancels_on_request() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut pw = "hunter2".to_string();
|
||||||
|
let (out, _) = confirm_frame(&ctx, &mut pw, false, Vec::new());
|
||||||
|
assert!(
|
||||||
|
!painted_text(&out).contains(&pw),
|
||||||
|
"the password was painted in the clear: {:?}",
|
||||||
|
painted_text(&out)
|
||||||
|
);
|
||||||
|
|
||||||
|
let submit = painted_text_center(&out, "Show key").expect("no submit button painted");
|
||||||
|
let (_, buttons) = confirm_frame(&ctx, &mut pw, false, click_at(submit));
|
||||||
|
assert_eq!(buttons, (true, false));
|
||||||
|
|
||||||
|
let cancel = painted_text_center(&out, "Cancel").expect("no cancel button painted");
|
||||||
|
let (_, buttons) = confirm_frame(&ctx, &mut pw, false, click_at(cancel));
|
||||||
|
assert_eq!(buttons, (false, true));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A retry has to say why it is asking again, or it reads as the dialog
|
||||||
|
/// having ignored the first attempt.
|
||||||
|
#[test]
|
||||||
|
fn a_retry_says_the_password_was_wrong() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut pw = String::new();
|
||||||
|
let (quiet, _) = confirm_frame(&ctx, &mut pw, false, Vec::new());
|
||||||
|
assert!(
|
||||||
|
!painted_text(&quiet)
|
||||||
|
.iter()
|
||||||
|
.any(|t| t.contains("not correct")),
|
||||||
|
"the first attempt was called wrong before it was made"
|
||||||
|
);
|
||||||
|
|
||||||
|
let (out, _) = confirm_frame(&ctx, &mut pw, true, Vec::new());
|
||||||
|
assert!(
|
||||||
|
painted_text(&out)
|
||||||
|
.iter()
|
||||||
|
.any(|t| t.contains("That password is not correct")),
|
||||||
|
"{:?}",
|
||||||
|
painted_text(&out)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The right password derives the installed key, and the key is shown in the
|
||||||
|
/// `0x` form other SQLCipher tools take.
|
||||||
|
#[test]
|
||||||
|
fn the_matching_password_reveals_the_installed_key() {
|
||||||
|
assert_eq!(reveal_display(KEY, KEY), Some(format!("0x{KEY}")));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A wrong password derives some other key. Nothing about the real one may
|
||||||
|
/// leak from the attempt, so the caller gets no display string at all.
|
||||||
|
#[test]
|
||||||
|
fn a_password_that_derives_another_key_reveals_nothing() {
|
||||||
|
assert_eq!(reveal_display(KEY, OTHER), None);
|
||||||
|
assert_eq!(reveal_display(KEY, ""), None);
|
||||||
|
// A prefix must not pass: the whole key is compared, not the start of it.
|
||||||
|
assert_eq!(reveal_display(KEY, &KEY[..62]), None);
|
||||||
|
// Both sides come from `to_hex`, which is always lowercase, so an
|
||||||
|
// uppercase spelling is a mismatch rather than a value to normalise.
|
||||||
|
assert_eq!(reveal_display(KEY, &KEY.to_uppercase()), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_reveal_shows_the_key_and_what_holding_it_means() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let display = format!("0x{KEY}");
|
||||||
|
let painted = painted_text(&frame(&ctx, &display, Vec::new()).0);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
painted.contains(&display),
|
||||||
|
"the key itself is not on screen: {painted:?}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
painted
|
||||||
|
.iter()
|
||||||
|
.any(|t| t.contains("read the index without the password")),
|
||||||
|
"no warning about what the key is: {painted:?}"
|
||||||
|
);
|
||||||
|
assert!(painted.contains(&"Copy".to_string()), "{painted:?}");
|
||||||
|
assert!(painted.contains(&"Close".to_string()), "{painted:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn both_of_the_reveal_buttons_report_their_click() {
|
||||||
|
let display = format!("0x{KEY}");
|
||||||
|
for (label, expected) in [("Copy", (true, false)), ("Close", (false, true))] {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let (out, _) = frame(&ctx, &display, Vec::new());
|
||||||
|
let pos =
|
||||||
|
painted_text_center(&out, label).unwrap_or_else(|| panic!("no {label} button painted"));
|
||||||
|
let (_, buttons) = frame(&ctx, &display, click_at(pos));
|
||||||
|
assert_eq!(
|
||||||
|
buttons, expected,
|
||||||
|
"clicking {label} reported the wrong pair"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The displayed string is the whole key and nothing else: a truncated or
|
||||||
|
/// annotated form would be pasted into other tools and fail there.
|
||||||
|
#[test]
|
||||||
|
fn the_display_form_is_the_prefix_and_the_whole_key() {
|
||||||
|
let display = reveal_display(KEY, KEY).expect("a match reveals");
|
||||||
|
assert_eq!(display.len(), 66);
|
||||||
|
assert!(display.starts_with("0x"));
|
||||||
|
assert!(display[2..].bytes().all(|b| b.is_ascii_hexdigit()));
|
||||||
|
}
|
||||||
|
|
@ -261,7 +261,7 @@ mod tests {
|
||||||
walked,
|
walked,
|
||||||
walk_total,
|
walk_total,
|
||||||
extracted: 0,
|
extracted: 0,
|
||||||
extract_total: 0,
|
extract_total: None,
|
||||||
current_file: None,
|
current_file: None,
|
||||||
active_workers: 2,
|
active_workers: 2,
|
||||||
total_workers: 4,
|
total_workers: 4,
|
||||||
|
|
@ -299,11 +299,11 @@ mod tests {
|
||||||
|
|
||||||
let mut extracting = root(RootPhase::Extracting, 1_000, None);
|
let mut extracting = root(RootPhase::Extracting, 1_000, None);
|
||||||
extracting.extracted = 200;
|
extracting.extracted = 200;
|
||||||
extracting.extract_total = 800;
|
extracting.extract_total = Some(800);
|
||||||
extracting.active_workers = 3;
|
extracting.active_workers = 3;
|
||||||
let mut done = root(RootPhase::Done, 500, None);
|
let mut done = root(RootPhase::Done, 500, None);
|
||||||
done.extracted = 500;
|
done.extracted = 500;
|
||||||
done.extract_total = 500;
|
done.extract_total = Some(500);
|
||||||
done.active_workers = 0;
|
done.active_workers = 0;
|
||||||
done.total_workers = 0;
|
done.total_workers = 0;
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
|
|
|
||||||
|
|
@ -30,61 +30,96 @@ fn a_stale_draft_cannot_revert_the_indexing_mode_or_security() {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The guard decision table for leaving a tab, however it is asked for.
|
/// The guard decision table for leaving a tab, however it is asked for.
|
||||||
|
/// Both draft-backed tabs guard their own departure, and neither answers for
|
||||||
|
/// the other.
|
||||||
#[test]
|
#[test]
|
||||||
fn leaving_a_dirty_manage_tab_is_guarded_however_it_is_asked_for() {
|
fn leaving_a_dirty_editor_tab_is_guarded_however_it_is_asked_for() {
|
||||||
assert!(switch_needs_guard(Tab::Manage, true, false));
|
// (the tab, whether *its* editor is the dirty one in the pair below)
|
||||||
|
for (tab, manage_dirty, settings_dirty) in
|
||||||
|
[(Tab::Manage, true, false), (Tab::Settings, false, true)]
|
||||||
|
{
|
||||||
assert!(
|
assert!(
|
||||||
!switch_needs_guard(Tab::Manage, false, false),
|
switch_needs_guard(tab, manage_dirty, settings_dirty, false),
|
||||||
"a clean editor has nothing to ask about"
|
"{tab:?} must guard its own unapplied edits"
|
||||||
);
|
);
|
||||||
assert!(
|
assert!(
|
||||||
!switch_needs_guard(Tab::Manage, true, true),
|
!switch_needs_guard(tab, false, false, false),
|
||||||
"one held navigation at a time"
|
"{tab:?}: a clean editor has nothing to ask about"
|
||||||
);
|
);
|
||||||
|
assert!(
|
||||||
|
!switch_needs_guard(tab, manage_dirty, settings_dirty, true),
|
||||||
|
"{tab:?}: one held navigation at a time"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!switch_needs_guard(tab, !manage_dirty, !settings_dirty, false),
|
||||||
|
"{tab:?} must not answer for the other editor"
|
||||||
|
);
|
||||||
|
}
|
||||||
for tab in [Tab::Search, Tab::Duplicates, Tab::Logs, Tab::Help] {
|
for tab in [Tab::Search, Tab::Duplicates, Tab::Logs, Tab::Help] {
|
||||||
assert!(
|
assert!(
|
||||||
!switch_needs_guard(tab, true, false),
|
!switch_needs_guard(tab, true, true, false),
|
||||||
"{tab:?} holds no unapplied edits of its own"
|
"{tab:?} holds no unapplied edits of its own"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn guard_source_orders_quit_prompts_options_first() {
|
fn guard_source_orders_quit_prompts_settings_first() {
|
||||||
use super::NavIntent::*;
|
use super::NavIntent::*;
|
||||||
let tab = SwitchTab(Tab::Search);
|
let leave = SwitchTab(Tab::Search);
|
||||||
|
|
||||||
assert_eq!(guard_source(tab, true, true), Some(UnsavedSource::Manage));
|
// A switch asks about the tab being left, and only about that one.
|
||||||
assert_eq!(guard_source(tab, true, false), Some(UnsavedSource::Manage));
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
guard_source(tab, false, true),
|
guard_source(leave, Tab::Manage, true, true),
|
||||||
|
Some(UnsavedSource::Manage)
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guard_source(leave, Tab::Manage, true, false),
|
||||||
|
Some(UnsavedSource::Manage)
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guard_source(leave, Tab::Manage, false, true),
|
||||||
None,
|
None,
|
||||||
"options guard its own close"
|
"the Settings draft is not what leaving Manage disturbs"
|
||||||
);
|
|
||||||
assert_eq!(guard_source(tab, false, false), None);
|
|
||||||
|
|
||||||
assert_eq!(
|
|
||||||
guard_source(CloseOptions, true, true),
|
|
||||||
Some(UnsavedSource::Options)
|
|
||||||
);
|
);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
guard_source(CloseOptions, false, true),
|
guard_source(leave, Tab::Settings, true, true),
|
||||||
Some(UnsavedSource::Options)
|
Some(UnsavedSource::Settings)
|
||||||
);
|
);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
guard_source(CloseOptions, true, false),
|
guard_source(leave, Tab::Settings, false, true),
|
||||||
|
Some(UnsavedSource::Settings)
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guard_source(leave, Tab::Settings, true, false),
|
||||||
None,
|
None,
|
||||||
"manage guards tab switches"
|
"the Manage draft is not what leaving Settings disturbs"
|
||||||
);
|
);
|
||||||
assert_eq!(guard_source(CloseOptions, false, false), None);
|
for tab in [Tab::Search, Tab::Duplicates, Tab::Logs, Tab::Help] {
|
||||||
|
|
||||||
assert_eq!(guard_source(Quit, true, true), Some(UnsavedSource::Options));
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
guard_source(Quit, false, true),
|
guard_source(leave, tab, true, true),
|
||||||
Some(UnsavedSource::Options)
|
None,
|
||||||
|
"{tab:?} stages nothing, so leaving it asks nothing"
|
||||||
);
|
);
|
||||||
assert_eq!(guard_source(Quit, true, false), Some(UnsavedSource::Manage));
|
}
|
||||||
assert_eq!(guard_source(Quit, false, false), None);
|
|
||||||
|
// Quit asks about both, Settings first.
|
||||||
|
for from in [Tab::Search, Tab::Manage, Tab::Settings] {
|
||||||
|
assert_eq!(
|
||||||
|
guard_source(Quit, from, true, true),
|
||||||
|
Some(UnsavedSource::Settings),
|
||||||
|
"{from:?}: quitting asks about Settings before Manage"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guard_source(Quit, from, false, true),
|
||||||
|
Some(UnsavedSource::Settings)
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guard_source(Quit, from, true, false),
|
||||||
|
Some(UnsavedSource::Manage)
|
||||||
|
);
|
||||||
|
assert_eq!(guard_source(Quit, from, false, false), None);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Only a Quit during a running reconcile warns; a tab switch does not end
|
/// Only a Quit during a running reconcile warns; a tab switch does not end
|
||||||
|
|
@ -95,10 +130,25 @@ fn only_quitting_during_a_reconcile_warns() {
|
||||||
assert!(quit_needs_reconcile_warning(Quit, true));
|
assert!(quit_needs_reconcile_warning(Quit, true));
|
||||||
assert!(!quit_needs_reconcile_warning(Quit, false));
|
assert!(!quit_needs_reconcile_warning(Quit, false));
|
||||||
assert!(!quit_needs_reconcile_warning(SwitchTab(Tab::Search), true));
|
assert!(!quit_needs_reconcile_warning(SwitchTab(Tab::Search), true));
|
||||||
assert!(!quit_needs_reconcile_warning(CloseOptions, true));
|
assert!(!quit_needs_reconcile_warning(
|
||||||
|
SwitchTab(Tab::Settings),
|
||||||
|
true
|
||||||
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The two values the Options window writes, plus hand-edited variants.
|
/// Every tab is placed on exactly one side of the guard, so a tab added
|
||||||
|
/// later cannot quietly inherit "stages nothing".
|
||||||
|
#[test]
|
||||||
|
fn only_the_two_draft_backed_tabs_have_an_editor() {
|
||||||
|
assert_eq!(tab_editor(Tab::Manage), Some(UnsavedSource::Manage));
|
||||||
|
assert_eq!(tab_editor(Tab::Settings), Some(UnsavedSource::Settings));
|
||||||
|
for tab in [Tab::Search, Tab::Duplicates, Tab::Logs, Tab::Help] {
|
||||||
|
assert_eq!(tab_editor(tab), None, "{tab:?} saves as it goes");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The two values the Settings tab's color-scheme box writes, plus
|
||||||
|
/// hand-edited variants.
|
||||||
#[test]
|
#[test]
|
||||||
fn only_light_is_light() {
|
fn only_light_is_light() {
|
||||||
assert_eq!(theme_for("light"), egui::Theme::Light);
|
assert_eq!(theme_for("light"), egui::Theme::Light);
|
||||||
|
|
|
||||||
239
crates/quicksearch-gui/src/app/verify.rs
Normal file
239
crates/quicksearch-gui/src/app/verify.rs
Normal file
|
|
@ -0,0 +1,239 @@
|
||||||
|
//! Byte-for-byte verification of one duplicate group, and the modal that
|
||||||
|
//! reports it.
|
||||||
|
//!
|
||||||
|
//! The Duplicates tab groups by a hash of each file's size and head, which is
|
||||||
|
//! all the indexer ever reads (see [`quicksearch_core::verify`]). This is the
|
||||||
|
//! second opinion, asked for one group at a time, and it exists because the
|
||||||
|
//! action it precedes is usually deletion.
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
use quicksearch_core::verify::{MemberVerdict, VerifyReport, VerifyUpdate};
|
||||||
|
|
||||||
|
use crate::format::{group_thousands, human_size};
|
||||||
|
use crate::ui_util::{centered_modal, hint, progress_bar};
|
||||||
|
|
||||||
|
const MODAL_WIDTH: f32 = 560.0;
|
||||||
|
|
||||||
|
pub(crate) enum VerifyState {
|
||||||
|
Running {
|
||||||
|
bytes_read: u64,
|
||||||
|
/// Zero until the worker's first progress update lands, which is what
|
||||||
|
/// puts the bar in its indeterminate state to begin with.
|
||||||
|
bytes_total: u64,
|
||||||
|
},
|
||||||
|
Done(Box<VerifyReport>),
|
||||||
|
Cancelled,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) struct VerifyModal {
|
||||||
|
pub paths: Vec<PathBuf>,
|
||||||
|
pub state: VerifyState,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl VerifyModal {
|
||||||
|
pub(crate) fn new(paths: Vec<PathBuf>) -> VerifyModal {
|
||||||
|
VerifyModal {
|
||||||
|
paths,
|
||||||
|
state: VerifyState::Running {
|
||||||
|
bytes_read: 0,
|
||||||
|
bytes_total: 0,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One line of the report: what happened to `path`, in the words the modal
|
||||||
|
/// paints. Split out from the rendering so the wording is testable without a
|
||||||
|
/// frame.
|
||||||
|
pub(crate) fn verdict_line(verdict: &MemberVerdict, reference: bool) -> String {
|
||||||
|
match verdict {
|
||||||
|
MemberVerdict::Identical if reference => "compared against".to_string(),
|
||||||
|
MemberVerdict::Identical => "identical".to_string(),
|
||||||
|
MemberVerdict::DiffersAt(offset) => {
|
||||||
|
format!("differs at byte {}", group_thousands(*offset))
|
||||||
|
}
|
||||||
|
MemberVerdict::LengthDiffers { len, reference_len } => format!(
|
||||||
|
"size differs: {} against {}",
|
||||||
|
human_size(*len),
|
||||||
|
human_size(*reference_len)
|
||||||
|
),
|
||||||
|
MemberVerdict::Unreadable(e) => format!("could not be read — {e}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The headline the report earns.
|
||||||
|
pub(crate) fn summary_line(report: &VerifyReport) -> String {
|
||||||
|
let total = report.verdicts.len();
|
||||||
|
if report.reference.is_none() {
|
||||||
|
return "None of these files could be read.".to_string();
|
||||||
|
}
|
||||||
|
let differing = report.differing();
|
||||||
|
if differing == 0 {
|
||||||
|
return match total {
|
||||||
|
0 | 1 => "Nothing to compare: the group holds one file.".to_string(),
|
||||||
|
n => format!("All {n} files are byte-for-byte identical."),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
format!(
|
||||||
|
"{} of {} files {} not identical.",
|
||||||
|
differing,
|
||||||
|
total,
|
||||||
|
if differing == 1 { "is" } else { "are" }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
impl QuickSearchApp {
|
||||||
|
/// Drain the worker and fold its updates into the modal.
|
||||||
|
pub(super) fn drain_verify(&mut self) {
|
||||||
|
use std::sync::mpsc::TryRecvError;
|
||||||
|
let Some(job) = &self.backend.verify_job else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let mut finished = false;
|
||||||
|
loop {
|
||||||
|
match job.rx.try_recv() {
|
||||||
|
Ok(VerifyUpdate::Progress {
|
||||||
|
bytes_read: read,
|
||||||
|
bytes_total: total,
|
||||||
|
}) => {
|
||||||
|
if let Some(modal) = &mut self.verify {
|
||||||
|
modal.state = VerifyState::Running {
|
||||||
|
bytes_read: read,
|
||||||
|
bytes_total: total,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(VerifyUpdate::Done(report)) => {
|
||||||
|
if let Some(modal) = &mut self.verify {
|
||||||
|
modal.state = VerifyState::Done(Box::new(report));
|
||||||
|
}
|
||||||
|
finished = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
Ok(VerifyUpdate::Cancelled) => {
|
||||||
|
if let Some(modal) = &mut self.verify {
|
||||||
|
modal.state = VerifyState::Cancelled;
|
||||||
|
}
|
||||||
|
finished = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
Err(TryRecvError::Empty) => break,
|
||||||
|
// The worker died without a terminal update. Nothing else can
|
||||||
|
// arrive, so say so rather than spinning on an empty channel.
|
||||||
|
Err(TryRecvError::Disconnected) => {
|
||||||
|
if let Some(modal) = &mut self.verify {
|
||||||
|
if matches!(modal.state, VerifyState::Running { .. }) {
|
||||||
|
modal.state = VerifyState::Cancelled;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
finished = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if finished {
|
||||||
|
self.backend.verify_job = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(super) fn verify_modal_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(modal) = &self.verify else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if verify_modal(ctx, modal) {
|
||||||
|
self.backend.cancel_verify();
|
||||||
|
self.verify = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Paint the modal; `true` when its dismiss button was clicked. A free
|
||||||
|
/// function rather than a method so it can be rendered against a bare
|
||||||
|
/// context, without an app and the index behind it.
|
||||||
|
pub(crate) fn verify_modal(ctx: &egui::Context, modal: &VerifyModal) -> bool {
|
||||||
|
centered_modal(ctx, "Verify duplicates", |ui| {
|
||||||
|
ui.set_max_width(MODAL_WIDTH);
|
||||||
|
match &modal.state {
|
||||||
|
VerifyState::Running {
|
||||||
|
bytes_read,
|
||||||
|
bytes_total,
|
||||||
|
} => {
|
||||||
|
ui.label(format!(
|
||||||
|
"Comparing {} files byte for byte…",
|
||||||
|
modal.paths.len()
|
||||||
|
));
|
||||||
|
// No denominator until the first update lands, which is what
|
||||||
|
// the indeterminate bar is for.
|
||||||
|
let fraction =
|
||||||
|
(*bytes_total > 0).then(|| (*bytes_read as f64 / *bytes_total as f64) as f32);
|
||||||
|
progress_bar(ui, fraction, MODAL_WIDTH);
|
||||||
|
ui.label(hint(match bytes_total {
|
||||||
|
0 => format!("{} read", human_size(*bytes_read)),
|
||||||
|
total => format!("{} of {}", human_size(*bytes_read), human_size(*total)),
|
||||||
|
}));
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.horizontal(|ui| ui.button("Cancel").clicked()).inner
|
||||||
|
}
|
||||||
|
VerifyState::Cancelled => {
|
||||||
|
ui.label("Verification cancelled.");
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.horizontal(|ui| ui.button("Close").clicked()).inner
|
||||||
|
}
|
||||||
|
VerifyState::Done(report) => {
|
||||||
|
let p = crate::color::palette(ui.visuals().dark_mode);
|
||||||
|
let identical = report.all_identical() && report.reference.is_some();
|
||||||
|
let color = if identical {
|
||||||
|
p.green
|
||||||
|
} else {
|
||||||
|
ui.visuals().error_fg_color
|
||||||
|
};
|
||||||
|
ui.colored_label(color, summary_line(report));
|
||||||
|
if !identical {
|
||||||
|
ui.label(hint(
|
||||||
|
"Files are grouped by size and how they begin, which is all \
|
||||||
|
indexing reads. This compared every byte.",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
ui.add_space(6.0);
|
||||||
|
// Listed even when everything matched: it is the record of
|
||||||
|
// what was actually read.
|
||||||
|
egui::ScrollArea::vertical()
|
||||||
|
.max_height(260.0)
|
||||||
|
.auto_shrink([false, true])
|
||||||
|
.show(ui, |ui| {
|
||||||
|
for (i, path) in modal.paths.iter().enumerate() {
|
||||||
|
let Some(verdict) = report.verdicts.get(i) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let is_reference = report.reference == Some(i);
|
||||||
|
ui.horizontal_wrapped(|ui| {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(path.display().to_string()).monospace(),
|
||||||
|
);
|
||||||
|
let line = verdict_line(verdict, is_reference);
|
||||||
|
if verdict.is_identical() {
|
||||||
|
ui.label(hint(line));
|
||||||
|
} else {
|
||||||
|
ui.colored_label(
|
||||||
|
ui.visuals().error_fg_color,
|
||||||
|
egui::RichText::new(line).small(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.label(hint(format!("{} read", human_size(report.bytes_read))));
|
||||||
|
ui.horizontal(|ui| ui.button("Close").clicked()).inner
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.unwrap_or(false)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
#[path = "verify_tests.rs"]
|
||||||
|
mod tests;
|
||||||
216
crates/quicksearch-gui/src/app/verify_tests.rs
Normal file
216
crates/quicksearch-gui/src/app/verify_tests.rs
Normal file
|
|
@ -0,0 +1,216 @@
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
use quicksearch_core::verify::MemberVerdict::{
|
||||||
|
DiffersAt, Identical, LengthDiffers, Unreadable as CannotRead,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::test_ui::{click_at, painted_text, painted_text_center, raw_input};
|
||||||
|
|
||||||
|
const SCREEN: egui::Vec2 = egui::vec2(1000.0, 700.0);
|
||||||
|
|
||||||
|
fn modal(state: VerifyState, n: usize) -> VerifyModal {
|
||||||
|
VerifyModal {
|
||||||
|
paths: (0..n)
|
||||||
|
.map(|i| PathBuf::from(format!("/d/copy{i}.bin")))
|
||||||
|
.collect(),
|
||||||
|
state,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn report(verdicts: Vec<MemberVerdict>, bytes_read: u64) -> VerifyState {
|
||||||
|
VerifyState::Done(Box::new(VerifyReport {
|
||||||
|
reference: Some(0),
|
||||||
|
verdicts,
|
||||||
|
bytes_read,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two passes: an `egui::Window` is measured on its first frame and placed on
|
||||||
|
/// the next, so a single pass paints nothing to read back.
|
||||||
|
fn frame(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
m: &VerifyModal,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (egui::FullOutput, bool) {
|
||||||
|
let _ = ctx.run(raw_input(SCREEN, Vec::new()), |ctx| {
|
||||||
|
verify_modal(ctx, m);
|
||||||
|
});
|
||||||
|
let mut closed = false;
|
||||||
|
let out = ctx.run(raw_input(SCREEN, events), |ctx| {
|
||||||
|
closed = verify_modal(ctx, m);
|
||||||
|
});
|
||||||
|
(out, closed)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_run_in_progress_says_what_it_is_doing_and_offers_a_way_out() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let m = modal(
|
||||||
|
VerifyState::Running {
|
||||||
|
bytes_read: 5 * 1024 * 1024,
|
||||||
|
bytes_total: 20 * 1024 * 1024,
|
||||||
|
},
|
||||||
|
3,
|
||||||
|
);
|
||||||
|
let painted = painted_text(&frame(&ctx, &m, Vec::new()).0);
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"Comparing 3 files byte for byte…".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
painted.iter().any(|t| t.contains("5.2 MB of 21.0 MB")),
|
||||||
|
"no byte counter: {painted:?}"
|
||||||
|
);
|
||||||
|
assert!(painted.contains(&"Cancel".to_string()), "{painted:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Before the worker's first update there is no denominator, so the modal
|
||||||
|
/// reports what it has rather than dividing by zero.
|
||||||
|
#[test]
|
||||||
|
fn a_run_with_no_denominator_yet_still_reports() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let m = modal(
|
||||||
|
VerifyState::Running {
|
||||||
|
bytes_read: 0,
|
||||||
|
bytes_total: 0,
|
||||||
|
},
|
||||||
|
2,
|
||||||
|
);
|
||||||
|
let painted = painted_text(&frame(&ctx, &m, Vec::new()).0);
|
||||||
|
assert!(
|
||||||
|
painted.iter().any(|t| t.contains("0 B read")),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_clean_result_says_so_and_lists_what_was_read() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let m = modal(report(vec![Identical, Identical, Identical], 300), 3);
|
||||||
|
let painted = painted_text(&frame(&ctx, &m, Vec::new()).0);
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"All 3 files are byte-for-byte identical.".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
assert!(painted.contains(&"/d/copy2.bin".to_string()), "{painted:?}");
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"compared against".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
assert!(painted.contains(&"Close".to_string()), "{painted:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The case the feature exists for: same size, same head, different bytes.
|
||||||
|
#[test]
|
||||||
|
fn a_mismatch_names_the_file_and_the_offset() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let m = modal(report(vec![Identical, DiffersAt(1_234_567)], 2), 2);
|
||||||
|
let painted = painted_text(&frame(&ctx, &m, Vec::new()).0);
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"1 of 2 files is not identical.".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"differs at byte 1,234,567".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
assert!(painted.contains(&"/d/copy1.bin".to_string()), "{painted:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_cancelled_run_says_so_rather_than_showing_a_verdict() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let m = modal(VerifyState::Cancelled, 2);
|
||||||
|
let painted = painted_text(&frame(&ctx, &m, Vec::new()).0);
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"Verification cancelled.".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!painted.iter().any(|t| t.contains("identical")),
|
||||||
|
"a cancelled run claimed a verdict: {painted:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every state's dismiss button reports the dismissal, whatever it is called.
|
||||||
|
#[test]
|
||||||
|
fn both_dismiss_buttons_report_the_dismissal() {
|
||||||
|
for (label, state) in [
|
||||||
|
(
|
||||||
|
"Cancel",
|
||||||
|
VerifyState::Running {
|
||||||
|
bytes_read: 1,
|
||||||
|
bytes_total: 2,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
("Close", VerifyState::Cancelled),
|
||||||
|
("Close", report(vec![Identical, Identical], 8)),
|
||||||
|
] {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let m = modal(state, 2);
|
||||||
|
let (out, _) = frame(&ctx, &m, Vec::new());
|
||||||
|
let pos =
|
||||||
|
painted_text_center(&out, label).unwrap_or_else(|| panic!("no {label} button painted"));
|
||||||
|
let (_, closed) = frame(&ctx, &m, click_at(pos));
|
||||||
|
assert!(closed, "clicking {label} did not dismiss the modal");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_verdict_reads_as_a_sentence_about_the_file() {
|
||||||
|
assert_eq!(verdict_line(&Identical, false), "identical");
|
||||||
|
assert_eq!(verdict_line(&Identical, true), "compared against");
|
||||||
|
assert_eq!(verdict_line(&DiffersAt(0), false), "differs at byte 0");
|
||||||
|
assert_eq!(
|
||||||
|
verdict_line(&DiffersAt(1_048_576), false),
|
||||||
|
"differs at byte 1,048,576"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
verdict_line(
|
||||||
|
&LengthDiffers {
|
||||||
|
len: 2048,
|
||||||
|
reference_len: 1024
|
||||||
|
},
|
||||||
|
false
|
||||||
|
),
|
||||||
|
"size differs: 2.0 KB against 1.0 KB"
|
||||||
|
);
|
||||||
|
assert!(verdict_line(&CannotRead("/d/x: denied".into()), false)
|
||||||
|
.contains("could not be read — /d/x: denied"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_summary_counts_what_it_found() {
|
||||||
|
let of = |verdicts: Vec<MemberVerdict>, reference| {
|
||||||
|
summary_line(&VerifyReport {
|
||||||
|
reference,
|
||||||
|
verdicts,
|
||||||
|
bytes_read: 0,
|
||||||
|
})
|
||||||
|
};
|
||||||
|
assert_eq!(
|
||||||
|
of(vec![Identical, Identical], Some(0)),
|
||||||
|
"All 2 files are byte-for-byte identical."
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
of(vec![Identical, DiffersAt(4)], Some(0)),
|
||||||
|
"1 of 2 files is not identical."
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
of(vec![Identical, DiffersAt(4), DiffersAt(9)], Some(0)),
|
||||||
|
"2 of 3 files are not identical."
|
||||||
|
);
|
||||||
|
// A group of one cannot disagree with itself, and saying "all 1 files are
|
||||||
|
// identical" would read as an answer to a question nobody asked.
|
||||||
|
assert_eq!(
|
||||||
|
of(vec![Identical], Some(0)),
|
||||||
|
"Nothing to compare: the group holds one file."
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
of(
|
||||||
|
vec![CannotRead("gone".into()), CannotRead("gone".into())],
|
||||||
|
None
|
||||||
|
),
|
||||||
|
"None of these files could be read."
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
@ -6,22 +6,48 @@
|
||||||
//! Every core thread wakes the UI through `ctx.request_repaint()`, which is
|
//! Every core thread wakes the UI through `ctx.request_repaint()`, which is
|
||||||
//! what makes polling enough.
|
//! what makes polling enough.
|
||||||
//!
|
//!
|
||||||
//! The duplicates scan is the only throwaway thread, and it fires on a user
|
//! The duplicates scan and the byte-for-byte verification of one of its
|
||||||
//! action, not a timer: a thread per refresh opens its own connection — a
|
//! groups are the throwaway threads, and both fire on a user action rather
|
||||||
//! page cache and an allocator arena glibc never gives back.
|
//! than a timer: a thread per refresh opens its own connection — a page cache
|
||||||
|
//! and an allocator arena glibc never gives back. The verification opens no
|
||||||
|
//! connection at all, but it can hold a large group's worth of file handles,
|
||||||
|
//! so it carries a cancel flag and shutdown raises it.
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
use std::sync::{mpsc, Arc};
|
use std::sync::{mpsc, Arc};
|
||||||
|
|
||||||
use quicksearch_core::config::Config;
|
use quicksearch_core::config::Config;
|
||||||
use quicksearch_core::coordinator::IndexCoordinator;
|
use quicksearch_core::coordinator::IndexCoordinator;
|
||||||
|
use quicksearch_core::live::{LiveUpdate, LiveWatcher};
|
||||||
use quicksearch_core::search::{DuplicateGroup, SearchService, SearchUpdate};
|
use quicksearch_core::search::{DuplicateGroup, SearchService, SearchUpdate};
|
||||||
use quicksearch_core::shutdown;
|
use quicksearch_core::shutdown;
|
||||||
|
use quicksearch_core::verify::{verify_identical, VerifyUpdate};
|
||||||
|
|
||||||
|
/// A duplicate group being read through. The thread is detached and owns
|
||||||
|
/// nothing the app needs back, so cancelling is just raising the flag: the
|
||||||
|
/// worker notices between chunks and drops the receiver's other end.
|
||||||
|
pub struct VerifyJob {
|
||||||
|
pub rx: mpsc::Receiver<VerifyUpdate>,
|
||||||
|
cancel: Arc<AtomicBool>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl VerifyJob {
|
||||||
|
pub fn cancel(&self) {
|
||||||
|
self.cancel.store(true, Ordering::Relaxed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct Backend {
|
pub struct Backend {
|
||||||
pub coordinator: Arc<IndexCoordinator>,
|
pub coordinator: Arc<IndexCoordinator>,
|
||||||
pub search: Option<SearchService>,
|
pub search: Option<SearchService>,
|
||||||
pub search_rx: mpsc::Receiver<SearchUpdate>,
|
pub search_rx: mpsc::Receiver<SearchUpdate>,
|
||||||
pub dup_job: Option<mpsc::Receiver<Result<Vec<DuplicateGroup>, String>>>,
|
pub dup_job: Option<mpsc::Receiver<Result<Vec<DuplicateGroup>, String>>>,
|
||||||
|
pub verify_job: Option<VerifyJob>,
|
||||||
|
/// Watches the results currently on screen; see [`quicksearch_core::live`].
|
||||||
|
/// `None` only after [`Backend::shutdown`].
|
||||||
|
pub live: Option<LiveWatcher>,
|
||||||
|
pub live_rx: mpsc::Receiver<LiveUpdate>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Backend {
|
impl Backend {
|
||||||
|
|
@ -44,14 +70,49 @@ impl Backend {
|
||||||
Arc::new(move || repaint_ctx.request_repaint()),
|
Arc::new(move || repaint_ctx.request_repaint()),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
let live_ctx = ctx.clone();
|
||||||
|
let (live, live_rx) = LiveWatcher::start(Arc::new(move || live_ctx.request_repaint()));
|
||||||
|
|
||||||
Ok(Backend {
|
Ok(Backend {
|
||||||
coordinator,
|
coordinator,
|
||||||
search: Some(search),
|
search: Some(search),
|
||||||
search_rx,
|
search_rx,
|
||||||
dup_job: None,
|
dup_job: None,
|
||||||
|
verify_job: None,
|
||||||
|
live: Some(live),
|
||||||
|
live_rx,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Point the live watcher at the rows currently on screen, or clear it
|
||||||
|
/// with an empty `targets`.
|
||||||
|
pub fn watch_live(
|
||||||
|
&self,
|
||||||
|
query: &str,
|
||||||
|
targets: Vec<quicksearch_core::live::Target>,
|
||||||
|
config: &Config,
|
||||||
|
) {
|
||||||
|
let Some(live) = &self.live else { return };
|
||||||
|
if targets.is_empty() {
|
||||||
|
live.clear();
|
||||||
|
} else {
|
||||||
|
live.watch(query, targets, config);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Ask the coordinator to bring the index in line with these paths — the
|
||||||
|
/// files the live watcher has just read from disk on the frontend's
|
||||||
|
/// behalf, so the index does not drift from what is on screen.
|
||||||
|
pub fn reindex_live_paths(&self, paths: Vec<PathBuf>) {
|
||||||
|
self.coordinator.update_paths(paths);
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn clear_live(&self) {
|
||||||
|
if let Some(live) = &self.live {
|
||||||
|
live.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// `None` only after [`Backend::shutdown`], i.e. during teardown frames.
|
/// `None` only after [`Backend::shutdown`], i.e. during teardown frames.
|
||||||
pub fn search(&self) -> Option<&SearchService> {
|
pub fn search(&self) -> Option<&SearchService> {
|
||||||
self.search.as_ref()
|
self.search.as_ref()
|
||||||
|
|
@ -70,12 +131,46 @@ impl Backend {
|
||||||
self.dup_job = Some(rx);
|
self.dup_job = Some(rx);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Read a duplicate group through on a worker thread, comparing every
|
||||||
|
/// member against the first byte for byte. Replaces any run already going.
|
||||||
|
pub fn start_verify(&mut self, paths: Vec<PathBuf>, ctx: egui::Context) {
|
||||||
|
if let Some(job) = &self.verify_job {
|
||||||
|
job.cancel();
|
||||||
|
}
|
||||||
|
let (tx, rx) = mpsc::channel();
|
||||||
|
let cancel = Arc::new(AtomicBool::new(false));
|
||||||
|
let worker_cancel = cancel.clone();
|
||||||
|
std::thread::spawn(move || {
|
||||||
|
verify_identical(&paths, &worker_cancel, &mut |update| {
|
||||||
|
// A closed receiver means the app moved on; the cancel flag
|
||||||
|
// is what stops the work, so there is nothing to do here.
|
||||||
|
let _ = tx.send(update);
|
||||||
|
ctx.request_repaint();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
self.verify_job = Some(VerifyJob { rx, cancel });
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stop a verification and forget it. The worker sees the flag between
|
||||||
|
/// chunks and exits on its own.
|
||||||
|
pub fn cancel_verify(&mut self) {
|
||||||
|
if let Some(job) = self.verify_job.take() {
|
||||||
|
job.cancel();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Join the search worker and stop the coordinator. Called once from
|
/// Join the search worker and stop the coordinator. Called once from
|
||||||
/// `on_exit`.
|
/// `on_exit`.
|
||||||
pub fn shutdown(&mut self) {
|
pub fn shutdown(&mut self) {
|
||||||
|
// Detached and holding open file handles: the flag is what makes a
|
||||||
|
// verification of a slow, large group let go on the way out.
|
||||||
|
self.cancel_verify();
|
||||||
if let Some(search) = self.search.take() {
|
if let Some(search) = self.search.take() {
|
||||||
search.shutdown();
|
search.shutdown();
|
||||||
}
|
}
|
||||||
|
if let Some(mut live) = self.live.take() {
|
||||||
|
live.stop();
|
||||||
|
}
|
||||||
self.coordinator.shutdown();
|
self.coordinator.shutdown();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -82,7 +82,7 @@ pub(crate) struct CaptureDriver {
|
||||||
/// Screenshot in flight: requested, PNG not yet written.
|
/// Screenshot in flight: requested, PNG not yet written.
|
||||||
shot: Option<PathBuf>,
|
shot: Option<PathBuf>,
|
||||||
rec: Option<Recorder>,
|
rec: Option<Recorder>,
|
||||||
/// Match-cell row the pointer is pinned to (`hover_match`), and the
|
/// Content Match cell row the pointer is pinned to (`hover_match`), and the
|
||||||
/// on-screen position it resolved to on the last rendered frame.
|
/// on-screen position it resolved to on the last rendered frame.
|
||||||
hover: Option<usize>,
|
hover: Option<usize>,
|
||||||
hover_pos: Option<egui::Pos2>,
|
hover_pos: Option<egui::Pos2>,
|
||||||
|
|
|
||||||
|
|
@ -11,9 +11,11 @@ use crate::app::Tab;
|
||||||
/// window INT INT # resize to width x height, in the same
|
/// window INT INT # resize to width x height, in the same
|
||||||
/// # logical points as the startup size
|
/// # logical points as the startup size
|
||||||
/// hover_match INT # pin the pointer over the Nth visible
|
/// hover_match INT # pin the pointer over the Nth visible
|
||||||
/// # Match cell (0-based) until hover_off
|
/// # Content Match cell (0-based) until
|
||||||
|
/// # hover_off. Counts every visible row,
|
||||||
|
/// # including those showing a dash.
|
||||||
/// hover_off # release the pinned pointer
|
/// hover_off # release the pinned pointer
|
||||||
/// tab (search|manage|duplicates|logs|help)
|
/// tab (search|manage|duplicates|logs|help|settings)
|
||||||
/// wait_index_running [max INT] # caps in ms; a capped wait cannot fail
|
/// wait_index_running [max INT] # caps in ms; a capped wait cannot fail
|
||||||
/// wait_index_idle [max INT]
|
/// wait_index_idle [max INT]
|
||||||
/// wait_search_done [max INT]
|
/// wait_search_done [max INT]
|
||||||
|
|
@ -187,9 +189,11 @@ fn parse_line(tokens: &[Token], line_no: usize) -> Result<Option<Cmd>, ParseErro
|
||||||
"duplicates" => Tab::Duplicates,
|
"duplicates" => Tab::Duplicates,
|
||||||
"logs" => Tab::Logs,
|
"logs" => Tab::Logs,
|
||||||
"help" => Tab::Help,
|
"help" => Tab::Help,
|
||||||
|
"settings" => Tab::Settings,
|
||||||
other => {
|
other => {
|
||||||
return Err(err(format!(
|
return Err(err(format!(
|
||||||
"unknown tab {other:?}: expected search, manage, duplicates, logs or help"
|
"unknown tab {other:?}: expected search, manage, duplicates, \
|
||||||
|
logs, help or settings"
|
||||||
)));
|
)));
|
||||||
}
|
}
|
||||||
}),
|
}),
|
||||||
|
|
@ -302,6 +306,7 @@ mod tests {
|
||||||
tab duplicates
|
tab duplicates
|
||||||
tab logs
|
tab logs
|
||||||
tab help
|
tab help
|
||||||
|
tab settings
|
||||||
wait_index_running
|
wait_index_running
|
||||||
wait_index_running max 15000
|
wait_index_running max 15000
|
||||||
wait_index_idle max 13000
|
wait_index_idle max 13000
|
||||||
|
|
@ -335,6 +340,7 @@ mod tests {
|
||||||
Cmd::Tab(Tab::Duplicates),
|
Cmd::Tab(Tab::Duplicates),
|
||||||
Cmd::Tab(Tab::Logs),
|
Cmd::Tab(Tab::Logs),
|
||||||
Cmd::Tab(Tab::Help),
|
Cmd::Tab(Tab::Help),
|
||||||
|
Cmd::Tab(Tab::Settings),
|
||||||
Cmd::WaitIndexRunning { max_ms: None },
|
Cmd::WaitIndexRunning { max_ms: None },
|
||||||
Cmd::WaitIndexRunning {
|
Cmd::WaitIndexRunning {
|
||||||
max_ms: Some(15000)
|
max_ms: Some(15000)
|
||||||
|
|
@ -447,7 +453,7 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn unknown_tab_and_bad_cps_are_rejected() {
|
fn unknown_tab_and_bad_cps_are_rejected() {
|
||||||
assert!(parse_err("tab settings").msg.contains("unknown tab"));
|
assert!(parse_err("tab preferences").msg.contains("unknown tab"));
|
||||||
assert!(parse_err(r#"type "x" cps 0"#).msg.contains("positive"));
|
assert!(parse_err(r#"type "x" cps 0"#).msg.contains("positive"));
|
||||||
assert!(parse_err(r#"type "x" cps -3"#).msg.contains("positive"));
|
assert!(parse_err(r#"type "x" cps -3"#).msg.contains("positive"));
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -21,8 +21,18 @@ pub struct DuplicatesTab {
|
||||||
#[derive(Default)]
|
#[derive(Default)]
|
||||||
pub struct DuplicatesActions {
|
pub struct DuplicatesActions {
|
||||||
pub refresh: bool,
|
pub refresh: bool,
|
||||||
|
/// Every member of one group, to be read through and compared byte for
|
||||||
|
/// byte. Group-scoped whichever row it was asked for from: the question
|
||||||
|
/// "is this row really a duplicate" is a question about the group.
|
||||||
|
pub verify: Option<Vec<String>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The entry both context menus carry. Named for what it settles, since the
|
||||||
|
/// grouping itself never claimed more than a shared size and head.
|
||||||
|
const VERIFY_LABEL: &str = "Verify copies are identical…";
|
||||||
|
const VERIFY_TIP: &str = "Reads every file in the group in full and compares them byte for \
|
||||||
|
byte. Grouping only reads each file's size and how it begins.";
|
||||||
|
|
||||||
impl DuplicatesTab {
|
impl DuplicatesTab {
|
||||||
pub fn new() -> DuplicatesTab {
|
pub fn new() -> DuplicatesTab {
|
||||||
DuplicatesTab {
|
DuplicatesTab {
|
||||||
|
|
@ -30,7 +40,10 @@ impl DuplicatesTab {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn ui(&mut self, ui: &mut egui::Ui) -> DuplicatesActions {
|
/// `verify_open` is the verification window being up — running or showing
|
||||||
|
/// a result. There is one of it, so the entry greys out rather than
|
||||||
|
/// replacing what someone is reading.
|
||||||
|
pub fn ui(&mut self, ui: &mut egui::Ui, verify_open: bool) -> DuplicatesActions {
|
||||||
let mut actions = DuplicatesActions::default();
|
let mut actions = DuplicatesActions::default();
|
||||||
|
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
|
|
@ -84,6 +97,7 @@ impl DuplicatesTab {
|
||||||
human_size(group.redundant_size.max(0) as u64),
|
human_size(group.redundant_size.max(0) as u64),
|
||||||
human_size(group.total_size.max(0) as u64),
|
human_size(group.total_size.max(0) as u64),
|
||||||
);
|
);
|
||||||
|
let header =
|
||||||
egui::CollapsingHeader::new(title)
|
egui::CollapsingHeader::new(title)
|
||||||
.id_salt(i)
|
.id_salt(i)
|
||||||
.show(ui, |ui| {
|
.show(ui, |ui| {
|
||||||
|
|
@ -100,18 +114,31 @@ impl DuplicatesTab {
|
||||||
platform::open_file(path);
|
platform::open_file(path);
|
||||||
}
|
}
|
||||||
response.context_menu(|ui| {
|
response.context_menu(|ui| {
|
||||||
if ui.button("Open").clicked() {
|
if ui.button("Open File").clicked() {
|
||||||
platform::open_file(path);
|
platform::open_file(path);
|
||||||
ui.close();
|
ui.close();
|
||||||
}
|
}
|
||||||
if ui.button("Open containing folder").clicked() {
|
if ui.button("Open containing folder").clicked()
|
||||||
|
{
|
||||||
platform::reveal_in_folder(path);
|
platform::reveal_in_folder(path);
|
||||||
ui.close();
|
ui.close();
|
||||||
}
|
}
|
||||||
|
ui.separator();
|
||||||
|
if verify_entry(ui, verify_open) {
|
||||||
|
actions.verify = Some(member_paths(group));
|
||||||
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
// Also on the group's own row: the question is
|
||||||
|
// about the group, and the rows it is about are
|
||||||
|
// behind a collapsed header until they are not.
|
||||||
|
header.header_response.context_menu(|ui| {
|
||||||
|
if verify_entry(ui, verify_open) {
|
||||||
|
actions.verify = Some(member_paths(group));
|
||||||
|
}
|
||||||
|
});
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
crate::ui_util::more_below_hint(ui, &scroll);
|
crate::ui_util::more_below_hint(ui, &scroll);
|
||||||
|
|
@ -120,3 +147,24 @@ impl DuplicatesTab {
|
||||||
actions
|
actions
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The shared menu entry. Returns whether it was clicked, and closes the menu
|
||||||
|
/// when it was.
|
||||||
|
fn verify_entry(ui: &mut egui::Ui, open: bool) -> bool {
|
||||||
|
let clicked = ui
|
||||||
|
.add_enabled(!open, egui::Button::new(VERIFY_LABEL))
|
||||||
|
.on_hover_text(VERIFY_TIP)
|
||||||
|
.on_disabled_hover_text("Close the verification window first.")
|
||||||
|
.clicked();
|
||||||
|
if clicked {
|
||||||
|
ui.close();
|
||||||
|
}
|
||||||
|
clicked
|
||||||
|
}
|
||||||
|
|
||||||
|
fn member_paths(group: &DuplicateGroup) -> Vec<String> {
|
||||||
|
group.members.iter().map(|m| m.2.clone()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests;
|
||||||
|
|
|
||||||
186
crates/quicksearch-gui/src/duplicates_tab/tests.rs
Normal file
186
crates/quicksearch-gui/src/duplicates_tab/tests.rs
Normal file
|
|
@ -0,0 +1,186 @@
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
use crate::test_ui::{painted_text, painted_text_center, raw_input};
|
||||||
|
|
||||||
|
const SCREEN: egui::Vec2 = egui::vec2(1000.0, 700.0);
|
||||||
|
|
||||||
|
fn group(paths: &[&str]) -> DuplicateGroup {
|
||||||
|
DuplicateGroup {
|
||||||
|
hash: vec![0xab; 32],
|
||||||
|
count: paths.len() as i64,
|
||||||
|
total_size: 100 * paths.len() as i64,
|
||||||
|
redundant_size: 100 * (paths.len() as i64 - 1),
|
||||||
|
members: paths
|
||||||
|
.iter()
|
||||||
|
.enumerate()
|
||||||
|
.map(|(i, p)| {
|
||||||
|
let name = p.rsplit('/').next().unwrap_or(p).to_string();
|
||||||
|
(i as i64, name, p.to_string(), 100u64, 1_700_000_000i64)
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn loaded(paths: &[&str]) -> DuplicatesTab {
|
||||||
|
DuplicatesTab {
|
||||||
|
state: DupState::Loaded(vec![group(paths)]),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn frame(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
tab: &mut DuplicatesTab,
|
||||||
|
busy: bool,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (egui::FullOutput, DuplicatesActions) {
|
||||||
|
let mut actions = DuplicatesActions::default();
|
||||||
|
let out = ctx.run(raw_input(SCREEN, events), |ctx| {
|
||||||
|
egui::CentralPanel::default().show(ctx, |ui| {
|
||||||
|
actions = tab.ui(ui, busy);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
(out, actions)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn click(pos: egui::Pos2, button: egui::PointerButton) -> Vec<egui::Event> {
|
||||||
|
let mut events = vec![egui::Event::PointerMoved(pos)];
|
||||||
|
events.extend(
|
||||||
|
[true, false]
|
||||||
|
.into_iter()
|
||||||
|
.map(|pressed| egui::Event::PointerButton {
|
||||||
|
pos,
|
||||||
|
button,
|
||||||
|
pressed,
|
||||||
|
modifiers: egui::Modifiers::default(),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
events
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Right-click `needle` and return what the menu it opened painted, plus the
|
||||||
|
/// actions from that frame.
|
||||||
|
fn context_menu_on(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
tab: &mut DuplicatesTab,
|
||||||
|
busy: bool,
|
||||||
|
needle: &str,
|
||||||
|
) -> (Vec<String>, egui::Pos2) {
|
||||||
|
let (out, _) = frame(ctx, tab, busy, Vec::new());
|
||||||
|
let target = painted_text_center(&out, needle)
|
||||||
|
.unwrap_or_else(|| panic!("nothing painted for {needle:?}"));
|
||||||
|
frame(
|
||||||
|
ctx,
|
||||||
|
tab,
|
||||||
|
busy,
|
||||||
|
click(target, egui::PointerButton::Secondary),
|
||||||
|
);
|
||||||
|
// The menu is its own area, painted on the frame after the click.
|
||||||
|
let (out, _) = frame(ctx, tab, busy, Vec::new());
|
||||||
|
(painted_text(&out), target)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The title line carries the group; find it without rebuilding its wording.
|
||||||
|
fn header_of(ctx: &egui::Context, tab: &mut DuplicatesTab) -> String {
|
||||||
|
let (out, _) = frame(ctx, tab, false, Vec::new());
|
||||||
|
painted_text(&out)
|
||||||
|
.into_iter()
|
||||||
|
.find(|t| t.contains("reclaimable"))
|
||||||
|
.expect("no group header painted")
|
||||||
|
}
|
||||||
|
|
||||||
|
const PATHS: [&str; 3] = ["/a/img.raw", "/b/img.raw", "/c/img.raw"];
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_group_header_offers_the_verification() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tab = loaded(&PATHS);
|
||||||
|
let header = header_of(&ctx, &mut tab);
|
||||||
|
let (menu, _) = context_menu_on(&ctx, &mut tab, false, &header);
|
||||||
|
assert!(
|
||||||
|
menu.contains(&VERIFY_LABEL.to_string()),
|
||||||
|
"the group's own row does not offer it: {menu:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Clicking it asks for the whole group, not the one row it was asked from.
|
||||||
|
#[test]
|
||||||
|
fn verifying_asks_for_every_member_of_the_group() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tab = loaded(&PATHS);
|
||||||
|
let header = header_of(&ctx, &mut tab);
|
||||||
|
context_menu_on(&ctx, &mut tab, false, &header);
|
||||||
|
|
||||||
|
let (out, _) = frame(&ctx, &mut tab, false, Vec::new());
|
||||||
|
let entry = painted_text_center(&out, VERIFY_LABEL).expect("no verify entry painted");
|
||||||
|
let (_, actions) = frame(
|
||||||
|
&ctx,
|
||||||
|
&mut tab,
|
||||||
|
false,
|
||||||
|
click(entry, egui::PointerButton::Primary),
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
actions.verify,
|
||||||
|
Some(PATHS.iter().map(|p| p.to_string()).collect::<Vec<_>>())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// There is one verification window, so a second run is refused where it is
|
||||||
|
/// asked for rather than replacing what someone is reading.
|
||||||
|
#[test]
|
||||||
|
fn a_second_verification_is_refused_while_the_window_is_open() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tab = loaded(&PATHS);
|
||||||
|
let header = header_of(&ctx, &mut tab);
|
||||||
|
context_menu_on(&ctx, &mut tab, true, &header);
|
||||||
|
|
||||||
|
let (out, _) = frame(&ctx, &mut tab, true, Vec::new());
|
||||||
|
let entry = painted_text_center(&out, VERIFY_LABEL).expect("the entry should still be listed");
|
||||||
|
let (_, actions) = frame(
|
||||||
|
&ctx,
|
||||||
|
&mut tab,
|
||||||
|
true,
|
||||||
|
click(entry, egui::PointerButton::Primary),
|
||||||
|
);
|
||||||
|
assert_eq!(actions.verify, None, "a disabled entry still fired");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Expanding a group and right-clicking one of its files offers the same
|
||||||
|
/// thing: the rows are what someone is looking at when the question occurs.
|
||||||
|
#[test]
|
||||||
|
fn a_member_row_offers_the_verification_too() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tab = loaded(&PATHS);
|
||||||
|
let header = header_of(&ctx, &mut tab);
|
||||||
|
|
||||||
|
let (out, _) = frame(&ctx, &mut tab, false, Vec::new());
|
||||||
|
let pos = painted_text_center(&out, &header).expect("no header painted");
|
||||||
|
frame(
|
||||||
|
&ctx,
|
||||||
|
&mut tab,
|
||||||
|
false,
|
||||||
|
click(pos, egui::PointerButton::Primary),
|
||||||
|
);
|
||||||
|
|
||||||
|
let (menu, _) = context_menu_on(&ctx, &mut tab, false, PATHS[1]);
|
||||||
|
assert!(
|
||||||
|
menu.contains(&VERIFY_LABEL.to_string()),
|
||||||
|
"an expanded member row does not offer it: {menu:?}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
menu.contains(&"Open File".to_string()),
|
||||||
|
"the existing entries went missing: {menu:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_result_says_so_rather_than_showing_an_empty_list() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tab = DuplicatesTab {
|
||||||
|
state: DupState::Loaded(Vec::new()),
|
||||||
|
};
|
||||||
|
let painted = painted_text(&frame(&ctx, &mut tab, false, Vec::new()).0);
|
||||||
|
assert!(
|
||||||
|
painted.contains(&"No duplicate files found.".to_string()),
|
||||||
|
"{painted:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
@ -1,7 +1,9 @@
|
||||||
//! The Help tab: a quickstart guide for first-time users. The complete
|
//! The Help tab: a quickstart guide for first-time users. The complete
|
||||||
//! technical reference stays in README.md.
|
//! technical reference stays in README.md.
|
||||||
|
|
||||||
pub fn ui(ui: &mut egui::Ui) {
|
/// Returns true when the "Show the introduction again" button was clicked.
|
||||||
|
pub fn ui(ui: &mut egui::Ui) -> bool {
|
||||||
|
let mut replay = false;
|
||||||
let scroll = egui::ScrollArea::vertical()
|
let scroll = egui::ScrollArea::vertical()
|
||||||
.auto_shrink([false; 2])
|
.auto_shrink([false; 2])
|
||||||
.show(ui, |ui| {
|
.show(ui, |ui| {
|
||||||
|
|
@ -16,6 +18,11 @@ pub fn ui(ui: &mut egui::Ui) {
|
||||||
files by name and by what is inside them, as you type.",
|
files by name and by what is inside them, as you type.",
|
||||||
);
|
);
|
||||||
|
|
||||||
|
ui.add_space(6.0);
|
||||||
|
if ui.button("Show the introduction again").clicked() {
|
||||||
|
replay = true;
|
||||||
|
}
|
||||||
|
|
||||||
ui.add_space(12.0);
|
ui.add_space(12.0);
|
||||||
ui.heading(egui::RichText::new("Getting started").strong());
|
ui.heading(egui::RichText::new("Getting started").strong());
|
||||||
ui.add_space(4.0);
|
ui.add_space(4.0);
|
||||||
|
|
@ -44,23 +51,26 @@ pub fn ui(ui: &mut egui::Ui) {
|
||||||
words:",
|
words:",
|
||||||
);
|
);
|
||||||
ui.monospace("type:Document modified:>=2024-01-01 report");
|
ui.monospace("type:Document modified:>=2024-01-01 report");
|
||||||
ui.label("The ? button next to the search box shows the full query syntax.");
|
ui.label("The ? button left of the search box shows the full query syntax.");
|
||||||
ui.add_space(6.0);
|
ui.add_space(6.0);
|
||||||
ui.label(
|
ui.label(
|
||||||
"• Tick Fuzzy to also find matches with typos in them, at some \
|
"• Tick Fuzzy to also find matches with typos in them, at some \
|
||||||
cost in speed.",
|
cost in speed.",
|
||||||
);
|
);
|
||||||
ui.label(
|
ui.label(
|
||||||
"• Click a column header — Name, Path, Size, Modified, Rank — to \
|
"• Click a column header to sort the results; click it again to \
|
||||||
sort the results; click it again to reverse the order.",
|
reverse the order. Right-click any header to choose which \
|
||||||
|
columns are shown — size and modified date start hidden.",
|
||||||
);
|
);
|
||||||
ui.label(
|
ui.label(
|
||||||
"• Right-click a result to open it, open its containing folder, \
|
"• Right-click a result to open it, open its containing folder, \
|
||||||
or hide files like it from the results.",
|
or hide files like it from the results.",
|
||||||
);
|
);
|
||||||
ui.label(
|
ui.label(
|
||||||
"• Matches inside a file's contents show a snippet of the \
|
"• A match in a file's name or path is highlighted in that \
|
||||||
surrounding text under the file name.",
|
column; a match in its contents shows a snippet of the \
|
||||||
|
surrounding text in the Content Match column, with the rest on \
|
||||||
|
hover.",
|
||||||
);
|
);
|
||||||
|
|
||||||
ui.add_space(12.0);
|
ui.add_space(12.0);
|
||||||
|
|
@ -92,7 +102,12 @@ pub fn ui(ui: &mut egui::Ui) {
|
||||||
"warnings from indexing and folder watching that a \
|
"warnings from indexing and folder watching that a \
|
||||||
terminal would have shown",
|
terminal would have shown",
|
||||||
);
|
);
|
||||||
row(ui, "⚙ (top right)", "application options");
|
row(
|
||||||
|
ui,
|
||||||
|
"Settings",
|
||||||
|
"everything QuickSearch can be told to do, in one \
|
||||||
|
place; hover any control for what it means",
|
||||||
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
ui.add_space(12.0);
|
ui.add_space(12.0);
|
||||||
|
|
@ -131,6 +146,7 @@ pub fn ui(ui: &mut egui::Ui) {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
crate::ui_util::more_below_hint(ui, &scroll);
|
crate::ui_util::more_below_hint(ui, &scroll);
|
||||||
|
replay
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Where this build left the README: under the install prefix's `share/doc`
|
/// Where this build left the README: under the install prefix's `share/doc`
|
||||||
|
|
|
||||||
|
|
@ -2,10 +2,10 @@
|
||||||
//! produce.
|
//! produce.
|
||||||
//!
|
//!
|
||||||
//! A shortcut is written in three different vocabularies before it reaches an
|
//! A shortcut is written in three different vocabularies before it reaches an
|
||||||
//! operating system: the text in `config.toml` and on the Options button, the
|
//! operating system: the text in `config.toml` and on the Settings tab's
|
||||||
//! token `global-hotkey` parses for `RegisterHotKey`/`XGrabKey`, and the
|
//! button, the token `global-hotkey` parses for `RegisterHotKey`/`XGrabKey`,
|
||||||
//! xkbcommon keysym name the XDG *shortcuts* specification wants for the
|
//! and the xkbcommon keysym name the XDG *shortcuts* specification wants for
|
||||||
//! Wayland portal. All three come out of [`KEYS`], so a key cannot be
|
//! the Wayland portal. All three come out of [`KEYS`], so a key cannot be
|
||||||
//! spelled correctly for one backend and wrongly for the other.
|
//! spelled correctly for one backend and wrongly for the other.
|
||||||
//!
|
//!
|
||||||
//! The config text and the `global-hotkey` token are the same string: every
|
//! The config text and the `global-hotkey` token are the same string: every
|
||||||
|
|
@ -103,7 +103,7 @@ const KEYS: &[(Key, &str, &str)] = &[
|
||||||
(Key::CloseBracket, "BracketRight", "bracketright"),
|
(Key::CloseBracket, "BracketRight", "bracketright"),
|
||||||
];
|
];
|
||||||
|
|
||||||
/// Escape is reserved: it cancels the Options window's capture, and a
|
/// Escape is reserved: it cancels the Settings tab's capture, and a
|
||||||
/// system-wide Escape would be unusable anyway.
|
/// system-wide Escape would be unusable anyway.
|
||||||
const RESERVED: &[Key] = &[Key::Escape];
|
const RESERVED: &[Key] = &[Key::Escape];
|
||||||
|
|
||||||
|
|
@ -111,7 +111,7 @@ const RESERVED: &[Key] = &[Key::Escape];
|
||||||
///
|
///
|
||||||
/// Super/Meta is absent because `egui::Modifiers` has no field for it — egui
|
/// Super/Meta is absent because `egui::Modifiers` has no field for it — egui
|
||||||
/// reports alt, ctrl, shift and the Mac command key only — so a Super combo
|
/// reports alt, ctrl, shift and the Mac command key only — so a Super combo
|
||||||
/// could never be captured in the Options window even if a backend could
|
/// could never be captured in the Settings tab even if a backend could
|
||||||
/// register it.
|
/// register it.
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub struct Binding {
|
pub struct Binding {
|
||||||
|
|
@ -122,7 +122,7 @@ pub struct Binding {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Why a string or a key press is not a usable shortcut; the wording is
|
/// Why a string or a key press is not a usable shortcut; the wording is
|
||||||
/// shown in the Options window.
|
/// shown in the Settings tab.
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub enum BindingError {
|
pub enum BindingError {
|
||||||
Empty,
|
Empty,
|
||||||
|
|
@ -149,7 +149,7 @@ impl fmt::Display for BindingError {
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Binding {
|
impl Binding {
|
||||||
/// Build from a key press egui reported, for the Options window's capture
|
/// Build from a key press egui reported, for the Settings tab's capture
|
||||||
/// widget. `None` for a press that cannot be a shortcut: a key with no
|
/// widget. `None` for a press that cannot be a shortcut: a key with no
|
||||||
/// row in [`KEYS`], a reserved key, or a bare key with no modifier held.
|
/// row in [`KEYS`], a reserved key, or a bare key with no modifier held.
|
||||||
///
|
///
|
||||||
|
|
|
||||||
|
|
@ -44,7 +44,7 @@ thread_local! {
|
||||||
static REGISTRY: RefCell<Option<Registry>> = const { RefCell::new(None) };
|
static REGISTRY: RefCell<Option<Registry>> = const { RefCell::new(None) };
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What the Options window says about the shortcut.
|
/// What the Settings tab says about the shortcut.
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub enum Status {
|
pub enum Status {
|
||||||
/// The setting is empty: no shortcut, by choice.
|
/// The setting is empty: no shortcut, by choice.
|
||||||
|
|
@ -256,7 +256,7 @@ mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
/// Nothing may touch an OS registration before `init`, so that the
|
/// Nothing may touch an OS registration before `init`, so that the
|
||||||
/// headless UI tests can render the Options row.
|
/// headless UI tests can render the Settings row.
|
||||||
#[test]
|
#[test]
|
||||||
fn an_uninitialised_registry_is_inert() {
|
fn an_uninitialised_registry_is_inert() {
|
||||||
apply("Ctrl+Shift+F");
|
apply("Ctrl+Shift+F");
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@
|
||||||
//! `preferred_trigger`, and the compositor is free to bind something else, to
|
//! `preferred_trigger`, and the compositor is free to bind something else, to
|
||||||
//! ask the user first, or to let them change it later in its own settings.
|
//! ask the user first, or to let them change it later in its own settings.
|
||||||
//! What it actually bound comes back as a human-readable
|
//! What it actually bound comes back as a human-readable
|
||||||
//! `trigger_description`, which is what the Options window shows.
|
//! `trigger_description`, which is what the Settings tab shows.
|
||||||
//!
|
//!
|
||||||
//! All of this lives on its own thread. The portal is D-Bus, so every call
|
//! All of this lives on its own thread. The portal is D-Bus, so every call
|
||||||
//! is a round trip that could block for as long as a dialog stays on screen,
|
//! is a round trip that could block for as long as a dialog stays on screen,
|
||||||
|
|
@ -56,7 +56,7 @@ impl Portal {
|
||||||
.spawn(move || pollster::block_on(run(ctx, status, rx)))
|
.spawn(move || pollster::block_on(run(ctx, status, rx)))
|
||||||
{
|
{
|
||||||
// Not worth taking the app down for — but the status must say
|
// Not worth taking the app down for — but the status must say
|
||||||
// so, or Options shows "Waiting for your desktop…" forever.
|
// so, or the Settings tab shows "Waiting for your desktop…" forever.
|
||||||
quicksearch_core::log_warn!("global shortcut portal thread: {}", e);
|
quicksearch_core::log_warn!("global shortcut portal thread: {}", e);
|
||||||
*lock_ok(&portal.status) =
|
*lock_ok(&portal.status) =
|
||||||
Status::Error(format!("the shortcut thread could not be started: {}", e));
|
Status::Error(format!("the shortcut thread could not be started: {}", e));
|
||||||
|
|
@ -194,7 +194,7 @@ fn unavailable(e: &ashpd::Error) -> String {
|
||||||
|
|
||||||
fn set(ctx: &egui::Context, status: &Mutex<Status>, next: Status) {
|
fn set(ctx: &egui::Context, status: &Mutex<Status>, next: Status) {
|
||||||
*lock_ok(status) = next;
|
*lock_ok(status) = next;
|
||||||
// The Options window may be open and waiting for this.
|
// The Settings tab may be on screen and waiting for this.
|
||||||
ctx.request_repaint();
|
ctx.request_repaint();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -22,14 +22,15 @@ mod hotkey;
|
||||||
mod keychain;
|
mod keychain;
|
||||||
mod logs_tab;
|
mod logs_tab;
|
||||||
mod manage_tab;
|
mod manage_tab;
|
||||||
mod options;
|
|
||||||
mod platform;
|
mod platform;
|
||||||
mod query_highlight;
|
mod query_highlight;
|
||||||
mod search_tab;
|
mod search_tab;
|
||||||
|
mod settings_tab;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod test_ui;
|
mod test_ui;
|
||||||
mod tips;
|
mod tips;
|
||||||
mod tracker;
|
mod tracker;
|
||||||
|
mod tutorial;
|
||||||
mod ui_util;
|
mod ui_util;
|
||||||
mod unlock;
|
mod unlock;
|
||||||
mod version;
|
mod version;
|
||||||
|
|
|
||||||
|
|
@ -434,7 +434,7 @@ impl ManageTab {
|
||||||
|
|
||||||
ui.label(hint(
|
ui.label(hint(
|
||||||
"Reindex interval, symlinks, hidden files, tokenizer, and size \
|
"Reindex interval, symlinks, hidden files, tokenizer, and size \
|
||||||
limits are in Options (⚙ in the toolbar).",
|
limits are on the Settings tab.",
|
||||||
));
|
));
|
||||||
ui.add_space(8.0);
|
ui.add_space(8.0);
|
||||||
|
|
||||||
|
|
@ -542,10 +542,10 @@ fn db_size_tooltip(ui: &mut egui::Ui) {
|
||||||
"Remove indexed folders you do not need, in Indexed folders above.",
|
"Remove indexed folders you do not need, in Indexed folders above.",
|
||||||
"Narrow the full-text extension whitelist, so text is only extracted \
|
"Narrow the full-text extension whitelist, so text is only extracted \
|
||||||
from the file types you actually search.",
|
from the file types you actually search.",
|
||||||
"Turn off \"Store text for snippets\" in Options: full-text search keeps \
|
"Turn off \"Store text for snippets\" on the Settings tab: full-text search \
|
||||||
working, but without previews, occurrence ranking or fuzzy matching \
|
keeps working, but without previews, occurrence ranking or fuzzy matching \
|
||||||
inside file contents.",
|
inside file contents.",
|
||||||
"Lower \"Max text file size\" and \"Max stored text\" in Options.",
|
"Lower \"Max text file size\" and \"Max stored text\", both on the Settings tab.",
|
||||||
] {
|
] {
|
||||||
ui.label(format!("• {}", lever));
|
ui.label(format!("• {}", lever));
|
||||||
}
|
}
|
||||||
|
|
@ -809,21 +809,35 @@ fn root_row(ui: &mut egui::Ui, r: &RootProgress) {
|
||||||
RootPhase::Extracting => {
|
RootPhase::Extracting => {
|
||||||
ui.label(egui::RichText::new("extracting text").color(phase.green));
|
ui.label(egui::RichText::new("extracting text").color(phase.green));
|
||||||
divider(ui);
|
divider(ui);
|
||||||
let frac = if r.extract_total > 0 {
|
let workers = format!("{}/{} workers", r.active_workers, r.total_workers);
|
||||||
(r.extracted as f32 / r.extract_total as f32).clamp(0.0, 1.0)
|
match r.extract_total {
|
||||||
|
Some(total) => {
|
||||||
|
let frac = if total > 0 {
|
||||||
|
(r.extracted as f32 / total as f32).clamp(0.0, 1.0)
|
||||||
} else {
|
} else {
|
||||||
1.0
|
1.0
|
||||||
};
|
};
|
||||||
ui.label(format!(
|
ui.label(format!(
|
||||||
"{} / {} ({:.0}%) · {}/{} workers",
|
"{} / {} ({:.0}%) · {}",
|
||||||
group_thousands(r.extracted as u64),
|
group_thousands(r.extracted as u64),
|
||||||
group_thousands(r.extract_total as u64),
|
group_thousands(total as u64),
|
||||||
frac * 100.0,
|
frac * 100.0,
|
||||||
r.active_workers,
|
workers
|
||||||
r.total_workers
|
|
||||||
));
|
));
|
||||||
crate::ui_util::progress_bar(ui, Some(frac), 160.0);
|
crate::ui_util::progress_bar(ui, Some(frac), 160.0);
|
||||||
}
|
}
|
||||||
|
// The pass is still counting its range — the same shape
|
||||||
|
// as a walk without a denominator yet.
|
||||||
|
None => {
|
||||||
|
ui.label(format!(
|
||||||
|
"{} files · {}",
|
||||||
|
group_thousands(r.extracted as u64),
|
||||||
|
workers
|
||||||
|
));
|
||||||
|
crate::ui_util::progress_bar(ui, None, 160.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
RootPhase::Done => {
|
RootPhase::Done => {
|
||||||
// Whole-root totals: `walked` counts every file the walk saw
|
// Whole-root totals: `walked` counts every file the walk saw
|
||||||
// and `extracted` all rows with searchable text, not just
|
// and `extracted` all rows with searchable text, not just
|
||||||
|
|
|
||||||
|
|
@ -64,7 +64,7 @@ fn running_state(roots: &[&str], current_file: Option<&str>) -> IndexerState {
|
||||||
walked: 100,
|
walked: 100,
|
||||||
walk_total: Some(1000),
|
walk_total: Some(1000),
|
||||||
extracted: 0,
|
extracted: 0,
|
||||||
extract_total: 0,
|
extract_total: None,
|
||||||
current_file: current_file.map(str::to_string),
|
current_file: current_file.map(str::to_string),
|
||||||
active_workers: 4,
|
active_workers: 4,
|
||||||
total_workers: 4,
|
total_workers: 4,
|
||||||
|
|
@ -304,7 +304,7 @@ fn root_progress(phase: RootPhase, walked: usize, walk_total: Option<usize>) ->
|
||||||
walked,
|
walked,
|
||||||
walk_total,
|
walk_total,
|
||||||
extracted: 0,
|
extracted: 0,
|
||||||
extract_total: 0,
|
extract_total: None,
|
||||||
current_file: None,
|
current_file: None,
|
||||||
active_workers: 4,
|
active_workers: 4,
|
||||||
total_workers: 4,
|
total_workers: 4,
|
||||||
|
|
@ -683,7 +683,7 @@ fn the_probe_caches_until_the_refresh_interval_is_up() {
|
||||||
let _ = std::fs::remove_dir_all(&dir);
|
let _ = std::fs::remove_dir_all(&dir);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A database path edited in Options must not keep showing the old
|
/// A database path edited on the Settings tab must not keep showing the old
|
||||||
/// database's size for the rest of the interval.
|
/// database's size for the rest of the interval.
|
||||||
#[test]
|
#[test]
|
||||||
fn the_probe_follows_a_changed_database_path() {
|
fn the_probe_follows_a_changed_database_path() {
|
||||||
|
|
@ -762,7 +762,7 @@ fn hovering_the_size_explains_how_to_shrink_the_index() {
|
||||||
"Indexed folders",
|
"Indexed folders",
|
||||||
"whitelist",
|
"whitelist",
|
||||||
"Store text for snippets",
|
"Store text for snippets",
|
||||||
"Options",
|
"Settings tab",
|
||||||
] {
|
] {
|
||||||
assert!(text.contains(lever), "tooltip never mentions {}", lever);
|
assert!(text.contains(lever), "tooltip never mentions {}", lever);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load diff
|
|
@ -37,6 +37,11 @@ const SNIPPET_LEAD: &str = "… ";
|
||||||
|
|
||||||
/// Append `window[range]` to `job`, highlighting whatever parts of `ranges`
|
/// Append `window[range]` to `job`, highlighting whatever parts of `ranges`
|
||||||
/// (byte offsets into `window`) fall inside it.
|
/// (byte offsets into `window`) fall inside it.
|
||||||
|
///
|
||||||
|
/// Ranges are clipped to the slice, so a caller rendering a string in pieces
|
||||||
|
/// can hand each piece the *whole* set: a range inside this one survives, one
|
||||||
|
/// straddling an edge survives as the part that is here, and one wholly
|
||||||
|
/// outside disappears.
|
||||||
fn append_marked(
|
fn append_marked(
|
||||||
job: &mut LayoutJob,
|
job: &mut LayoutJob,
|
||||||
fmt: &SnippetFormats,
|
fmt: &SnippetFormats,
|
||||||
|
|
@ -61,6 +66,18 @@ fn append_marked(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A whole field — a filename — with its matched spans marked.
|
||||||
|
///
|
||||||
|
/// Wrapping is left at the job's defaults on purpose: `egui::Label` overwrites
|
||||||
|
/// only `wrap.max_width`, so this is laid out exactly like the plain string it
|
||||||
|
/// replaces, and the cell keeps the height and clipping it had before.
|
||||||
|
pub(super) fn marked_field_job(ui: &egui::Ui, text: &str, ranges: &[(usize, usize)]) -> LayoutJob {
|
||||||
|
let fmt = snippet_formats(ui);
|
||||||
|
let mut job = LayoutJob::default();
|
||||||
|
append_marked(&mut job, &fmt, text, ranges, 0..text.len());
|
||||||
|
job
|
||||||
|
}
|
||||||
|
|
||||||
/// The byte offset in `snip.window` that rendering has to start at for the
|
/// The byte offset in `snip.window` that rendering has to start at for the
|
||||||
/// first match to land on a row that survives `max_rows`; `0` when it
|
/// first match to land on a row that survives `max_rows`; `0` when it
|
||||||
/// already does. epaint stops at `wrap.max_rows` and *every* `\n` costs a
|
/// already does. epaint stops at `wrap.max_rows` and *every* `\n` costs a
|
||||||
|
|
@ -159,16 +176,14 @@ pub(super) fn snippet_job(ui: &egui::Ui, snip: &Snippet, max_rows: usize) -> Lay
|
||||||
job
|
job
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The Match column cell: one line with the (first) matched span centered
|
/// The Content Match column cell: one line with the (first) matched span
|
||||||
/// and an equal amount of context on both sides, trimmed to what fits the
|
/// centered and an equal amount of context on both sides, trimmed to what fits
|
||||||
/// column width. Matches on a whole field — a filename or a path — are
|
/// the column width.
|
||||||
/// wrapped in brackets: `[name]`.
|
///
|
||||||
pub(super) fn centered_match_job(
|
/// Only ever called with a content snippet. Name and path matches are
|
||||||
ui: &egui::Ui,
|
/// highlighted in their own columns and leave a dash here, so the bracketed
|
||||||
snip: &Snippet,
|
/// `[whole field]` rendering this used to carry is gone.
|
||||||
width_px: f32,
|
pub(super) fn centered_match_job(ui: &egui::Ui, snip: &Snippet, width_px: f32) -> LayoutJob {
|
||||||
whole_field: bool,
|
|
||||||
) -> LayoutJob {
|
|
||||||
let fmt = snippet_formats(ui);
|
let fmt = snippet_formats(ui);
|
||||||
|
|
||||||
// Newlines force line breaks even in a one-row LayoutJob; flatten them
|
// Newlines force line breaks even in a one-row LayoutJob; flatten them
|
||||||
|
|
@ -189,11 +204,6 @@ pub(super) fn centered_match_job(
|
||||||
let font_id = &fmt.normal.font_id;
|
let font_id = &fmt.normal.font_id;
|
||||||
let width_of = |c: char| f.glyph_width(font_id, c);
|
let width_of = |c: char| f.glyph_width(font_id, c);
|
||||||
let ellipsis = width_of('…');
|
let ellipsis = width_of('…');
|
||||||
let brackets = if whole_field {
|
|
||||||
width_of('[') + width_of(']')
|
|
||||||
} else {
|
|
||||||
0.0
|
|
||||||
};
|
|
||||||
let mut marks = 0.0;
|
let mut marks = 0.0;
|
||||||
if snip.truncated_start {
|
if snip.truncated_start {
|
||||||
marks += ellipsis;
|
marks += ellipsis;
|
||||||
|
|
@ -201,13 +211,13 @@ pub(super) fn centered_match_job(
|
||||||
if snip.truncated_end {
|
if snip.truncated_end {
|
||||||
marks += ellipsis;
|
marks += ellipsis;
|
||||||
}
|
}
|
||||||
if fits_within(window, width_px - brackets - marks, width_of) {
|
if fits_within(window, width_px - marks, width_of) {
|
||||||
return (0, window.len(), true);
|
return (0, window.len(), true);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Something has to go, so either end may gain a mark; reserve for
|
// Something has to go, so either end may gain a mark; reserve for
|
||||||
// both so a cut never overflows the column.
|
// both so a cut never overflows the column.
|
||||||
let budget = width_px - brackets - 2.0 * ellipsis;
|
let budget = width_px - 2.0 * ellipsis;
|
||||||
let Some(&(a, b)) = snip.ranges.first() else {
|
let Some(&(a, b)) = snip.ranges.first() else {
|
||||||
// No ranges (shouldn't happen for match cells) — head trim.
|
// No ranges (shouldn't happen for match cells) — head trim.
|
||||||
return (0, take_forward(window, 0, budget.max(0.0), width_of), true);
|
return (0, take_forward(window, 0, budget.max(0.0), width_of), true);
|
||||||
|
|
@ -261,22 +271,52 @@ pub(super) fn centered_match_job(
|
||||||
let mut job = LayoutJob::default();
|
let mut job = LayoutJob::default();
|
||||||
job.wrap.max_rows = 1;
|
job.wrap.max_rows = 1;
|
||||||
job.wrap.break_anywhere = true;
|
job.wrap.break_anywhere = true;
|
||||||
if whole_field && decorate {
|
|
||||||
job.append("[", 0.0, fmt.weak.clone());
|
|
||||||
}
|
|
||||||
if decorate && (start > 0 || snip.truncated_start) {
|
if decorate && (start > 0 || snip.truncated_start) {
|
||||||
job.append("…", 0.0, fmt.weak.clone());
|
job.append("…", 0.0, fmt.weak.clone());
|
||||||
}
|
}
|
||||||
append_marked(&mut job, &fmt, window, &snip.ranges, start..end);
|
append_marked(&mut job, &fmt, window, &snip.ranges, start..end);
|
||||||
if decorate && (end < window.len() || snip.truncated_end) {
|
if decorate && (end < window.len() || snip.truncated_end) {
|
||||||
job.append("…", 0.0, fmt.weak.clone());
|
job.append("…", 0.0, fmt.weak);
|
||||||
}
|
|
||||||
if whole_field && decorate {
|
|
||||||
job.append("]", 0.0, fmt.weak);
|
|
||||||
}
|
}
|
||||||
job
|
job
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The Path column cell: middle-elided to `width_px`, with whatever of a
|
||||||
|
/// path-tier match survives the cut highlighted.
|
||||||
|
///
|
||||||
|
/// The path reads at full strength — it is the one column that identifies a
|
||||||
|
/// result on its own. Only the elision mark is weak, since it is punctuation
|
||||||
|
/// this renderer added rather than anything the file is named.
|
||||||
|
///
|
||||||
|
/// Returns the job and whether anything was actually elided — the caller's
|
||||||
|
/// trigger for a full-path tooltip, since egui offers one only when *it* did
|
||||||
|
/// the eliding and it is handed an already-shortened string.
|
||||||
|
pub(super) fn path_cell_job(
|
||||||
|
ui: &egui::Ui,
|
||||||
|
path: &str,
|
||||||
|
ranges: &[(usize, usize)],
|
||||||
|
width_px: f32,
|
||||||
|
font_id: &egui::FontId,
|
||||||
|
) -> (LayoutJob, bool) {
|
||||||
|
let fmt = snippet_formats(ui);
|
||||||
|
let mut job = LayoutJob::default();
|
||||||
|
match crate::ui_util::middle_elide_cut(ui, path, width_px, font_id) {
|
||||||
|
// It fits: the whole path, marked — which is exactly
|
||||||
|
// [`marked_field_job`].
|
||||||
|
None => (marked_field_job(ui, path, ranges), false),
|
||||||
|
// The two surviving ends are appended straight from `path` at their
|
||||||
|
// original offsets: `append_marked` clips the ranges to each end, so a
|
||||||
|
// match that fell in the dropped middle drops with it rather than
|
||||||
|
// landing on whatever glyphs moved into those offsets.
|
||||||
|
Some((head, tail)) => {
|
||||||
|
append_marked(&mut job, &fmt, path, ranges, 0..head);
|
||||||
|
job.append("…", 0.0, fmt.weak.clone());
|
||||||
|
append_marked(&mut job, &fmt, path, ranges, tail..path.len());
|
||||||
|
(job, true)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Whether the whole of `text` fits in `budget` pixels; stops at the first
|
/// Whether the whole of `text` fits in `budget` pixels; stops at the first
|
||||||
/// character that does not.
|
/// character that does not.
|
||||||
fn fits_within(text: &str, budget: f32, width_of: impl Fn(char) -> f32) -> bool {
|
fn fits_within(text: &str, budget: f32, width_of: impl Fn(char) -> f32) -> bool {
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,11 +1,11 @@
|
||||||
//! The Options window and the shared config editor used by both the
|
//! The Settings tab: every configuration control the GUI offers, grouped
|
||||||
//! window and the Manage Index tab. Edits happen on a draft; Apply
|
//! into sections. Edits happen on a draft; Apply validates, saves, and hands
|
||||||
//! validates, saves, and hands the new config to the app.
|
//! the new config to the app.
|
||||||
|
|
||||||
use crate::keychain;
|
use crate::keychain;
|
||||||
use crate::tips::{self, tip_row, Tipped};
|
use crate::tips::{self, tip_row, Tipped};
|
||||||
use crate::ui_util::hint;
|
use crate::ui_util::hint;
|
||||||
use quicksearch_core::config::Config;
|
use quicksearch_core::config::{ColumnsConfig, Config};
|
||||||
|
|
||||||
/// A [`tip_row`] holding one numeric [`egui::DragValue`] — the shape of most
|
/// A [`tip_row`] holding one numeric [`egui::DragValue`] — the shape of most
|
||||||
/// rows in the config editor.
|
/// rows in the config editor.
|
||||||
|
|
@ -22,7 +22,7 @@ fn drag_row<N: egui::emath::Numeric>(
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub enum Section {
|
enum Section {
|
||||||
Indexing,
|
Indexing,
|
||||||
Processing,
|
Processing,
|
||||||
Search,
|
Search,
|
||||||
|
|
@ -36,24 +36,26 @@ pub enum SecurityAction {
|
||||||
Disable,
|
Disable,
|
||||||
ChangePassword,
|
ChangePassword,
|
||||||
SetKeychain(bool),
|
SetKeychain(bool),
|
||||||
|
ShowKey,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What one frame of the Options window produced.
|
/// What one frame of the Settings tab produced.
|
||||||
#[derive(Default)]
|
#[derive(Default)]
|
||||||
pub struct OptionsOutput {
|
pub struct SettingsOutput {
|
||||||
/// "Apply & Save" was clicked with this draft.
|
/// "Apply & Save" was clicked with this draft.
|
||||||
pub applied: Option<Config>,
|
pub applied: Option<Config>,
|
||||||
/// A Security block action was clicked.
|
/// A Security block action was clicked.
|
||||||
pub security: Option<SecurityAction>,
|
pub security: Option<SecurityAction>,
|
||||||
/// The title-bar close was clicked while the draft holds unapplied
|
/// The Columns block changed. Like Security, it edits the live config
|
||||||
/// edits; the window is held open and the app raises the guard.
|
/// rather than the draft, so it takes effect without Apply.
|
||||||
pub close_requested: bool,
|
pub columns: Option<ColumnsConfig>,
|
||||||
}
|
}
|
||||||
|
|
||||||
pub struct OptionsWindow {
|
pub struct SettingsTab {
|
||||||
pub open: bool,
|
/// The staged config, built from the live one the first frame the tab is
|
||||||
|
/// shown and dropped again when it is left.
|
||||||
draft: Option<Config>,
|
draft: Option<Config>,
|
||||||
/// Cached answer from [`OptionsWindow::keychain_active`], with the
|
/// Cached answer from [`SettingsTab::keychain_active`], with the
|
||||||
/// `use_keychain` preference it was probed under.
|
/// `use_keychain` preference it was probed under.
|
||||||
keychain_probed_for: Option<bool>,
|
keychain_probed_for: Option<bool>,
|
||||||
keychain_active: bool,
|
keychain_active: bool,
|
||||||
|
|
@ -61,10 +63,9 @@ pub struct OptionsWindow {
|
||||||
capturing_hotkey: bool,
|
capturing_hotkey: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl OptionsWindow {
|
impl SettingsTab {
|
||||||
pub fn new() -> OptionsWindow {
|
pub fn new() -> SettingsTab {
|
||||||
OptionsWindow {
|
SettingsTab {
|
||||||
open: false,
|
|
||||||
draft: None,
|
draft: None,
|
||||||
keychain_probed_for: None,
|
keychain_probed_for: None,
|
||||||
keychain_active: false,
|
keychain_active: false,
|
||||||
|
|
@ -73,21 +74,16 @@ impl OptionsWindow {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether the shortcut button is reading a key press right now, so the
|
/// Whether the shortcut button is reading a key press right now, so the
|
||||||
/// app can hold the shortcut it is about to replace. See
|
/// app can hold the shortcut it is about to replace. The app gates this
|
||||||
|
/// on the tab being the one on screen. See
|
||||||
/// [`crate::unlock::Gate::handle_hotkey`].
|
/// [`crate::unlock::Gate::handle_hotkey`].
|
||||||
pub fn capturing_hotkey(&self) -> bool {
|
pub fn capturing_hotkey(&self) -> bool {
|
||||||
self.open && self.capturing_hotkey
|
self.capturing_hotkey
|
||||||
}
|
|
||||||
|
|
||||||
pub fn open_with(&mut self, current: &Config) {
|
|
||||||
self.open = true;
|
|
||||||
self.draft = Some(current.clone());
|
|
||||||
self.keychain_probed_for = None;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether the draft differs from the live config. The fields the app
|
/// Whether the draft differs from the live config. The fields the app
|
||||||
/// pins on apply are neutralized first, so the Security block never
|
/// pins on apply are neutralized first, so the Security block never
|
||||||
/// makes the window read as dirty.
|
/// makes the tab read as dirty.
|
||||||
pub fn is_dirty(&self, current: &Config) -> bool {
|
pub fn is_dirty(&self, current: &Config) -> bool {
|
||||||
let Some(draft) = &self.draft else {
|
let Some(draft) = &self.draft else {
|
||||||
return false;
|
return false;
|
||||||
|
|
@ -102,34 +98,27 @@ impl OptionsWindow {
|
||||||
self.draft.clone()
|
self.draft.clone()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Close and drop the draft (Discard, or a clean close).
|
/// Drop the draft: Discard, or leaving the tab. The next frame that
|
||||||
pub fn close_discard(&mut self) {
|
/// shows the tab stages a fresh copy of the live config, which is what
|
||||||
self.open = false;
|
/// keeps a draft from going stale against edits made on Manage Index.
|
||||||
|
pub fn discard(&mut self) {
|
||||||
self.draft = None;
|
self.draft = None;
|
||||||
self.capturing_hotkey = false;
|
self.capturing_hotkey = false;
|
||||||
|
self.keychain_probed_for = None;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Adopt the window's open flag for this frame. A dirty close is
|
/// Take a draft if there is none: the first frame the tab is shown, and
|
||||||
/// intercepted: the window is held open and the caller is told to raise
|
/// the first frame after it was left.
|
||||||
/// the unsaved-changes guard instead.
|
fn stage(&mut self, current: &Config) {
|
||||||
fn intercept_close(&mut self, still_open: bool, current: &Config) -> bool {
|
if self.draft.is_none() {
|
||||||
self.open = still_open;
|
self.draft = Some(current.clone());
|
||||||
if self.open {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
if self.is_dirty(current) {
|
|
||||||
self.open = true;
|
|
||||||
true
|
|
||||||
} else {
|
|
||||||
self.draft = None;
|
|
||||||
false
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// True when this index's key really is in the OS keychain: the
|
/// True when this index's key really is in the OS keychain: the
|
||||||
/// preference is on *and* the keychain answers with an entry (a dead
|
/// preference is on *and* the keychain answers with an entry (a dead
|
||||||
/// daemon, a locked keyring or a denied prompt all read as "no").
|
/// daemon, a locked keyring or a denied prompt all read as "no").
|
||||||
/// Probed when the window opens and when the preference changes — a
|
/// Probed when the tab is entered and when the preference changes — a
|
||||||
/// keychain read is an IPC round trip.
|
/// keychain read is an IPC round trip.
|
||||||
fn keychain_active(&mut self, current: &Config) -> bool {
|
fn keychain_active(&mut self, current: &Config) -> bool {
|
||||||
if self.keychain_probed_for != Some(current.security.use_keychain) {
|
if self.keychain_probed_for != Some(current.security.use_keychain) {
|
||||||
|
|
@ -142,29 +131,21 @@ impl OptionsWindow {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Render; reports an applied draft config and/or a security action.
|
/// Render; reports an applied draft config and/or a security action.
|
||||||
pub fn ui(&mut self, ctx: &egui::Context, current: &Config) -> OptionsOutput {
|
pub fn ui(&mut self, ui: &mut egui::Ui, current: &Config) -> SettingsOutput {
|
||||||
if !self.open {
|
self.stage(current);
|
||||||
self.draft = None;
|
let mut out = SettingsOutput::default();
|
||||||
return OptionsOutput::default();
|
|
||||||
}
|
|
||||||
if self.draft.is_none() {
|
|
||||||
self.draft = Some(current.clone());
|
|
||||||
}
|
|
||||||
let mut out = OptionsOutput::default();
|
|
||||||
let mut open = self.open;
|
|
||||||
let keychain_active = self.keychain_active(current);
|
let keychain_active = self.keychain_active(current);
|
||||||
let dirty = self.is_dirty(current);
|
let dirty = self.is_dirty(current);
|
||||||
let capturing = &mut self.capturing_hotkey;
|
let capturing = &mut self.capturing_hotkey;
|
||||||
let draft = self.draft.as_mut().unwrap();
|
let draft = self.draft.as_mut().unwrap();
|
||||||
|
|
||||||
egui::Window::new("Options")
|
|
||||||
.open(&mut open)
|
|
||||||
.resizable(false)
|
|
||||||
.default_width(420.0)
|
|
||||||
.show(ctx, |ui| {
|
|
||||||
let scroll = egui::ScrollArea::vertical()
|
let scroll = egui::ScrollArea::vertical()
|
||||||
.max_height(480.0)
|
.auto_shrink([false; 2])
|
||||||
.show(ui, |ui| {
|
.show(ui, |ui| {
|
||||||
|
// Cap the column like a document page: a maximized window
|
||||||
|
// would otherwise stretch every hint into one long line.
|
||||||
|
ui.set_max_width(620.0);
|
||||||
|
|
||||||
ui.heading(egui::RichText::new("Paths").strong());
|
ui.heading(egui::RichText::new("Paths").strong());
|
||||||
egui::Grid::new("opt-paths").num_columns(2).show(ui, |ui| {
|
egui::Grid::new("opt-paths").num_columns(2).show(ui, |ui| {
|
||||||
tip_row(ui, "Database file", &tips::DATABASE_PATH, |ui| {
|
tip_row(ui, "Database file", &tips::DATABASE_PATH, |ui| {
|
||||||
|
|
@ -191,6 +172,9 @@ impl OptionsWindow {
|
||||||
|
|
||||||
ui.heading(egui::RichText::new("Search").strong());
|
ui.heading(egui::RichText::new("Search").strong());
|
||||||
config_editor_ui(ui, draft, Section::Search);
|
config_editor_ui(ui, draft, Section::Search);
|
||||||
|
ui.add_space(6.0);
|
||||||
|
// Live, not drafted — see `columns_ui`.
|
||||||
|
out.columns = columns_ui(ui, ¤t.search.columns);
|
||||||
ui.separator();
|
ui.separator();
|
||||||
|
|
||||||
ui.heading(egui::RichText::new("Interface").strong());
|
ui.heading(egui::RichText::new("Interface").strong());
|
||||||
|
|
@ -212,15 +196,14 @@ impl OptionsWindow {
|
||||||
hotkey_note(ui, &draft.ui.search_hotkey, ¤t.ui.search_hotkey);
|
hotkey_note(ui, &draft.ui.search_hotkey, ¤t.ui.search_hotkey);
|
||||||
ui.separator();
|
ui.separator();
|
||||||
|
|
||||||
// Security acts on the live config, not the draft.
|
// Security acts on the live config, not the draft. The KDF
|
||||||
// The KDF salt is never shown here or anywhere else
|
// salt is never shown here or anywhere else in the GUI.
|
||||||
// in the GUI.
|
|
||||||
ui.heading(egui::RichText::new("Security").strong());
|
ui.heading(egui::RichText::new("Security").strong());
|
||||||
out.security = security_ui(ui, current, keychain_active);
|
out.security = security_ui(ui, current, keychain_active);
|
||||||
});
|
|
||||||
crate::ui_util::more_below_hint(ui, &scroll);
|
|
||||||
|
|
||||||
ui.separator();
|
ui.separator();
|
||||||
|
|
||||||
|
// Last in the scroll, where the Manage Index tab also puts
|
||||||
|
// it, so the two draft-backed editors read the same way.
|
||||||
let p = crate::color::palette(ui.visuals().dark_mode);
|
let p = crate::color::palette(ui.visuals().dark_mode);
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
let apply = ui
|
let apply = ui
|
||||||
|
|
@ -250,8 +233,8 @@ impl OptionsWindow {
|
||||||
and hash length require a full rebuild.",
|
and hash length require a full rebuild.",
|
||||||
));
|
));
|
||||||
});
|
});
|
||||||
|
crate::ui_util::more_below_hint(ui, &scroll);
|
||||||
|
|
||||||
out.close_requested = self.intercept_close(open, current);
|
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -398,6 +381,39 @@ fn hotkey_note(ui: &mut egui::Ui, draft: &str, live: &str) {
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The Search-tab column picker, mirroring the right-click menu on the table
|
||||||
|
/// headers. Returns the new set when a checkbox moved.
|
||||||
|
///
|
||||||
|
/// Acts on the **live** config, not the draft, for the same reason the
|
||||||
|
/// Security block does: the header menu writes columns the instant they
|
||||||
|
/// change, and a draft-backed copy here would silently revert that on the next
|
||||||
|
/// Apply. `app::pin_live_fields` keeps the draft out of this field entirely.
|
||||||
|
fn columns_ui(ui: &mut egui::Ui, current: &ColumnsConfig) -> Option<ColumnsConfig> {
|
||||||
|
let mut next = current.clone();
|
||||||
|
ui.label("Search columns").on_hover_text(tips::COLUMNS.body);
|
||||||
|
ui.horizontal_wrapped(|ui| {
|
||||||
|
ui.checkbox(&mut next.name, "Name").tip(&tips::COLUMNS);
|
||||||
|
// Checked and greyed rather than absent: an omitted entry reads as an
|
||||||
|
// oversight, a disabled one answers the question.
|
||||||
|
ui.add_enabled(false, egui::Checkbox::new(&mut true, "Path"))
|
||||||
|
.on_disabled_hover_text(
|
||||||
|
"The path is always shown — it is the only column that \
|
||||||
|
identifies a result on its own.",
|
||||||
|
);
|
||||||
|
ui.checkbox(&mut next.content_match, "Content Match")
|
||||||
|
.tip(&tips::COLUMNS);
|
||||||
|
ui.checkbox(&mut next.size, "Size").tip(&tips::COLUMNS);
|
||||||
|
ui.checkbox(&mut next.modified, "Modified")
|
||||||
|
.tip(&tips::COLUMNS);
|
||||||
|
ui.checkbox(&mut next.rank, "Rank").tip(&tips::COLUMNS);
|
||||||
|
});
|
||||||
|
ui.label(hint(
|
||||||
|
"Also on the Search tab: right-click any column header. Applied and \
|
||||||
|
saved immediately.",
|
||||||
|
));
|
||||||
|
(next != *current).then_some(next)
|
||||||
|
}
|
||||||
|
|
||||||
/// The Security block: status plus action buttons. Never renders the salt.
|
/// The Security block: status plus action buttons. Never renders the salt.
|
||||||
fn security_ui(
|
fn security_ui(
|
||||||
ui: &mut egui::Ui,
|
ui: &mut egui::Ui,
|
||||||
|
|
@ -430,6 +446,14 @@ fn security_ui(
|
||||||
action = Some(SecurityAction::Disable);
|
action = Some(SecurityAction::Disable);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
// Its own row: three buttons do not fit the window's width.
|
||||||
|
if ui
|
||||||
|
.button("Show database key…")
|
||||||
|
.tip(&tips::SHOW_KEY)
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
action = Some(SecurityAction::ShowKey);
|
||||||
|
}
|
||||||
let mut remember = current.security.use_keychain;
|
let mut remember = current.security.use_keychain;
|
||||||
if ui
|
if ui
|
||||||
.checkbox(&mut remember, "Remember on this device")
|
.checkbox(&mut remember, "Remember on this device")
|
||||||
|
|
@ -455,10 +479,10 @@ fn security_ui(
|
||||||
action
|
action
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The per-section config controls of the Options window. Every row goes
|
/// The per-section config controls of the Settings tab. Every row goes
|
||||||
/// through [`crate::tips::tip_row`], so a setting cannot arrive here
|
/// through [`crate::tips::tip_row`], so a setting cannot arrive here
|
||||||
/// without a tooltip.
|
/// without a tooltip.
|
||||||
pub fn config_editor_ui(ui: &mut egui::Ui, config: &mut Config, section: Section) {
|
fn config_editor_ui(ui: &mut egui::Ui, config: &mut Config, section: Section) {
|
||||||
match section {
|
match section {
|
||||||
Section::Indexing => {
|
Section::Indexing => {
|
||||||
egui::Grid::new("cfg-indexing")
|
egui::Grid::new("cfg-indexing")
|
||||||
|
|
@ -598,6 +622,10 @@ pub fn config_editor_ui(ui: &mut egui::Ui, config: &mut Config, section: Section
|
||||||
&mut config.search.debounce_ms,
|
&mut config.search.debounce_ms,
|
||||||
0..=2000,
|
0..=2000,
|
||||||
);
|
);
|
||||||
|
|
||||||
|
tip_row(ui, "Live results", &tips::LIVE_RESULTS, |ui| {
|
||||||
|
ui.checkbox(&mut config.search.live_results, "")
|
||||||
|
});
|
||||||
});
|
});
|
||||||
// The warning comes and goes as the value is edited; keep it off
|
// The warning comes and goes as the value is edited; keep it off
|
||||||
// the ids of what follows (`ui_util::stable_section`).
|
// the ids of what follows (`ui_util::stable_section`).
|
||||||
|
|
@ -5,62 +5,74 @@ use super::*;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_fresh_draft_is_not_dirty() {
|
fn a_fresh_draft_is_not_dirty() {
|
||||||
let mut w = OptionsWindow::new();
|
let mut w = SettingsTab::new();
|
||||||
let cfg = Config::default();
|
let cfg = Config::default();
|
||||||
assert!(!w.is_dirty(&cfg), "no draft at all");
|
assert!(!w.is_dirty(&cfg), "no draft at all");
|
||||||
w.open_with(&cfg);
|
w.stage(&cfg);
|
||||||
assert!(!w.is_dirty(&cfg));
|
assert!(!w.is_dirty(&cfg));
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn an_edited_draft_is_dirty_until_discarded() {
|
fn an_edited_draft_is_dirty_until_discarded() {
|
||||||
let mut w = OptionsWindow::new();
|
let mut w = SettingsTab::new();
|
||||||
let cfg = Config::default();
|
let cfg = Config::default();
|
||||||
w.open_with(&cfg);
|
w.stage(&cfg);
|
||||||
w.draft.as_mut().unwrap().search.debounce_ms += 100;
|
w.draft.as_mut().unwrap().search.debounce_ms += 100;
|
||||||
assert!(w.is_dirty(&cfg));
|
assert!(w.is_dirty(&cfg));
|
||||||
w.close_discard();
|
w.discard();
|
||||||
assert!(!w.open);
|
assert!(w.draft.is_none());
|
||||||
assert!(!w.is_dirty(&cfg), "the draft is gone");
|
assert!(!w.is_dirty(&cfg), "the draft is gone");
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The Security block and the mode buttons act on the live config while
|
/// The Security block and the mode buttons act on the live config while
|
||||||
/// the window sits open; the stale copies in the draft are not edits.
|
/// the tab is on screen; the stale copies in the draft are not edits.
|
||||||
#[test]
|
#[test]
|
||||||
fn live_security_and_mode_changes_are_not_dirty() {
|
fn live_security_and_mode_changes_are_not_dirty() {
|
||||||
let mut w = OptionsWindow::new();
|
let mut w = SettingsTab::new();
|
||||||
let mut cfg = Config::default();
|
let mut cfg = Config::default();
|
||||||
w.open_with(&cfg);
|
w.stage(&cfg);
|
||||||
cfg.security.use_keychain = !cfg.security.use_keychain;
|
cfg.security.use_keychain = !cfg.security.use_keychain;
|
||||||
cfg.indexing.auto_index = !cfg.indexing.auto_index;
|
cfg.indexing.auto_index = !cfg.indexing.auto_index;
|
||||||
assert!(!w.is_dirty(&cfg));
|
assert!(!w.is_dirty(&cfg));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Leaving the tab drops the draft, so the next visit stages the config as
|
||||||
|
/// it stands *then*. Without this an edit made on the Manage Index tab in
|
||||||
|
/// between would be reverted by a later Apply: `pin_live_fields` protects
|
||||||
|
/// the fields saved live, but not the indexed folders or the filters.
|
||||||
#[test]
|
#[test]
|
||||||
fn a_dirty_close_is_held_and_a_clean_one_drops_the_draft() {
|
fn a_draft_is_restaged_from_the_live_config_after_leaving() {
|
||||||
let mut w = OptionsWindow::new();
|
let mut w = SettingsTab::new();
|
||||||
let cfg = Config::default();
|
let mut cfg = Config::default();
|
||||||
w.open_with(&cfg);
|
w.stage(&cfg);
|
||||||
w.draft.as_mut().unwrap().search.debounce_ms += 100;
|
w.discard();
|
||||||
|
|
||||||
assert!(
|
cfg.indexing.ignore_patterns.push("*.tmp".to_string());
|
||||||
w.intercept_close(false, &cfg),
|
w.stage(&cfg);
|
||||||
"dirty close raises the guard"
|
assert!(!w.is_dirty(&cfg), "the fresh draft matches the live config");
|
||||||
|
assert_eq!(
|
||||||
|
w.draft_config().unwrap().indexing.ignore_patterns,
|
||||||
|
cfg.indexing.ignore_patterns,
|
||||||
|
"the filter added while the tab was away survives"
|
||||||
);
|
);
|
||||||
assert!(w.open, "the window is held open until the user decides");
|
}
|
||||||
assert!(w.draft.is_some(), "the draft survives");
|
|
||||||
|
|
||||||
assert!(!w.intercept_close(true, &cfg), "still open: nothing to do");
|
/// A key capture in progress cannot outlive the tab: the app stops asking
|
||||||
|
/// [`SettingsTab::capturing_hotkey`] once another tab is up, and the button
|
||||||
w.draft = Some(cfg.clone());
|
/// must not be waiting when the tab comes back either.
|
||||||
assert!(!w.intercept_close(false, &cfg), "a clean close just closes");
|
#[test]
|
||||||
assert!(!w.open);
|
fn leaving_the_tab_ends_a_shortcut_capture() {
|
||||||
assert!(w.draft.is_none());
|
let mut w = SettingsTab::new();
|
||||||
|
let cfg = Config::default();
|
||||||
|
w.stage(&cfg);
|
||||||
|
w.capturing_hotkey = true;
|
||||||
|
w.discard();
|
||||||
|
assert!(!w.capturing_hotkey());
|
||||||
}
|
}
|
||||||
|
|
||||||
use crate::test_ui::{click_at, painted_text, painted_text_center};
|
use crate::test_ui::{click_at, painted_text, painted_text_center};
|
||||||
|
|
||||||
/// One frame of the shortcut control on its own, outside the window's
|
/// One frame of the shortcut control on its own, outside the tab's
|
||||||
/// scroll area so it is never below the fold.
|
/// scroll area so it is never below the fold.
|
||||||
fn run_hotkey_edit(
|
fn run_hotkey_edit(
|
||||||
ctx: &egui::Context,
|
ctx: &egui::Context,
|
||||||
|
|
@ -295,10 +307,11 @@ const ROWS: &[(Section, &str, &tips::Tip)] = &[
|
||||||
&tips::RESULTS_PER_PAGE,
|
&tips::RESULTS_PER_PAGE,
|
||||||
),
|
),
|
||||||
(Section::Search, "Debounce (ms)", &tips::DEBOUNCE),
|
(Section::Search, "Debounce (ms)", &tips::DEBOUNCE),
|
||||||
|
(Section::Search, "Live results", &tips::LIVE_RESULTS),
|
||||||
];
|
];
|
||||||
|
|
||||||
/// Hovering a row's name paints that row's own explanation. Rendered
|
/// Hovering a row's name paints that row's own explanation. Rendered
|
||||||
/// without the window's scroll area so nothing sits below the fold.
|
/// without the tab's scroll area so nothing sits below the fold.
|
||||||
#[test]
|
#[test]
|
||||||
fn every_row_shows_its_own_tip() {
|
fn every_row_shows_its_own_tip() {
|
||||||
for (section, label, tip) in ROWS {
|
for (section, label, tip) in ROWS {
|
||||||
|
|
@ -348,27 +361,20 @@ fn hovering_a_setting_label_explains_it() {
|
||||||
s.interaction.show_tooltips_only_when_still = false;
|
s.interaction.show_tooltips_only_when_still = false;
|
||||||
});
|
});
|
||||||
let cfg = Config::default();
|
let cfg = Config::default();
|
||||||
let mut w = OptionsWindow::new();
|
let mut w = SettingsTab::new();
|
||||||
w.open_with(&cfg);
|
|
||||||
|
|
||||||
let run = |w: &mut OptionsWindow, events: Vec<egui::Event>| {
|
let run = |w: &mut SettingsTab, events: Vec<egui::Event>| {
|
||||||
let input = crate::test_ui::raw_input(egui::vec2(1000.0, 900.0), events);
|
let input = crate::test_ui::raw_input(egui::vec2(1000.0, 900.0), events);
|
||||||
ctx.run(input, |ctx| {
|
ctx.run(input, |ctx| {
|
||||||
w.ui(ctx, &cfg);
|
egui::CentralPanel::default().show(ctx, |ui| {
|
||||||
|
w.ui(ui, &cfg);
|
||||||
|
});
|
||||||
})
|
})
|
||||||
};
|
};
|
||||||
|
|
||||||
// The window spends its first frames sizing itself and painting
|
|
||||||
// nothing; run until the label is on screen.
|
|
||||||
let mut target = None;
|
|
||||||
for _ in 0..5 {
|
|
||||||
let full = run(&mut w, vec![]);
|
let full = run(&mut w, vec![]);
|
||||||
target = painted_text_center(&full, "Tokenizer");
|
let target =
|
||||||
if target.is_some() {
|
painted_text_center(&full, "Tokenizer").expect("the Tokenizer label was not painted");
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
let target = target.expect("the Tokenizer label was not painted");
|
|
||||||
|
|
||||||
// The tooltip is an area of its own, so it can land a frame late.
|
// The tooltip is an area of its own, so it can land a frame late.
|
||||||
let mut out = run(&mut w, vec![egui::Event::PointerMoved(target)]);
|
let mut out = run(&mut w, vec![egui::Event::PointerMoved(target)]);
|
||||||
|
|
@ -384,37 +390,29 @@ fn hovering_a_setting_label_explains_it() {
|
||||||
panic!("no tooltip appeared over the Tokenizer label");
|
panic!("no tooltip appeared over the Tokenizer label");
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One real frame of the window in a headless context: it renders, and
|
/// One real frame of the tab in a headless context: it renders, and the
|
||||||
/// the Apply & Save click comes back out as `applied`.
|
/// Apply & Save click comes back out as `applied`.
|
||||||
#[test]
|
#[test]
|
||||||
fn the_window_renders_and_apply_reports_the_draft() {
|
fn the_tab_renders_and_apply_reports_the_draft() {
|
||||||
let ctx = egui::Context::default();
|
let ctx = egui::Context::default();
|
||||||
let cfg = Config::default();
|
let cfg = Config::default();
|
||||||
let mut w = OptionsWindow::new();
|
let mut w = SettingsTab::new();
|
||||||
w.open_with(&cfg);
|
w.stage(&cfg);
|
||||||
w.draft.as_mut().unwrap().search.debounce_ms += 100;
|
w.draft.as_mut().unwrap().search.debounce_ms += 100;
|
||||||
|
|
||||||
let run = |w: &mut OptionsWindow, events: Vec<egui::Event>| {
|
let run = |w: &mut SettingsTab, events: Vec<egui::Event>| {
|
||||||
let input = crate::test_ui::raw_input(egui::vec2(1000.0, 900.0), events);
|
let input = crate::test_ui::raw_input(egui::vec2(1000.0, 900.0), events);
|
||||||
let mut out = OptionsOutput::default();
|
let mut out = SettingsOutput::default();
|
||||||
let full = ctx.run(input, |ctx| out = w.ui(ctx, &cfg));
|
let full = ctx.run(input, |ctx| {
|
||||||
|
egui::CentralPanel::default().show(ctx, |ui| out = w.ui(ui, &cfg));
|
||||||
|
});
|
||||||
(out, full)
|
(out, full)
|
||||||
};
|
};
|
||||||
|
|
||||||
// A new egui window spends its first frames in sizing passes that
|
|
||||||
// suppress painting; run untouched frames until the settled button
|
|
||||||
// is actually on screen.
|
|
||||||
let mut target = None;
|
|
||||||
for _ in 0..5 {
|
|
||||||
let (untouched, full) = run(&mut w, vec![]);
|
let (untouched, full) = run(&mut w, vec![]);
|
||||||
assert!(untouched.applied.is_none());
|
assert!(untouched.applied.is_none());
|
||||||
assert!(!untouched.close_requested);
|
let target = painted_text_center(&full, "Apply & Save")
|
||||||
target = painted_text_center(&full, "Apply & Save");
|
.expect("the Apply & Save button was not painted");
|
||||||
if target.is_some() {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
let target = target.expect("the Apply & Save button was not painted");
|
|
||||||
let clicks = [true, false]
|
let clicks = [true, false]
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|pressed| egui::Event::PointerButton {
|
.map(|pressed| egui::Event::PointerButton {
|
||||||
|
|
@ -432,3 +430,156 @@ fn the_window_renders_and_apply_reports_the_draft() {
|
||||||
"the click reported the edited draft"
|
"the click reported the edited draft"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// One frame of the column picker on its own, outside the tab's scroll
|
||||||
|
/// area so it is never below the fold — the same shape as [`run_hotkey_edit`].
|
||||||
|
fn run_columns(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
current: &ColumnsConfig,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (Option<ColumnsConfig>, egui::FullOutput) {
|
||||||
|
let input = crate::test_ui::raw_input(egui::vec2(700.0, 200.0), events);
|
||||||
|
let mut picked = None;
|
||||||
|
let full = ctx.run(input, |ctx| {
|
||||||
|
egui::CentralPanel::default().show(ctx, |ui| {
|
||||||
|
picked = columns_ui(ui, current);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
(picked, full)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Settings copy of the column picker acts on the *live* config, not the
|
||||||
|
/// draft — the same arrangement the Security block uses, and the reason it and
|
||||||
|
/// the table header's menu cannot end up disagreeing. So it reports a change
|
||||||
|
/// the moment a box moves, with no Apply.
|
||||||
|
#[test]
|
||||||
|
fn the_columns_block_reports_a_change_immediately() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let current = ColumnsConfig::default();
|
||||||
|
assert!(!current.size, "the fixture assumes Size ships off");
|
||||||
|
|
||||||
|
let (quiet, full) = run_columns(&ctx, ¤t, vec![]);
|
||||||
|
assert!(quiet.is_none(), "reported a change nobody made");
|
||||||
|
let target = painted_text_center(&full, "Size").expect("no Size checkbox");
|
||||||
|
|
||||||
|
let (picked, _) = run_columns(&ctx, ¤t, click_at(target));
|
||||||
|
let picked = picked.expect("the click reported nothing");
|
||||||
|
assert!(picked.size, "clicking Size did not switch it on");
|
||||||
|
// Only that one moved.
|
||||||
|
assert_eq!(
|
||||||
|
picked,
|
||||||
|
ColumnsConfig {
|
||||||
|
size: true,
|
||||||
|
..current
|
||||||
|
}
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The path is not offered: it is the one column that identifies a result on
|
||||||
|
/// its own. It is shown checked and greyed rather than left out, so the
|
||||||
|
/// question "why can I not remove it?" has an answer on screen.
|
||||||
|
#[test]
|
||||||
|
fn the_columns_block_offers_every_column_but_the_path() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let (_, full) = run_columns(&ctx, &ColumnsConfig::default(), vec![]);
|
||||||
|
let painted = painted_text(&full);
|
||||||
|
for label in ["Name", "Path", "Content Match", "Size", "Modified", "Rank"] {
|
||||||
|
assert!(
|
||||||
|
painted.iter().any(|t| t == label),
|
||||||
|
"{label} missing: {painted:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clicking it does nothing, because it is disabled.
|
||||||
|
let target = painted_text_center(&full, "Path").expect("no Path entry");
|
||||||
|
let (picked, _) = run_columns(&ctx, &ColumnsConfig::default(), click_at(target));
|
||||||
|
assert!(picked.is_none(), "the path column was switched off");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One frame of the Security block on its own, in the shape of
|
||||||
|
/// [`run_columns`]. `keychain_active` is passed straight through, so nothing
|
||||||
|
/// here touches the OS keychain.
|
||||||
|
fn run_security(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
current: &Config,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (Option<SecurityAction>, egui::FullOutput) {
|
||||||
|
let input = crate::test_ui::raw_input(egui::vec2(700.0, 300.0), events);
|
||||||
|
let mut action = None;
|
||||||
|
let full = ctx.run(input, |ctx| {
|
||||||
|
egui::CentralPanel::default().show(ctx, |ui| {
|
||||||
|
action = security_ui(ui, current, false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
(action, full)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An unprotected index has no key at all, so there is nothing the button
|
||||||
|
/// could show and it is left out rather than shown dead.
|
||||||
|
#[test]
|
||||||
|
fn the_key_button_appears_only_while_the_index_is_encrypted() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut cfg = Config::default();
|
||||||
|
assert!(
|
||||||
|
!cfg.security.password_protected,
|
||||||
|
"the fixture assumes protection ships off"
|
||||||
|
);
|
||||||
|
|
||||||
|
let (_, full) = run_security(&ctx, &cfg, vec![]);
|
||||||
|
assert!(
|
||||||
|
painted_text_center(&full, "Show database key…").is_none(),
|
||||||
|
"offered the key of an unencrypted index: {:?}",
|
||||||
|
painted_text(&full)
|
||||||
|
);
|
||||||
|
|
||||||
|
cfg.security.password_protected = true;
|
||||||
|
let (_, full) = run_security(&ctx, &cfg, vec![]);
|
||||||
|
assert!(
|
||||||
|
painted_text_center(&full, "Show database key…").is_some(),
|
||||||
|
"no key button while encrypted: {:?}",
|
||||||
|
painted_text(&full)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The click only asks for the flow; the password confirmation and the reveal
|
||||||
|
/// both live in the app, so nothing about the key is decided here.
|
||||||
|
#[test]
|
||||||
|
fn clicking_the_key_button_reports_show_key() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let cfg = Config {
|
||||||
|
security: quicksearch_core::config::SecurityConfig {
|
||||||
|
password_protected: true,
|
||||||
|
..Default::default()
|
||||||
|
},
|
||||||
|
..Default::default()
|
||||||
|
};
|
||||||
|
|
||||||
|
let (quiet, full) = run_security(&ctx, &cfg, vec![]);
|
||||||
|
assert!(quiet.is_none(), "reported an action nobody clicked");
|
||||||
|
let target = painted_text_center(&full, "Show database key…").expect("no key button");
|
||||||
|
|
||||||
|
let (action, _) = run_security(&ctx, &cfg, click_at(target));
|
||||||
|
assert_eq!(action, Some(SecurityAction::ShowKey));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Columns are live state, so a draft taken before one changed must not carry
|
||||||
|
/// the old set back on Apply — `app::pin_live_fields` is what prevents that,
|
||||||
|
/// and this is the assertion that it covers this field.
|
||||||
|
#[test]
|
||||||
|
fn a_stale_draft_cannot_revert_the_columns() {
|
||||||
|
let mut w = SettingsTab::new();
|
||||||
|
let mut cfg = Config::default();
|
||||||
|
w.stage(&cfg);
|
||||||
|
|
||||||
|
// The header menu switches a column on while the tab is on screen.
|
||||||
|
cfg.search.columns.size = true;
|
||||||
|
assert!(!w.is_dirty(&cfg), "a live column change read as an edit");
|
||||||
|
|
||||||
|
let draft = w.draft_config().expect("a draft");
|
||||||
|
let mut applied = draft;
|
||||||
|
crate::app::pin_live_fields(&mut applied, &cfg);
|
||||||
|
assert!(
|
||||||
|
applied.search.columns.size,
|
||||||
|
"applying the stale draft reverted the column"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
@ -101,6 +101,31 @@ pub fn painted_spans(out: &egui::FullOutput) -> Vec<(String, egui::Color32)> {
|
||||||
.collect()
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Every styled run painted this frame that has a background behind it, with
|
||||||
|
/// that background, in paint order.
|
||||||
|
///
|
||||||
|
/// The distinguishing mark of a highlighted match: the column headers and the
|
||||||
|
/// strong parts of a snippet are painted in the same *text* color, so
|
||||||
|
/// [`painted_spans`] alone cannot tell a match from a header.
|
||||||
|
pub fn painted_backgrounds(out: &egui::FullOutput) -> Vec<(String, egui::Color32)> {
|
||||||
|
painted_galleys(out)
|
||||||
|
.into_iter()
|
||||||
|
.flat_map(|(g, _)| {
|
||||||
|
g.job
|
||||||
|
.sections
|
||||||
|
.iter()
|
||||||
|
.filter(|s| s.format.background != egui::Color32::TRANSPARENT)
|
||||||
|
.map(|s| {
|
||||||
|
(
|
||||||
|
g.job.text[s.byte_range.clone()].to_string(),
|
||||||
|
s.format.background,
|
||||||
|
)
|
||||||
|
})
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
/// Every *visible* row of every galley painted this frame, in paint order.
|
/// Every *visible* row of every galley painted this frame, in paint order.
|
||||||
/// Not the same as [`painted_text`]: a galley's `text()` is the job it was
|
/// Not the same as [`painted_text`]: a galley's `text()` is the job it was
|
||||||
/// laid out from, including the rows epaint dropped at `wrap.max_rows` —
|
/// laid out from, including the rows epaint dropped at `wrap.max_rows` —
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
//! Plain-language tooltips for the configuration controls: every setting in
|
//! Plain-language tooltips for the configuration controls: every setting in
|
||||||
//! the Options window, and every configuration control on the Manage Index
|
//! the Settings tab, and every configuration control on the Manage Index
|
||||||
//! tab, explains itself on hover.
|
//! tab, explains itself on hover.
|
||||||
|
|
||||||
/// How wide a tooltip may get; matches `manage_tab::db_size_tooltip`.
|
/// How wide a tooltip may get; matches `manage_tab::db_size_tooltip`.
|
||||||
|
|
@ -73,7 +73,7 @@ pub fn tip_row(
|
||||||
ui.end_row();
|
ui.end_row();
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Options: Paths ------------------------------------------------------
|
// --- Settings: Paths ------------------------------------------------------
|
||||||
|
|
||||||
pub static DATABASE_PATH: Tip = Tip {
|
pub static DATABASE_PATH: Tip = Tip {
|
||||||
title: "Database file",
|
title: "Database file",
|
||||||
|
|
@ -90,7 +90,7 @@ pub static DATABASE_PATH: Tip = Tip {
|
||||||
caution: None,
|
caution: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// --- Options: Indexing ---------------------------------------------------
|
// --- Settings: Indexing ---------------------------------------------------
|
||||||
|
|
||||||
pub static REINDEX_INTERVAL: Tip = Tip {
|
pub static REINDEX_INTERVAL: Tip = Tip {
|
||||||
title: "Full reindex every",
|
title: "Full reindex every",
|
||||||
|
|
@ -139,7 +139,7 @@ pub static INCLUDE_HIDDEN: Tip = Tip {
|
||||||
caution: None,
|
caution: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// --- Options: Processing -------------------------------------------------
|
// --- Settings: Processing -------------------------------------------------
|
||||||
|
|
||||||
pub static TOKENIZER: Tip = Tip {
|
pub static TOKENIZER: Tip = Tip {
|
||||||
title: "Tokenizer",
|
title: "Tokenizer",
|
||||||
|
|
@ -257,7 +257,7 @@ pub static STORE_TEXT: Tip = Tip {
|
||||||
caution: None,
|
caution: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// --- Options: Search -----------------------------------------------------
|
// --- Settings: Search -----------------------------------------------------
|
||||||
|
|
||||||
pub static FUZZY_DEFAULT: Tip = Tip {
|
pub static FUZZY_DEFAULT: Tip = Tip {
|
||||||
title: "Fuzzy search ON by default",
|
title: "Fuzzy search ON by default",
|
||||||
|
|
@ -329,7 +329,43 @@ pub static DEBOUNCE: Tip = Tip {
|
||||||
caution: None,
|
caution: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// --- Options: Interface --------------------------------------------------
|
pub static LIVE_RESULTS: Tip = Tip {
|
||||||
|
title: "Live results",
|
||||||
|
body: "Watches the results currently on screen and updates them as the \
|
||||||
|
files change, so a file you rename or edit in another window does \
|
||||||
|
not sit there showing its old name or its old text.\n\n\
|
||||||
|
Only the rows you can actually see are watched, and every one of \
|
||||||
|
them is dropped the moment you change the search. Nothing is ever \
|
||||||
|
added, removed or re-sorted while you read — a file that stops \
|
||||||
|
matching stays where it is until you search again.",
|
||||||
|
examples: &[
|
||||||
|
"Renames, deletions and edited contents all show up within a second, \
|
||||||
|
whatever the indexer is doing — what a row says is read from the \
|
||||||
|
file, not from the index.",
|
||||||
|
"A row is also checked against the disk as it comes on screen, so one \
|
||||||
|
the index was out of date about puts itself right. That check is all \
|
||||||
|
you get over a network share, where the system does not report other \
|
||||||
|
machines' writes.",
|
||||||
|
],
|
||||||
|
caution: Some(
|
||||||
|
"The files behind the rows you are looking at are kept up to date in \
|
||||||
|
the index too, even while indexing is stopped. Turn this off if a \
|
||||||
|
stopped index must mean nothing is written at all.",
|
||||||
|
),
|
||||||
|
};
|
||||||
|
|
||||||
|
pub static COLUMNS: Tip = Tip {
|
||||||
|
title: "Search columns",
|
||||||
|
body: "Which columns the results table shows. The path is always there — \
|
||||||
|
it is the only column that identifies a result on its own.\n\n\
|
||||||
|
Size and modified date start switched off: the width they take is \
|
||||||
|
usually better spent on the path and the matched text. Turning a \
|
||||||
|
column on also makes it available to sort by.",
|
||||||
|
examples: &["Right-clicking any column header on the Search tab does the same thing."],
|
||||||
|
caution: None,
|
||||||
|
};
|
||||||
|
|
||||||
|
// --- Settings: Interface --------------------------------------------------
|
||||||
|
|
||||||
pub static UI_SCALE: Tip = Tip {
|
pub static UI_SCALE: Tip = Tip {
|
||||||
title: "UI scale",
|
title: "UI scale",
|
||||||
|
|
@ -380,7 +416,7 @@ pub static COLOR_SCHEME: Tip = Tip {
|
||||||
caution: None,
|
caution: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// --- Options: Security ---------------------------------------------------
|
// --- Settings: Security ---------------------------------------------------
|
||||||
|
|
||||||
pub static ENABLE_PASSWORD: Tip = Tip {
|
pub static ENABLE_PASSWORD: Tip = Tip {
|
||||||
title: "Enable password protection",
|
title: "Enable password protection",
|
||||||
|
|
@ -419,6 +455,21 @@ pub static DISABLE_PASSWORD: Tip = Tip {
|
||||||
),
|
),
|
||||||
};
|
};
|
||||||
|
|
||||||
|
pub static SHOW_KEY: Tip = Tip {
|
||||||
|
title: "Show database key",
|
||||||
|
body: "Reveals the raw SQLCipher key the index is encrypted with, once you \
|
||||||
|
have confirmed your password. Tools such as DB Browser for SQLCipher \
|
||||||
|
accept it in the 0x form shown and can then open the index file \
|
||||||
|
directly.\n\n\
|
||||||
|
The key is worked out from your password and the salt in the config \
|
||||||
|
file, so it stays the same until the password changes.",
|
||||||
|
examples: &[],
|
||||||
|
caution: Some(
|
||||||
|
"Anyone holding this key can read the index without the password. Treat a copy \
|
||||||
|
of it as carefully as the password itself.",
|
||||||
|
),
|
||||||
|
};
|
||||||
|
|
||||||
pub static REMEMBER_KEYCHAIN: Tip = Tip {
|
pub static REMEMBER_KEYCHAIN: Tip = Tip {
|
||||||
title: "Remember on this device",
|
title: "Remember on this device",
|
||||||
body: "Hands the key to the password store your system already has, such \
|
body: "Hands the key to the password store your system already has, such \
|
||||||
|
|
@ -616,6 +667,7 @@ mod tests {
|
||||||
&ENABLE_PASSWORD,
|
&ENABLE_PASSWORD,
|
||||||
&CHANGE_PASSWORD,
|
&CHANGE_PASSWORD,
|
||||||
&DISABLE_PASSWORD,
|
&DISABLE_PASSWORD,
|
||||||
|
&SHOW_KEY,
|
||||||
&REMEMBER_KEYCHAIN,
|
&REMEMBER_KEYCHAIN,
|
||||||
&START_NOW,
|
&START_NOW,
|
||||||
&STOP_INDEXING,
|
&STOP_INDEXING,
|
||||||
|
|
|
||||||
441
crates/quicksearch-gui/src/tutorial.rs
Normal file
441
crates/quicksearch-gui/src/tutorial.rs
Normal file
|
|
@ -0,0 +1,441 @@
|
||||||
|
//! The first-start tour: a few pages explaining what QuickSearch indexes,
|
||||||
|
//! how results are ranked, and what the parts of the Search tab do.
|
||||||
|
//!
|
||||||
|
//! Shown once, to an installation that has never run before — `[ui]
|
||||||
|
//! tutorial_seen` is `Some(false)` only in a config file this version created,
|
||||||
|
//! so upgrading into this version does not summon it. The Help tab can bring
|
||||||
|
//! it back afterwards, which is also what keeps this from being write-only.
|
||||||
|
|
||||||
|
use crate::ui_util::{centered_modal, hint};
|
||||||
|
|
||||||
|
/// One page of the tour. Static text, so the pages are a table rather than a
|
||||||
|
/// match arm each.
|
||||||
|
struct Page {
|
||||||
|
title: &'static str,
|
||||||
|
/// Paragraphs. Rendered in order with a little space between them.
|
||||||
|
body: &'static [&'static str],
|
||||||
|
/// Rendered small and de-emphasised under the body — where to go, rather
|
||||||
|
/// than what the thing is.
|
||||||
|
pointer: Option<&'static str>,
|
||||||
|
}
|
||||||
|
|
||||||
|
const PAGES: &[Page] = &[
|
||||||
|
Page {
|
||||||
|
title: "Welcome to QuickSearch",
|
||||||
|
body: &[
|
||||||
|
"QuickSearch keeps an index of the folders you choose, and searches \
|
||||||
|
as you type.",
|
||||||
|
"By default only your user folder is indexed and searchable.",
|
||||||
|
"Because the answers come from the index rather than from reading \
|
||||||
|
your disk, results appear as fast as you can type, even across \
|
||||||
|
hundreds of thousands of files.",
|
||||||
|
],
|
||||||
|
pointer: Some("A quick tutorial for new users! Hit Skip to Exit."),
|
||||||
|
},
|
||||||
|
Page {
|
||||||
|
title: "What is indexed",
|
||||||
|
body: &[
|
||||||
|
"Indexing is QuickSearch reading through your folders once and \
|
||||||
|
remembering what it found, so that searching later is instant. It \
|
||||||
|
runs on its own in the background and keeps up with changes as you \
|
||||||
|
make them.",
|
||||||
|
"QuickSearch never connects to the internet, and always respects your privacy. \
|
||||||
|
QuickSearch can encrypt your index to make this remembered data more secure.",
|
||||||
|
"These are the folders being indexed right now:",
|
||||||
|
],
|
||||||
|
pointer: Some(
|
||||||
|
"To index more locations, open the Manage Index tab and add a folder. \
|
||||||
|
To set an index password, look near the bottom of the Settings tab.",
|
||||||
|
),
|
||||||
|
},
|
||||||
|
Page {
|
||||||
|
title: "How results are ranked",
|
||||||
|
body: &[
|
||||||
|
"The best search matches come first (have the lowest rank). \
|
||||||
|
Exact file-name matchs are best, then names that contain what you typed; then \
|
||||||
|
files whose contents contain the search terms, the ones mentioning it most often \
|
||||||
|
first; and last, files matched only by their full folder path.",
|
||||||
|
"The coloured number in the Rank column is which of those tiers a \
|
||||||
|
result came from: blue is a great match, red is a distant one. \
|
||||||
|
Clicking a column heading sorts by something else instead.",
|
||||||
|
],
|
||||||
|
pointer: Some("Right-click any column heading to choose which columns are shown."),
|
||||||
|
},
|
||||||
|
Page {
|
||||||
|
title: "The status bar",
|
||||||
|
body: &[
|
||||||
|
"The line along the bottom of the window is what QuickSearch is \
|
||||||
|
doing. While it is indexing it shows the phase, how far through it \
|
||||||
|
is, and how fast; when it has nothing to do it shows how many files \
|
||||||
|
are indexed.",
|
||||||
|
"Searching works the whole time, including during that first indexing run, \
|
||||||
|
but some files might not be shown in the results until the scan completes.",
|
||||||
|
],
|
||||||
|
pointer: None,
|
||||||
|
},
|
||||||
|
Page {
|
||||||
|
title: "Typos, and what a result can do",
|
||||||
|
body: &[
|
||||||
|
"Tick \"Fuzzy\"beside the search box to also match words with typos \
|
||||||
|
in them — \"repot\" will find \"report\". It searches more \
|
||||||
|
thoroughly, so it is a little slower; leave it off until you need \
|
||||||
|
it.",
|
||||||
|
"Right-click any result for more: open it, open the folder holding \
|
||||||
|
it, copy its path, or build a filter that hides files like it from \
|
||||||
|
future searches.",
|
||||||
|
],
|
||||||
|
pointer: Some(
|
||||||
|
"The ? button left of the search box lists the filters you can type \
|
||||||
|
into a query, like type:Document or modified:>=2024-01-01.",
|
||||||
|
),
|
||||||
|
},
|
||||||
|
Page {
|
||||||
|
title: "Duplicates",
|
||||||
|
body: &[
|
||||||
|
"The Duplicates tab looks for files across all indexed folders for identical copies. \
|
||||||
|
They are shown grouped together, with the largest wasted space first.",
|
||||||
|
"It is a quick way to find the same download sitting in three \
|
||||||
|
places. QuickSearch only shows you the groups; deleting anything is \
|
||||||
|
left to you.",
|
||||||
|
],
|
||||||
|
pointer: Some(
|
||||||
|
"For speed, files are compared by size and by how they begin (first 8KB). \
|
||||||
|
This is not a guarantee of an exact match. You can right click a result to verify before you delete anything.",
|
||||||
|
),
|
||||||
|
},
|
||||||
|
Page {
|
||||||
|
title: "Settings",
|
||||||
|
body: &[
|
||||||
|
"The Settings tab, at the right-hand end of the tab strip, is where \
|
||||||
|
you can tweak and tune the software. Mouse over any of \
|
||||||
|
the settings for a brief description of what they do.",
|
||||||
|
"Most changes wait for the Apply & Save button at the bottom.",
|
||||||
|
"QuickSearch is completely free for anyone to use. If you love it, please let your friends know about us!",
|
||||||
|
],
|
||||||
|
pointer: None,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
/// The open tour.
|
||||||
|
pub struct Tutorial {
|
||||||
|
page: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Tutorial {
|
||||||
|
pub fn new() -> Tutorial {
|
||||||
|
Tutorial { page: 0 }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Render. `roots` is the live indexed-folder list, named on the page
|
||||||
|
/// about indexing so the tour describes this installation rather than a
|
||||||
|
/// generic one.
|
||||||
|
/// Returns true once the tour is finished with — skipped or read to the
|
||||||
|
/// end — which is the caller's cue to remember that and drop it.
|
||||||
|
pub fn ui(&mut self, ctx: &egui::Context, roots: &[String]) -> bool {
|
||||||
|
let page = &PAGES[self.page.min(PAGES.len() - 1)];
|
||||||
|
let (first, last) = (self.page == 0, self.page + 1 == PAGES.len());
|
||||||
|
let mut dismissed = false;
|
||||||
|
let mut step: i64 = 0;
|
||||||
|
|
||||||
|
centered_modal(ctx, page.title, |ui| {
|
||||||
|
ui.set_max_width(520.0);
|
||||||
|
for paragraph in page.body {
|
||||||
|
ui.label(*paragraph);
|
||||||
|
ui.add_space(6.0);
|
||||||
|
}
|
||||||
|
// The one page that shows live state rather than static text.
|
||||||
|
if self.page == 1 {
|
||||||
|
if roots.is_empty() {
|
||||||
|
ui.label(hint("No folders are indexed yet."));
|
||||||
|
} else {
|
||||||
|
for root in roots {
|
||||||
|
ui.monospace(root);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ui.add_space(6.0);
|
||||||
|
}
|
||||||
|
if let Some(pointer) = page.pointer {
|
||||||
|
ui.label(hint(pointer));
|
||||||
|
}
|
||||||
|
|
||||||
|
ui.add_space(10.0);
|
||||||
|
ui.separator();
|
||||||
|
// Three equal thirds rather than one row: it is the only layout
|
||||||
|
// that puts Skip in the middle without measuring the buttons
|
||||||
|
// either side of it, whose widths change with the page ("Next"
|
||||||
|
// becoming "Finish") and with the counter's digits.
|
||||||
|
ui.columns(3, |cols| {
|
||||||
|
// A column lays its contents out *justified*, so a button put
|
||||||
|
// straight into one is stretched to the full third. The other
|
||||||
|
// two escape that by nesting their own layout; this one has to
|
||||||
|
// say so.
|
||||||
|
cols[0].with_layout(egui::Layout::left_to_right(egui::Align::Min), |ui| {
|
||||||
|
if ui.add_enabled(!first, egui::Button::new("Back")).clicked() {
|
||||||
|
step = -1;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
cols[1].vertical_centered(|ui| {
|
||||||
|
if ui.button("Skip").clicked() {
|
||||||
|
dismissed = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// `Align::Min`, not `Center`: a column is as tall as the rest
|
||||||
|
// of the window, so centring in it drops the button a hundred
|
||||||
|
// points below the two beside it.
|
||||||
|
cols[2].with_layout(egui::Layout::right_to_left(egui::Align::Min), |ui| {
|
||||||
|
let p = crate::color::palette(ui.visuals().dark_mode);
|
||||||
|
let next = if last { "Finish" } else { "Next" };
|
||||||
|
if ui
|
||||||
|
.add(crate::ui_util::bordered_button(next, p.blue))
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
if last {
|
||||||
|
dismissed = true;
|
||||||
|
} else {
|
||||||
|
step = 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ui.label(hint(format!("{} of {}", self.page + 1, PAGES.len())));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Applied after the closure so the page a frame rendered stays the page
|
||||||
|
// its buttons were laid out for.
|
||||||
|
if step != 0 {
|
||||||
|
let next = self.page as i64 + step;
|
||||||
|
self.page = next.clamp(0, PAGES.len() as i64 - 1) as usize;
|
||||||
|
}
|
||||||
|
dismissed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::test_ui::{click_at, painted_text, raw_input};
|
||||||
|
|
||||||
|
const SCREEN: egui::Vec2 = egui::vec2(1000.0, 700.0);
|
||||||
|
|
||||||
|
/// Two passes: an `egui::Window` is measured on its first frame and only
|
||||||
|
/// placed on the next, so a single pass paints nothing to read back and
|
||||||
|
/// has nothing at a known position to click.
|
||||||
|
fn frame(
|
||||||
|
ctx: &egui::Context,
|
||||||
|
tour: &mut Tutorial,
|
||||||
|
events: Vec<egui::Event>,
|
||||||
|
) -> (egui::FullOutput, bool) {
|
||||||
|
let roots = ["/home/me".to_string()];
|
||||||
|
let _ = ctx.run(raw_input(SCREEN, Vec::new()), |ctx| {
|
||||||
|
tour.ui(ctx, &roots);
|
||||||
|
});
|
||||||
|
let mut dismissed = false;
|
||||||
|
let out = ctx.run(raw_input(SCREEN, events), |ctx| {
|
||||||
|
dismissed = tour.ui(ctx, &roots);
|
||||||
|
});
|
||||||
|
(out, dismissed)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every page has something to say, and says it.
|
||||||
|
#[test]
|
||||||
|
fn every_page_paints_its_own_title_and_body() {
|
||||||
|
for (page, spec) in PAGES.iter().enumerate() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tour = Tutorial { page };
|
||||||
|
let (out, _) = frame(&ctx, &mut tour, Vec::new());
|
||||||
|
let painted = painted_text(&out);
|
||||||
|
assert!(
|
||||||
|
painted.iter().any(|t| t == spec.title),
|
||||||
|
"page {page} painted no title: {painted:?}"
|
||||||
|
);
|
||||||
|
assert!(!spec.body.is_empty(), "page {page} has an empty body");
|
||||||
|
assert!(
|
||||||
|
painted
|
||||||
|
.iter()
|
||||||
|
.any(|t| t == &format!("{} of {}", page + 1, PAGES.len())),
|
||||||
|
"page {page} did not say where it is: {painted:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The page about indexing names *this* installation's folders, not a
|
||||||
|
/// generic example.
|
||||||
|
#[test]
|
||||||
|
fn the_indexing_page_lists_the_configured_folders() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tour = Tutorial { page: 1 };
|
||||||
|
let roots = ["/srv/projects".to_string()];
|
||||||
|
let _ = ctx.run(raw_input(SCREEN, Vec::new()), |ctx| {
|
||||||
|
tour.ui(ctx, &roots);
|
||||||
|
});
|
||||||
|
let out = ctx.run(raw_input(SCREEN, Vec::new()), |ctx| {
|
||||||
|
tour.ui(ctx, &roots);
|
||||||
|
});
|
||||||
|
let painted = painted_text(&out);
|
||||||
|
assert!(painted.iter().any(|t| t == "/srv/projects"), "{painted:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The y of the footer row on `page`, found by walking down the middle
|
||||||
|
/// column, which only Skip occupies. Wrapped text height moves the row
|
||||||
|
/// from page to page, so it is probed rather than guessed.
|
||||||
|
fn footer_y(ctx: &egui::Context, page: usize) -> f32 {
|
||||||
|
for y in (150..600).step_by(2) {
|
||||||
|
let mut t = Tutorial { page };
|
||||||
|
let (_, dismissed) = frame(ctx, &mut t, click_at(egui::pos2(500.0, y as f32)));
|
||||||
|
if dismissed {
|
||||||
|
return y as f32;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
panic!("no Skip button down the middle of page {page}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The stretch of x along the footer row that one button answers a click
|
||||||
|
/// on — where it is, and how wide.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
struct Span {
|
||||||
|
lo: f32,
|
||||||
|
hi: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Span {
|
||||||
|
fn width(&self) -> f32 {
|
||||||
|
self.hi - self.lo
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The three footer buttons, found by what clicking each one does: Back
|
||||||
|
/// steps a page back, Next steps one forward, Skip dismisses.
|
||||||
|
///
|
||||||
|
/// Must be run on a middle page — on the last page Finish and Skip both
|
||||||
|
/// dismiss without moving, and on the first Back is disabled.
|
||||||
|
fn footer_spans(ctx: &egui::Context, page: usize) -> [Option<Span>; 3] {
|
||||||
|
assert!(page > 0 && page + 1 < PAGES.len(), "probe a middle page");
|
||||||
|
let y = footer_y(ctx, page);
|
||||||
|
let mut spans: [Option<Span>; 3] = [None; 3];
|
||||||
|
for x in 150..850 {
|
||||||
|
let mut t = Tutorial { page };
|
||||||
|
let (_, dismissed) = frame(ctx, &mut t, click_at(egui::pos2(x as f32, y)));
|
||||||
|
let x = x as f32;
|
||||||
|
let which = if dismissed {
|
||||||
|
1 // Skip
|
||||||
|
} else if t.page + 1 == page {
|
||||||
|
0 // Back
|
||||||
|
} else if t.page == page + 1 {
|
||||||
|
2 // Next
|
||||||
|
} else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
match &mut spans[which] {
|
||||||
|
Some(span) => span.hi = x,
|
||||||
|
slot => *slot = Some(Span { lo: x, hi: x }),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
spans
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The footer reads Back, then Skip, then Next — and each does what its
|
||||||
|
/// label says. Positions are probed rather than asserted against numbers:
|
||||||
|
/// the window auto-sizes to the page's text, so the thirds move.
|
||||||
|
///
|
||||||
|
/// The widths are the other half of it. `Ui::columns` lays a column out
|
||||||
|
/// justified, so a button dropped straight into one comes out as wide as
|
||||||
|
/// the whole third — which is what Back was until it was given a layout of
|
||||||
|
/// its own. Two buttons with four-letter labels either side of the row
|
||||||
|
/// have to come out the same size.
|
||||||
|
#[test]
|
||||||
|
fn the_footer_runs_back_then_skip_then_next_at_the_same_size() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let [back, skip, next] = footer_spans(&ctx, 1);
|
||||||
|
let back = back.expect("no Back button in the footer");
|
||||||
|
let skip = skip.expect("no Skip button in the footer");
|
||||||
|
let next = next.expect("no Next button in the footer");
|
||||||
|
assert!(
|
||||||
|
back.lo < skip.lo,
|
||||||
|
"Back ({back:?}) is not left of Skip ({skip:?})"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
skip.lo < next.lo,
|
||||||
|
"Skip ({skip:?}) is not left of Next ({next:?})"
|
||||||
|
);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
(back.width() - next.width()).abs() <= 2.0,
|
||||||
|
"Back is {} wide against Next's {}",
|
||||||
|
back.width(),
|
||||||
|
next.width()
|
||||||
|
);
|
||||||
|
// Belt and braces on the shape of the bug: a stretched button fills
|
||||||
|
// its third of a 520-point modal, which no four-letter label does.
|
||||||
|
assert!(
|
||||||
|
back.width() < 80.0,
|
||||||
|
"Back is stretched to {} points",
|
||||||
|
back.width()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Back is disabled on the first page, so nothing in the footer can walk
|
||||||
|
/// the tour off the front.
|
||||||
|
#[test]
|
||||||
|
fn the_first_page_cannot_go_back() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let y = footer_y(&ctx, 0);
|
||||||
|
for x in (150..850).step_by(4) {
|
||||||
|
let mut t = Tutorial { page: 0 };
|
||||||
|
let _ = frame(&ctx, &mut t, click_at(egui::pos2(x as f32, y)));
|
||||||
|
assert!(
|
||||||
|
t.page == 0 || t.page == 1,
|
||||||
|
"clicking x={x} left page {}",
|
||||||
|
t.page
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Clicking anywhere in the button row, on any page; collects what fired.
|
||||||
|
fn sweep(ctx: &egui::Context, tour: &mut Tutorial) -> Vec<usize> {
|
||||||
|
let mut seen = Vec::new();
|
||||||
|
for y in (200..500).step_by(4) {
|
||||||
|
for x in (240..760).step_by(8) {
|
||||||
|
let mut t = Tutorial { page: tour.page };
|
||||||
|
let (_, dismissed) = frame(ctx, &mut t, click_at(egui::pos2(x as f32, y as f32)));
|
||||||
|
if dismissed {
|
||||||
|
seen.push(t.page);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
seen
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Both ways out of the tour report the dismissal, so the flag gets set
|
||||||
|
/// whichever the user takes. Positions depend on wrapped text height, so
|
||||||
|
/// the button row is swept rather than guessed at — the same approach the
|
||||||
|
/// confirmation modals' tests take.
|
||||||
|
#[test]
|
||||||
|
fn skip_and_finish_both_dismiss() {
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
|
||||||
|
// Skip is on every page.
|
||||||
|
let mut tour = Tutorial { page: 0 };
|
||||||
|
assert!(
|
||||||
|
!sweep(&ctx, &mut tour).is_empty(),
|
||||||
|
"Skip never fired on the first page"
|
||||||
|
);
|
||||||
|
|
||||||
|
// Finish only on the last, where it replaces Next.
|
||||||
|
let mut tour = Tutorial {
|
||||||
|
page: PAGES.len() - 1,
|
||||||
|
};
|
||||||
|
assert!(
|
||||||
|
!sweep(&ctx, &mut tour).is_empty(),
|
||||||
|
"Finish never fired on the last page"
|
||||||
|
);
|
||||||
|
let ctx = egui::Context::default();
|
||||||
|
let mut tour = Tutorial {
|
||||||
|
page: PAGES.len() - 1,
|
||||||
|
};
|
||||||
|
let (out, _) = frame(&ctx, &mut tour, Vec::new());
|
||||||
|
assert!(
|
||||||
|
painted_text(&out).contains(&"Finish".to_string()),
|
||||||
|
"the last page still offers Next"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -122,8 +122,10 @@ pub fn pattern_edit(
|
||||||
(response, valid)
|
(response, valid)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Middle-elide `text` so it fits `max_width` pixels when laid out in
|
/// Where a middle-elide cuts `text` to fit `max_width` pixels in `font_id`:
|
||||||
/// `font_id`, returning it borrowed and untouched when it already fits.
|
/// the byte offset the head keeps up to, and the one the tail resumes from,
|
||||||
|
/// with a single `…` standing for everything between. `None` when the whole
|
||||||
|
/// string fits and nothing is dropped.
|
||||||
///
|
///
|
||||||
/// The budget is in pixels, summed from the font's own glyph advances (the
|
/// The budget is in pixels, summed from the font's own glyph advances (the
|
||||||
/// same numbers egui's layout adds up), not a character count scaled by one
|
/// same numbers egui's layout adds up), not a character count scaled by one
|
||||||
|
|
@ -131,18 +133,20 @@ pub fn pattern_edit(
|
||||||
/// directions: overshoot and egui elides the result a *second* time,
|
/// directions: overshoot and egui elides the result a *second* time,
|
||||||
/// painting two ellipses; undershoot and the column sits visibly short.
|
/// painting two ellipses; undershoot and the column sits visibly short.
|
||||||
///
|
///
|
||||||
/// The borrowed/owned distinction is the caller's signal that something was
|
/// Split out from [`middle_elide`] because a caller that also has *ranges* to
|
||||||
/// dropped, which is what a "full text on hover" tooltip keys off.
|
/// highlight needs the cut itself, not just the shortened string: it renders
|
||||||
pub fn middle_elide<'a>(
|
/// the two surviving ends separately so its marks keep the offsets they had
|
||||||
|
/// (see `snippet_render::path_cell_job`).
|
||||||
|
pub fn middle_elide_cut(
|
||||||
ui: &egui::Ui,
|
ui: &egui::Ui,
|
||||||
text: &'a str,
|
text: &str,
|
||||||
max_width: f32,
|
max_width: f32,
|
||||||
font_id: &egui::FontId,
|
font_id: &egui::FontId,
|
||||||
) -> Cow<'a, str> {
|
) -> Option<(usize, usize)> {
|
||||||
ui.fonts(|f| {
|
ui.fonts(|f| {
|
||||||
let width_of = |c: char| f.glyph_width(font_id, c);
|
let width_of = |c: char| f.glyph_width(font_id, c);
|
||||||
if text.chars().map(width_of).sum::<f32>() <= max_width {
|
if text.chars().map(width_of).sum::<f32>() <= max_width {
|
||||||
return Cow::Borrowed(text);
|
return None;
|
||||||
}
|
}
|
||||||
let budget = max_width - width_of('…');
|
let budget = max_width - width_of('…');
|
||||||
|
|
||||||
|
|
@ -178,15 +182,33 @@ pub fn middle_elide<'a>(
|
||||||
}
|
}
|
||||||
if head >= tail {
|
if head >= tail {
|
||||||
// The two halves met without dropping anything.
|
// The two halves met without dropping anything.
|
||||||
return Cow::Borrowed(text);
|
return None;
|
||||||
}
|
}
|
||||||
|
Some((head, tail))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Middle-elide `text` so it fits `max_width` pixels when laid out in
|
||||||
|
/// `font_id`, returning it borrowed and untouched when it already fits.
|
||||||
|
///
|
||||||
|
/// The borrowed/owned distinction is the caller's signal that something was
|
||||||
|
/// dropped, which is what a "full text on hover" tooltip keys off.
|
||||||
|
pub fn middle_elide<'a>(
|
||||||
|
ui: &egui::Ui,
|
||||||
|
text: &'a str,
|
||||||
|
max_width: f32,
|
||||||
|
font_id: &egui::FontId,
|
||||||
|
) -> Cow<'a, str> {
|
||||||
|
match middle_elide_cut(ui, text, max_width, font_id) {
|
||||||
|
None => Cow::Borrowed(text),
|
||||||
|
Some((head, tail)) => {
|
||||||
let mut out = String::with_capacity(head + '…'.len_utf8() + (text.len() - tail));
|
let mut out = String::with_capacity(head + '…'.len_utf8() + (text.len() - tail));
|
||||||
out.push_str(&text[..head]);
|
out.push_str(&text[..head]);
|
||||||
out.push('…');
|
out.push('…');
|
||||||
out.push_str(&text[tail..]);
|
out.push_str(&text[tail..]);
|
||||||
Cow::Owned(out)
|
Cow::Owned(out)
|
||||||
})
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Paint a semitransparent down-arrow near the bottom edge of a scroll
|
/// Paint a semitransparent down-arrow near the bottom edge of a scroll
|
||||||
|
|
|
||||||
|
|
@ -72,12 +72,12 @@ impl Gate {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if let Gate::Running(app) = self {
|
if let Gate::Running(app) = self {
|
||||||
// The Options window is waiting for a key press to bind; the
|
// The Settings tab is waiting for a key press to bind; the
|
||||||
// shortcut must not reshuffle the window underneath that dialog.
|
// shortcut must not reshuffle the window underneath it.
|
||||||
if app.capturing_hotkey() {
|
if app.capturing_hotkey() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
app.activate_search();
|
app.activate_search(ctx);
|
||||||
}
|
}
|
||||||
crate::hotkey::raise(ctx, frame);
|
crate::hotkey::raise(ctx, frame);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@
|
||||||
# "capture"). Edit this file to re-choreograph the captures; timings are in
|
# "capture"). Edit this file to re-choreograph the captures; timings are in
|
||||||
# milliseconds and `max` caps bound clip length without failing the run.
|
# milliseconds and `max` caps bound clip length without failing the run.
|
||||||
|
|
||||||
# The demo config sets [ui] scale = 1.5, so every window size here is 1.5x
|
# The demo config sets [ui] scale = 1.25, so every window size here is 1.25x
|
||||||
# the layout it shows: high-resolution captures of an unchanged layout.
|
# the layout it shows: high-resolution captures of an unchanged layout.
|
||||||
|
|
||||||
# --- 1. manage-indexing.webm: the fresh auto-index in progress -------------
|
# --- 1. manage-indexing.webm: the fresh auto-index in progress -------------
|
||||||
|
|
@ -26,8 +26,12 @@ focus_search # tab switches drop egui focus; re-arm it
|
||||||
clear_query
|
clear_query
|
||||||
window 1200 600 # compact clip: the smallest layout at
|
window 1200 600 # compact clip: the smallest layout at
|
||||||
# which every results column still fits
|
# which every results column still fits
|
||||||
# (any narrower clips the Match column
|
# (any narrower clips the Content Match
|
||||||
# away, defeating the demo), at 1.5x
|
# column away, defeating the demo), at
|
||||||
|
# 1.25x. Sized for the four columns the
|
||||||
|
# demo config pins — Name, Path, Content
|
||||||
|
# Match, Rank — so turning another one on
|
||||||
|
# there means revisiting this width.
|
||||||
wait_ms 800 # the resize lands asynchronously
|
wait_ms 800 # the resize lands asynchronously
|
||||||
record_start search
|
record_start search
|
||||||
# The query is typed in quick bursts with human hesitations; each pause
|
# The query is typed in quick bursts with human hesitations; each pause
|
||||||
|
|
@ -42,8 +46,8 @@ wait_ms 300
|
||||||
type "ition" cps 10
|
type "ition" cps 10
|
||||||
wait_search_done max 8000
|
wait_search_done max 8000
|
||||||
wait_ms 1000
|
wait_ms 1000
|
||||||
hover_match 2 # pin the pointer on the 3rd result's Match
|
hover_match 2 # pin the pointer on the 3rd result's Content
|
||||||
# cell: the tooltip expands the snippet
|
# Match cell: the tooltip expands the snippet
|
||||||
# with surrounding file content
|
# with surrounding file content
|
||||||
wait_ms 2500 # tooltip delay, then linger on it
|
wait_ms 2500 # tooltip delay, then linger on it
|
||||||
record_stop # end the clip with the tooltip on screen
|
record_stop # end the clip with the tooltip on screen
|
||||||
|
|
|
||||||
|
|
@ -68,8 +68,34 @@ auto_index = true
|
||||||
include_hidden = true
|
include_hidden = true
|
||||||
ignore_patterns = []
|
ignore_patterns = []
|
||||||
|
|
||||||
# 1.5x zoom + proportionally larger windows (set in the scenario) render the
|
[search]
|
||||||
# same layout at ~1.5x the pixel density, for crisper website assets.
|
# Off for the run: on, the watcher arms over the visible rows mid-recording,
|
||||||
|
# re-reads them from disk and hands the paths back for reindexing — index
|
||||||
|
# writes inside a clip that is supposed to be reproducible.
|
||||||
|
live_results = false
|
||||||
|
|
||||||
|
[search.columns]
|
||||||
|
# Exactly the four the captures are choreographed around: Name, Path, Content
|
||||||
|
# Match, Rank. Every one is pinned rather than defaulted — the scenario's
|
||||||
|
# window widths are chosen for this column set, and `hover_match` addresses
|
||||||
|
# Content Match cells by index, so a shipped default that moved would silently
|
||||||
|
# re-frame or break the clips.
|
||||||
|
#
|
||||||
|
# Path is absent because it is not optional: it is always drawn (see
|
||||||
|
# `ColumnsConfig`, which deliberately does not represent it).
|
||||||
|
name = true
|
||||||
|
content_match = true
|
||||||
|
rank = true
|
||||||
|
size = false
|
||||||
|
modified = false
|
||||||
|
|
||||||
|
# 1.25x zoom + proportionally larger windows (set in the scenario) render the
|
||||||
|
# same layout at ~1.25x the pixel density, for crisper website assets.
|
||||||
|
#
|
||||||
|
# No `tutorial_seen` key here, deliberately: absent it deserializes to `None`,
|
||||||
|
# which reads as "an installation that upgraded into this version". Writing
|
||||||
|
# `false` would open the first-run tour over the first capture and wedge the
|
||||||
|
# run.
|
||||||
[ui]
|
[ui]
|
||||||
scale = 1.25
|
scale = 1.25
|
||||||
EOF
|
EOF
|
||||||
|
|
|
||||||
|
|
@ -136,7 +136,7 @@ text stays part of the search phrase.
|
||||||
.BR AND ", " OR
|
.BR AND ", " OR
|
||||||
and parentheses are treated as plain words.
|
and parentheses are treated as plain words.
|
||||||
.SH PASSWORD PROTECTION
|
.SH PASSWORD PROTECTION
|
||||||
The index can be encrypted with a password (application Options, Security).
|
The index can be encrypted with a password (the Settings tab, Security).
|
||||||
A protected index must be unlocked every time either binary starts. The
|
A protected index must be unlocked every time either binary starts. The
|
||||||
application shows an unlock screen; terminal mode resolves the key from, in
|
application shows an unlock screen; terminal mode resolves the key from, in
|
||||||
order: the OS keychain (when \(lqRemember on this device\(rq is enabled),
|
order: the OS keychain (when \(lqRemember on this device\(rq is enabled),
|
||||||
|
|
|
||||||
|
|
@ -61,6 +61,18 @@ VIAddVersionKey "FileDescription" "${APP} ${VERSION} installer"
|
||||||
!insertmacro MUI_PAGE_COMPONENTS
|
!insertmacro MUI_PAGE_COMPONENTS
|
||||||
!insertmacro MUI_PAGE_DIRECTORY
|
!insertmacro MUI_PAGE_DIRECTORY
|
||||||
!insertmacro MUI_PAGE_INSTFILES
|
!insertmacro MUI_PAGE_INSTFILES
|
||||||
|
; The payload is two binaries and three documents, so this install finishes in
|
||||||
|
; about a second - fast enough that people reported it as a failure. The finish
|
||||||
|
; page is the only place that can say otherwise, so it lists what was installed
|
||||||
|
; and where instead of the stock "Setup was completed successfully."
|
||||||
|
; Kept to six lines: the finish page's text field is a fixed height, and MUI
|
||||||
|
; clips rather than scrolls what does not fit.
|
||||||
|
!define MUI_FINISHPAGE_TITLE "${APP} ${VERSION} is installed"
|
||||||
|
!define MUI_FINISHPAGE_TEXT "Installed into $INSTDIR:$\r$\n\
|
||||||
|
quicksearch.exe (the app), quicksearch-cli.exe (terminal search),$\r$\n\
|
||||||
|
README.md, LICENSE.txt and config_example.toml.$\r$\n$\r$\n\
|
||||||
|
Your settings and search index are created on first run, under your \
|
||||||
|
own account. Upgrading and uninstalling leave both alone."
|
||||||
!define MUI_FINISHPAGE_RUN
|
!define MUI_FINISHPAGE_RUN
|
||||||
!define MUI_FINISHPAGE_RUN_TEXT "Run ${APP}"
|
!define MUI_FINISHPAGE_RUN_TEXT "Run ${APP}"
|
||||||
!define MUI_FINISHPAGE_RUN_FUNCTION LaunchAsUser
|
!define MUI_FINISHPAGE_RUN_FUNCTION LaunchAsUser
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue