Added GPL license, deb packaging script, complete GUI rewrite, added CLI mode, better default config options, config now lives in user folder, reduced syscalls by 5x.
6
.gitattributes
vendored
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
# cmd.exe is sensitive to line endings — a run.bat checked out with LF can
|
||||||
|
# misparse labels and multi-line constructs — while sh requires LF regardless
|
||||||
|
# of the platform it is checked out on.
|
||||||
|
*.bat text eol=crlf
|
||||||
|
*.cmd text eol=crlf
|
||||||
|
*.sh text eol=lf
|
||||||
1
.gitignore
vendored
|
|
@ -1,3 +1,4 @@
|
||||||
/target
|
/target
|
||||||
|
/dist
|
||||||
*.db
|
*.db
|
||||||
config.toml
|
config.toml
|
||||||
4
.vscode/launch.json
vendored
|
|
@ -12,7 +12,7 @@
|
||||||
"args": [
|
"args": [
|
||||||
"build",
|
"build",
|
||||||
"--bin=quicksearch",
|
"--bin=quicksearch",
|
||||||
"--package=quicksearch"
|
"--package=quicksearch-gui"
|
||||||
],
|
],
|
||||||
"filter": {
|
"filter": {
|
||||||
"name": "quicksearch",
|
"name": "quicksearch",
|
||||||
|
|
@ -31,7 +31,7 @@
|
||||||
"test",
|
"test",
|
||||||
"--no-run",
|
"--no-run",
|
||||||
"--bin=quicksearch",
|
"--bin=quicksearch",
|
||||||
"--package=quicksearch"
|
"--package=quicksearch-gui"
|
||||||
],
|
],
|
||||||
"filter": {
|
"filter": {
|
||||||
"name": "quicksearch",
|
"name": "quicksearch",
|
||||||
|
|
|
||||||
4335
Cargo.lock
generated
|
|
@ -6,5 +6,8 @@ members = [
|
||||||
]
|
]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
version = "0.1.0"
|
version = "0.8.0"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
|
license = "GPL-3.0-or-later"
|
||||||
|
authors = ["Jeremy <jeremy@karsttech.com>"]
|
||||||
|
repository = "https://code.karsttech.com/jeremy/quick_search.git"
|
||||||
|
|
|
||||||
674
LICENSE
Normal file
|
|
@ -0,0 +1,674 @@
|
||||||
|
GNU GENERAL PUBLIC LICENSE
|
||||||
|
Version 3, 29 June 2007
|
||||||
|
|
||||||
|
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||||
|
Everyone is permitted to copy and distribute verbatim copies
|
||||||
|
of this license document, but changing it is not allowed.
|
||||||
|
|
||||||
|
Preamble
|
||||||
|
|
||||||
|
The GNU General Public License is a free, copyleft license for
|
||||||
|
software and other kinds of works.
|
||||||
|
|
||||||
|
The licenses for most software and other practical works are designed
|
||||||
|
to take away your freedom to share and change the works. By contrast,
|
||||||
|
the GNU General Public License is intended to guarantee your freedom to
|
||||||
|
share and change all versions of a program--to make sure it remains free
|
||||||
|
software for all its users. We, the Free Software Foundation, use the
|
||||||
|
GNU General Public License for most of our software; it applies also to
|
||||||
|
any other work released this way by its authors. You can apply it to
|
||||||
|
your programs, too.
|
||||||
|
|
||||||
|
When we speak of free software, we are referring to freedom, not
|
||||||
|
price. Our General Public Licenses are designed to make sure that you
|
||||||
|
have the freedom to distribute copies of free software (and charge for
|
||||||
|
them if you wish), that you receive source code or can get it if you
|
||||||
|
want it, that you can change the software or use pieces of it in new
|
||||||
|
free programs, and that you know you can do these things.
|
||||||
|
|
||||||
|
To protect your rights, we need to prevent others from denying you
|
||||||
|
these rights or asking you to surrender the rights. Therefore, you have
|
||||||
|
certain responsibilities if you distribute copies of the software, or if
|
||||||
|
you modify it: responsibilities to respect the freedom of others.
|
||||||
|
|
||||||
|
For example, if you distribute copies of such a program, whether
|
||||||
|
gratis or for a fee, you must pass on to the recipients the same
|
||||||
|
freedoms that you received. You must make sure that they, too, receive
|
||||||
|
or can get the source code. And you must show them these terms so they
|
||||||
|
know their rights.
|
||||||
|
|
||||||
|
Developers that use the GNU GPL protect your rights with two steps:
|
||||||
|
(1) assert copyright on the software, and (2) offer you this License
|
||||||
|
giving you legal permission to copy, distribute and/or modify it.
|
||||||
|
|
||||||
|
For the developers' and authors' protection, the GPL clearly explains
|
||||||
|
that there is no warranty for this free software. For both users' and
|
||||||
|
authors' sake, the GPL requires that modified versions be marked as
|
||||||
|
changed, so that their problems will not be attributed erroneously to
|
||||||
|
authors of previous versions.
|
||||||
|
|
||||||
|
Some devices are designed to deny users access to install or run
|
||||||
|
modified versions of the software inside them, although the manufacturer
|
||||||
|
can do so. This is fundamentally incompatible with the aim of
|
||||||
|
protecting users' freedom to change the software. The systematic
|
||||||
|
pattern of such abuse occurs in the area of products for individuals to
|
||||||
|
use, which is precisely where it is most unacceptable. Therefore, we
|
||||||
|
have designed this version of the GPL to prohibit the practice for those
|
||||||
|
products. If such problems arise substantially in other domains, we
|
||||||
|
stand ready to extend this provision to those domains in future versions
|
||||||
|
of the GPL, as needed to protect the freedom of users.
|
||||||
|
|
||||||
|
Finally, every program is threatened constantly by software patents.
|
||||||
|
States should not allow patents to restrict development and use of
|
||||||
|
software on general-purpose computers, but in those that do, we wish to
|
||||||
|
avoid the special danger that patents applied to a free program could
|
||||||
|
make it effectively proprietary. To prevent this, the GPL assures that
|
||||||
|
patents cannot be used to render the program non-free.
|
||||||
|
|
||||||
|
The precise terms and conditions for copying, distribution and
|
||||||
|
modification follow.
|
||||||
|
|
||||||
|
TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
0. Definitions.
|
||||||
|
|
||||||
|
"This License" refers to version 3 of the GNU General Public License.
|
||||||
|
|
||||||
|
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||||
|
works, such as semiconductor masks.
|
||||||
|
|
||||||
|
"The Program" refers to any copyrightable work licensed under this
|
||||||
|
License. Each licensee is addressed as "you". "Licensees" and
|
||||||
|
"recipients" may be individuals or organizations.
|
||||||
|
|
||||||
|
To "modify" a work means to copy from or adapt all or part of the work
|
||||||
|
in a fashion requiring copyright permission, other than the making of an
|
||||||
|
exact copy. The resulting work is called a "modified version" of the
|
||||||
|
earlier work or a work "based on" the earlier work.
|
||||||
|
|
||||||
|
A "covered work" means either the unmodified Program or a work based
|
||||||
|
on the Program.
|
||||||
|
|
||||||
|
To "propagate" a work means to do anything with it that, without
|
||||||
|
permission, would make you directly or secondarily liable for
|
||||||
|
infringement under applicable copyright law, except executing it on a
|
||||||
|
computer or modifying a private copy. Propagation includes copying,
|
||||||
|
distribution (with or without modification), making available to the
|
||||||
|
public, and in some countries other activities as well.
|
||||||
|
|
||||||
|
To "convey" a work means any kind of propagation that enables other
|
||||||
|
parties to make or receive copies. Mere interaction with a user through
|
||||||
|
a computer network, with no transfer of a copy, is not conveying.
|
||||||
|
|
||||||
|
An interactive user interface displays "Appropriate Legal Notices"
|
||||||
|
to the extent that it includes a convenient and prominently visible
|
||||||
|
feature that (1) displays an appropriate copyright notice, and (2)
|
||||||
|
tells the user that there is no warranty for the work (except to the
|
||||||
|
extent that warranties are provided), that licensees may convey the
|
||||||
|
work under this License, and how to view a copy of this License. If
|
||||||
|
the interface presents a list of user commands or options, such as a
|
||||||
|
menu, a prominent item in the list meets this criterion.
|
||||||
|
|
||||||
|
1. Source Code.
|
||||||
|
|
||||||
|
The "source code" for a work means the preferred form of the work
|
||||||
|
for making modifications to it. "Object code" means any non-source
|
||||||
|
form of a work.
|
||||||
|
|
||||||
|
A "Standard Interface" means an interface that either is an official
|
||||||
|
standard defined by a recognized standards body, or, in the case of
|
||||||
|
interfaces specified for a particular programming language, one that
|
||||||
|
is widely used among developers working in that language.
|
||||||
|
|
||||||
|
The "System Libraries" of an executable work include anything, other
|
||||||
|
than the work as a whole, that (a) is included in the normal form of
|
||||||
|
packaging a Major Component, but which is not part of that Major
|
||||||
|
Component, and (b) serves only to enable use of the work with that
|
||||||
|
Major Component, or to implement a Standard Interface for which an
|
||||||
|
implementation is available to the public in source code form. A
|
||||||
|
"Major Component", in this context, means a major essential component
|
||||||
|
(kernel, window system, and so on) of the specific operating system
|
||||||
|
(if any) on which the executable work runs, or a compiler used to
|
||||||
|
produce the work, or an object code interpreter used to run it.
|
||||||
|
|
||||||
|
The "Corresponding Source" for a work in object code form means all
|
||||||
|
the source code needed to generate, install, and (for an executable
|
||||||
|
work) run the object code and to modify the work, including scripts to
|
||||||
|
control those activities. However, it does not include the work's
|
||||||
|
System Libraries, or general-purpose tools or generally available free
|
||||||
|
programs which are used unmodified in performing those activities but
|
||||||
|
which are not part of the work. For example, Corresponding Source
|
||||||
|
includes interface definition files associated with source files for
|
||||||
|
the work, and the source code for shared libraries and dynamically
|
||||||
|
linked subprograms that the work is specifically designed to require,
|
||||||
|
such as by intimate data communication or control flow between those
|
||||||
|
subprograms and other parts of the work.
|
||||||
|
|
||||||
|
The Corresponding Source need not include anything that users
|
||||||
|
can regenerate automatically from other parts of the Corresponding
|
||||||
|
Source.
|
||||||
|
|
||||||
|
The Corresponding Source for a work in source code form is that
|
||||||
|
same work.
|
||||||
|
|
||||||
|
2. Basic Permissions.
|
||||||
|
|
||||||
|
All rights granted under this License are granted for the term of
|
||||||
|
copyright on the Program, and are irrevocable provided the stated
|
||||||
|
conditions are met. This License explicitly affirms your unlimited
|
||||||
|
permission to run the unmodified Program. The output from running a
|
||||||
|
covered work is covered by this License only if the output, given its
|
||||||
|
content, constitutes a covered work. This License acknowledges your
|
||||||
|
rights of fair use or other equivalent, as provided by copyright law.
|
||||||
|
|
||||||
|
You may make, run and propagate covered works that you do not
|
||||||
|
convey, without conditions so long as your license otherwise remains
|
||||||
|
in force. You may convey covered works to others for the sole purpose
|
||||||
|
of having them make modifications exclusively for you, or provide you
|
||||||
|
with facilities for running those works, provided that you comply with
|
||||||
|
the terms of this License in conveying all material for which you do
|
||||||
|
not control copyright. Those thus making or running the covered works
|
||||||
|
for you must do so exclusively on your behalf, under your direction
|
||||||
|
and control, on terms that prohibit them from making any copies of
|
||||||
|
your copyrighted material outside their relationship with you.
|
||||||
|
|
||||||
|
Conveying under any other circumstances is permitted solely under
|
||||||
|
the conditions stated below. Sublicensing is not allowed; section 10
|
||||||
|
makes it unnecessary.
|
||||||
|
|
||||||
|
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||||
|
|
||||||
|
No covered work shall be deemed part of an effective technological
|
||||||
|
measure under any applicable law fulfilling obligations under article
|
||||||
|
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||||
|
similar laws prohibiting or restricting circumvention of such
|
||||||
|
measures.
|
||||||
|
|
||||||
|
When you convey a covered work, you waive any legal power to forbid
|
||||||
|
circumvention of technological measures to the extent such circumvention
|
||||||
|
is effected by exercising rights under this License with respect to
|
||||||
|
the covered work, and you disclaim any intention to limit operation or
|
||||||
|
modification of the work as a means of enforcing, against the work's
|
||||||
|
users, your or third parties' legal rights to forbid circumvention of
|
||||||
|
technological measures.
|
||||||
|
|
||||||
|
4. Conveying Verbatim Copies.
|
||||||
|
|
||||||
|
You may convey verbatim copies of the Program's source code as you
|
||||||
|
receive it, in any medium, provided that you conspicuously and
|
||||||
|
appropriately publish on each copy an appropriate copyright notice;
|
||||||
|
keep intact all notices stating that this License and any
|
||||||
|
non-permissive terms added in accord with section 7 apply to the code;
|
||||||
|
keep intact all notices of the absence of any warranty; and give all
|
||||||
|
recipients a copy of this License along with the Program.
|
||||||
|
|
||||||
|
You may charge any price or no price for each copy that you convey,
|
||||||
|
and you may offer support or warranty protection for a fee.
|
||||||
|
|
||||||
|
5. Conveying Modified Source Versions.
|
||||||
|
|
||||||
|
You may convey a work based on the Program, or the modifications to
|
||||||
|
produce it from the Program, in the form of source code under the
|
||||||
|
terms of section 4, provided that you also meet all of these conditions:
|
||||||
|
|
||||||
|
a) The work must carry prominent notices stating that you modified
|
||||||
|
it, and giving a relevant date.
|
||||||
|
|
||||||
|
b) The work must carry prominent notices stating that it is
|
||||||
|
released under this License and any conditions added under section
|
||||||
|
7. This requirement modifies the requirement in section 4 to
|
||||||
|
"keep intact all notices".
|
||||||
|
|
||||||
|
c) You must license the entire work, as a whole, under this
|
||||||
|
License to anyone who comes into possession of a copy. This
|
||||||
|
License will therefore apply, along with any applicable section 7
|
||||||
|
additional terms, to the whole of the work, and all its parts,
|
||||||
|
regardless of how they are packaged. This License gives no
|
||||||
|
permission to license the work in any other way, but it does not
|
||||||
|
invalidate such permission if you have separately received it.
|
||||||
|
|
||||||
|
d) If the work has interactive user interfaces, each must display
|
||||||
|
Appropriate Legal Notices; however, if the Program has interactive
|
||||||
|
interfaces that do not display Appropriate Legal Notices, your
|
||||||
|
work need not make them do so.
|
||||||
|
|
||||||
|
A compilation of a covered work with other separate and independent
|
||||||
|
works, which are not by their nature extensions of the covered work,
|
||||||
|
and which are not combined with it such as to form a larger program,
|
||||||
|
in or on a volume of a storage or distribution medium, is called an
|
||||||
|
"aggregate" if the compilation and its resulting copyright are not
|
||||||
|
used to limit the access or legal rights of the compilation's users
|
||||||
|
beyond what the individual works permit. Inclusion of a covered work
|
||||||
|
in an aggregate does not cause this License to apply to the other
|
||||||
|
parts of the aggregate.
|
||||||
|
|
||||||
|
6. Conveying Non-Source Forms.
|
||||||
|
|
||||||
|
You may convey a covered work in object code form under the terms
|
||||||
|
of sections 4 and 5, provided that you also convey the
|
||||||
|
machine-readable Corresponding Source under the terms of this License,
|
||||||
|
in one of these ways:
|
||||||
|
|
||||||
|
a) Convey the object code in, or embodied in, a physical product
|
||||||
|
(including a physical distribution medium), accompanied by the
|
||||||
|
Corresponding Source fixed on a durable physical medium
|
||||||
|
customarily used for software interchange.
|
||||||
|
|
||||||
|
b) Convey the object code in, or embodied in, a physical product
|
||||||
|
(including a physical distribution medium), accompanied by a
|
||||||
|
written offer, valid for at least three years and valid for as
|
||||||
|
long as you offer spare parts or customer support for that product
|
||||||
|
model, to give anyone who possesses the object code either (1) a
|
||||||
|
copy of the Corresponding Source for all the software in the
|
||||||
|
product that is covered by this License, on a durable physical
|
||||||
|
medium customarily used for software interchange, for a price no
|
||||||
|
more than your reasonable cost of physically performing this
|
||||||
|
conveying of source, or (2) access to copy the
|
||||||
|
Corresponding Source from a network server at no charge.
|
||||||
|
|
||||||
|
c) Convey individual copies of the object code with a copy of the
|
||||||
|
written offer to provide the Corresponding Source. This
|
||||||
|
alternative is allowed only occasionally and noncommercially, and
|
||||||
|
only if you received the object code with such an offer, in accord
|
||||||
|
with subsection 6b.
|
||||||
|
|
||||||
|
d) Convey the object code by offering access from a designated
|
||||||
|
place (gratis or for a charge), and offer equivalent access to the
|
||||||
|
Corresponding Source in the same way through the same place at no
|
||||||
|
further charge. You need not require recipients to copy the
|
||||||
|
Corresponding Source along with the object code. If the place to
|
||||||
|
copy the object code is a network server, the Corresponding Source
|
||||||
|
may be on a different server (operated by you or a third party)
|
||||||
|
that supports equivalent copying facilities, provided you maintain
|
||||||
|
clear directions next to the object code saying where to find the
|
||||||
|
Corresponding Source. Regardless of what server hosts the
|
||||||
|
Corresponding Source, you remain obligated to ensure that it is
|
||||||
|
available for as long as needed to satisfy these requirements.
|
||||||
|
|
||||||
|
e) Convey the object code using peer-to-peer transmission, provided
|
||||||
|
you inform other peers where the object code and Corresponding
|
||||||
|
Source of the work are being offered to the general public at no
|
||||||
|
charge under subsection 6d.
|
||||||
|
|
||||||
|
A separable portion of the object code, whose source code is excluded
|
||||||
|
from the Corresponding Source as a System Library, need not be
|
||||||
|
included in conveying the object code work.
|
||||||
|
|
||||||
|
A "User Product" is either (1) a "consumer product", which means any
|
||||||
|
tangible personal property which is normally used for personal, family,
|
||||||
|
or household purposes, or (2) anything designed or sold for incorporation
|
||||||
|
into a dwelling. In determining whether a product is a consumer product,
|
||||||
|
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||||
|
product received by a particular user, "normally used" refers to a
|
||||||
|
typical or common use of that class of product, regardless of the status
|
||||||
|
of the particular user or of the way in which the particular user
|
||||||
|
actually uses, or expects or is expected to use, the product. A product
|
||||||
|
is a consumer product regardless of whether the product has substantial
|
||||||
|
commercial, industrial or non-consumer uses, unless such uses represent
|
||||||
|
the only significant mode of use of the product.
|
||||||
|
|
||||||
|
"Installation Information" for a User Product means any methods,
|
||||||
|
procedures, authorization keys, or other information required to install
|
||||||
|
and execute modified versions of a covered work in that User Product from
|
||||||
|
a modified version of its Corresponding Source. The information must
|
||||||
|
suffice to ensure that the continued functioning of the modified object
|
||||||
|
code is in no case prevented or interfered with solely because
|
||||||
|
modification has been made.
|
||||||
|
|
||||||
|
If you convey an object code work under this section in, or with, or
|
||||||
|
specifically for use in, a User Product, and the conveying occurs as
|
||||||
|
part of a transaction in which the right of possession and use of the
|
||||||
|
User Product is transferred to the recipient in perpetuity or for a
|
||||||
|
fixed term (regardless of how the transaction is characterized), the
|
||||||
|
Corresponding Source conveyed under this section must be accompanied
|
||||||
|
by the Installation Information. But this requirement does not apply
|
||||||
|
if neither you nor any third party retains the ability to install
|
||||||
|
modified object code on the User Product (for example, the work has
|
||||||
|
been installed in ROM).
|
||||||
|
|
||||||
|
The requirement to provide Installation Information does not include a
|
||||||
|
requirement to continue to provide support service, warranty, or updates
|
||||||
|
for a work that has been modified or installed by the recipient, or for
|
||||||
|
the User Product in which it has been modified or installed. Access to a
|
||||||
|
network may be denied when the modification itself materially and
|
||||||
|
adversely affects the operation of the network or violates the rules and
|
||||||
|
protocols for communication across the network.
|
||||||
|
|
||||||
|
Corresponding Source conveyed, and Installation Information provided,
|
||||||
|
in accord with this section must be in a format that is publicly
|
||||||
|
documented (and with an implementation available to the public in
|
||||||
|
source code form), and must require no special password or key for
|
||||||
|
unpacking, reading or copying.
|
||||||
|
|
||||||
|
7. Additional Terms.
|
||||||
|
|
||||||
|
"Additional permissions" are terms that supplement the terms of this
|
||||||
|
License by making exceptions from one or more of its conditions.
|
||||||
|
Additional permissions that are applicable to the entire Program shall
|
||||||
|
be treated as though they were included in this License, to the extent
|
||||||
|
that they are valid under applicable law. If additional permissions
|
||||||
|
apply only to part of the Program, that part may be used separately
|
||||||
|
under those permissions, but the entire Program remains governed by
|
||||||
|
this License without regard to the additional permissions.
|
||||||
|
|
||||||
|
When you convey a copy of a covered work, you may at your option
|
||||||
|
remove any additional permissions from that copy, or from any part of
|
||||||
|
it. (Additional permissions may be written to require their own
|
||||||
|
removal in certain cases when you modify the work.) You may place
|
||||||
|
additional permissions on material, added by you to a covered work,
|
||||||
|
for which you have or can give appropriate copyright permission.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, for material you
|
||||||
|
add to a covered work, you may (if authorized by the copyright holders of
|
||||||
|
that material) supplement the terms of this License with terms:
|
||||||
|
|
||||||
|
a) Disclaiming warranty or limiting liability differently from the
|
||||||
|
terms of sections 15 and 16 of this License; or
|
||||||
|
|
||||||
|
b) Requiring preservation of specified reasonable legal notices or
|
||||||
|
author attributions in that material or in the Appropriate Legal
|
||||||
|
Notices displayed by works containing it; or
|
||||||
|
|
||||||
|
c) Prohibiting misrepresentation of the origin of that material, or
|
||||||
|
requiring that modified versions of such material be marked in
|
||||||
|
reasonable ways as different from the original version; or
|
||||||
|
|
||||||
|
d) Limiting the use for publicity purposes of names of licensors or
|
||||||
|
authors of the material; or
|
||||||
|
|
||||||
|
e) Declining to grant rights under trademark law for use of some
|
||||||
|
trade names, trademarks, or service marks; or
|
||||||
|
|
||||||
|
f) Requiring indemnification of licensors and authors of that
|
||||||
|
material by anyone who conveys the material (or modified versions of
|
||||||
|
it) with contractual assumptions of liability to the recipient, for
|
||||||
|
any liability that these contractual assumptions directly impose on
|
||||||
|
those licensors and authors.
|
||||||
|
|
||||||
|
All other non-permissive additional terms are considered "further
|
||||||
|
restrictions" within the meaning of section 10. If the Program as you
|
||||||
|
received it, or any part of it, contains a notice stating that it is
|
||||||
|
governed by this License along with a term that is a further
|
||||||
|
restriction, you may remove that term. If a license document contains
|
||||||
|
a further restriction but permits relicensing or conveying under this
|
||||||
|
License, you may add to a covered work material governed by the terms
|
||||||
|
of that license document, provided that the further restriction does
|
||||||
|
not survive such relicensing or conveying.
|
||||||
|
|
||||||
|
If you add terms to a covered work in accord with this section, you
|
||||||
|
must place, in the relevant source files, a statement of the
|
||||||
|
additional terms that apply to those files, or a notice indicating
|
||||||
|
where to find the applicable terms.
|
||||||
|
|
||||||
|
Additional terms, permissive or non-permissive, may be stated in the
|
||||||
|
form of a separately written license, or stated as exceptions;
|
||||||
|
the above requirements apply either way.
|
||||||
|
|
||||||
|
8. Termination.
|
||||||
|
|
||||||
|
You may not propagate or modify a covered work except as expressly
|
||||||
|
provided under this License. Any attempt otherwise to propagate or
|
||||||
|
modify it is void, and will automatically terminate your rights under
|
||||||
|
this License (including any patent licenses granted under the third
|
||||||
|
paragraph of section 11).
|
||||||
|
|
||||||
|
However, if you cease all violation of this License, then your
|
||||||
|
license from a particular copyright holder is reinstated (a)
|
||||||
|
provisionally, unless and until the copyright holder explicitly and
|
||||||
|
finally terminates your license, and (b) permanently, if the copyright
|
||||||
|
holder fails to notify you of the violation by some reasonable means
|
||||||
|
prior to 60 days after the cessation.
|
||||||
|
|
||||||
|
Moreover, your license from a particular copyright holder is
|
||||||
|
reinstated permanently if the copyright holder notifies you of the
|
||||||
|
violation by some reasonable means, this is the first time you have
|
||||||
|
received notice of violation of this License (for any work) from that
|
||||||
|
copyright holder, and you cure the violation prior to 30 days after
|
||||||
|
your receipt of the notice.
|
||||||
|
|
||||||
|
Termination of your rights under this section does not terminate the
|
||||||
|
licenses of parties who have received copies or rights from you under
|
||||||
|
this License. If your rights have been terminated and not permanently
|
||||||
|
reinstated, you do not qualify to receive new licenses for the same
|
||||||
|
material under section 10.
|
||||||
|
|
||||||
|
9. Acceptance Not Required for Having Copies.
|
||||||
|
|
||||||
|
You are not required to accept this License in order to receive or
|
||||||
|
run a copy of the Program. Ancillary propagation of a covered work
|
||||||
|
occurring solely as a consequence of using peer-to-peer transmission
|
||||||
|
to receive a copy likewise does not require acceptance. However,
|
||||||
|
nothing other than this License grants you permission to propagate or
|
||||||
|
modify any covered work. These actions infringe copyright if you do
|
||||||
|
not accept this License. Therefore, by modifying or propagating a
|
||||||
|
covered work, you indicate your acceptance of this License to do so.
|
||||||
|
|
||||||
|
10. Automatic Licensing of Downstream Recipients.
|
||||||
|
|
||||||
|
Each time you convey a covered work, the recipient automatically
|
||||||
|
receives a license from the original licensors, to run, modify and
|
||||||
|
propagate that work, subject to this License. You are not responsible
|
||||||
|
for enforcing compliance by third parties with this License.
|
||||||
|
|
||||||
|
An "entity transaction" is a transaction transferring control of an
|
||||||
|
organization, or substantially all assets of one, or subdividing an
|
||||||
|
organization, or merging organizations. If propagation of a covered
|
||||||
|
work results from an entity transaction, each party to that
|
||||||
|
transaction who receives a copy of the work also receives whatever
|
||||||
|
licenses to the work the party's predecessor in interest had or could
|
||||||
|
give under the previous paragraph, plus a right to possession of the
|
||||||
|
Corresponding Source of the work from the predecessor in interest, if
|
||||||
|
the predecessor has it or can get it with reasonable efforts.
|
||||||
|
|
||||||
|
You may not impose any further restrictions on the exercise of the
|
||||||
|
rights granted or affirmed under this License. For example, you may
|
||||||
|
not impose a license fee, royalty, or other charge for exercise of
|
||||||
|
rights granted under this License, and you may not initiate litigation
|
||||||
|
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||||
|
any patent claim is infringed by making, using, selling, offering for
|
||||||
|
sale, or importing the Program or any portion of it.
|
||||||
|
|
||||||
|
11. Patents.
|
||||||
|
|
||||||
|
A "contributor" is a copyright holder who authorizes use under this
|
||||||
|
License of the Program or a work on which the Program is based. The
|
||||||
|
work thus licensed is called the contributor's "contributor version".
|
||||||
|
|
||||||
|
A contributor's "essential patent claims" are all patent claims
|
||||||
|
owned or controlled by the contributor, whether already acquired or
|
||||||
|
hereafter acquired, that would be infringed by some manner, permitted
|
||||||
|
by this License, of making, using, or selling its contributor version,
|
||||||
|
but do not include claims that would be infringed only as a
|
||||||
|
consequence of further modification of the contributor version. For
|
||||||
|
purposes of this definition, "control" includes the right to grant
|
||||||
|
patent sublicenses in a manner consistent with the requirements of
|
||||||
|
this License.
|
||||||
|
|
||||||
|
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||||
|
patent license under the contributor's essential patent claims, to
|
||||||
|
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||||
|
propagate the contents of its contributor version.
|
||||||
|
|
||||||
|
In the following three paragraphs, a "patent license" is any express
|
||||||
|
agreement or commitment, however denominated, not to enforce a patent
|
||||||
|
(such as an express permission to practice a patent or covenant not to
|
||||||
|
sue for patent infringement). To "grant" such a patent license to a
|
||||||
|
party means to make such an agreement or commitment not to enforce a
|
||||||
|
patent against the party.
|
||||||
|
|
||||||
|
If you convey a covered work, knowingly relying on a patent license,
|
||||||
|
and the Corresponding Source of the work is not available for anyone
|
||||||
|
to copy, free of charge and under the terms of this License, through a
|
||||||
|
publicly available network server or other readily accessible means,
|
||||||
|
then you must either (1) cause the Corresponding Source to be so
|
||||||
|
available, or (2) arrange to deprive yourself of the benefit of the
|
||||||
|
patent license for this particular work, or (3) arrange, in a manner
|
||||||
|
consistent with the requirements of this License, to extend the patent
|
||||||
|
license to downstream recipients. "Knowingly relying" means you have
|
||||||
|
actual knowledge that, but for the patent license, your conveying the
|
||||||
|
covered work in a country, or your recipient's use of the covered work
|
||||||
|
in a country, would infringe one or more identifiable patents in that
|
||||||
|
country that you have reason to believe are valid.
|
||||||
|
|
||||||
|
If, pursuant to or in connection with a single transaction or
|
||||||
|
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||||
|
covered work, and grant a patent license to some of the parties
|
||||||
|
receiving the covered work authorizing them to use, propagate, modify
|
||||||
|
or convey a specific copy of the covered work, then the patent license
|
||||||
|
you grant is automatically extended to all recipients of the covered
|
||||||
|
work and works based on it.
|
||||||
|
|
||||||
|
A patent license is "discriminatory" if it does not include within
|
||||||
|
the scope of its coverage, prohibits the exercise of, or is
|
||||||
|
conditioned on the non-exercise of one or more of the rights that are
|
||||||
|
specifically granted under this License. You may not convey a covered
|
||||||
|
work if you are a party to an arrangement with a third party that is
|
||||||
|
in the business of distributing software, under which you make payment
|
||||||
|
to the third party based on the extent of your activity of conveying
|
||||||
|
the work, and under which the third party grants, to any of the
|
||||||
|
parties who would receive the covered work from you, a discriminatory
|
||||||
|
patent license (a) in connection with copies of the covered work
|
||||||
|
conveyed by you (or copies made from those copies), or (b) primarily
|
||||||
|
for and in connection with specific products or compilations that
|
||||||
|
contain the covered work, unless you entered into that arrangement,
|
||||||
|
or that patent license was granted, prior to 28 March 2007.
|
||||||
|
|
||||||
|
Nothing in this License shall be construed as excluding or limiting
|
||||||
|
any implied license or other defenses to infringement that may
|
||||||
|
otherwise be available to you under applicable patent law.
|
||||||
|
|
||||||
|
12. No Surrender of Others' Freedom.
|
||||||
|
|
||||||
|
If conditions are imposed on you (whether by court order, agreement or
|
||||||
|
otherwise) that contradict the conditions of this License, they do not
|
||||||
|
excuse you from the conditions of this License. If you cannot convey a
|
||||||
|
covered work so as to satisfy simultaneously your obligations under this
|
||||||
|
License and any other pertinent obligations, then as a consequence you may
|
||||||
|
not convey it at all. For example, if you agree to terms that obligate you
|
||||||
|
to collect a royalty for further conveying from those to whom you convey
|
||||||
|
the Program, the only way you could satisfy both those terms and this
|
||||||
|
License would be to refrain entirely from conveying the Program.
|
||||||
|
|
||||||
|
13. Use with the GNU Affero General Public License.
|
||||||
|
|
||||||
|
Notwithstanding any other provision of this License, you have
|
||||||
|
permission to link or combine any covered work with a work licensed
|
||||||
|
under version 3 of the GNU Affero General Public License into a single
|
||||||
|
combined work, and to convey the resulting work. The terms of this
|
||||||
|
License will continue to apply to the part which is the covered work,
|
||||||
|
but the special requirements of the GNU Affero General Public License,
|
||||||
|
section 13, concerning interaction through a network will apply to the
|
||||||
|
combination as such.
|
||||||
|
|
||||||
|
14. Revised Versions of this License.
|
||||||
|
|
||||||
|
The Free Software Foundation may publish revised and/or new versions of
|
||||||
|
the GNU General Public License from time to time. Such new versions will
|
||||||
|
be similar in spirit to the present version, but may differ in detail to
|
||||||
|
address new problems or concerns.
|
||||||
|
|
||||||
|
Each version is given a distinguishing version number. If the
|
||||||
|
Program specifies that a certain numbered version of the GNU General
|
||||||
|
Public License "or any later version" applies to it, you have the
|
||||||
|
option of following the terms and conditions either of that numbered
|
||||||
|
version or of any later version published by the Free Software
|
||||||
|
Foundation. If the Program does not specify a version number of the
|
||||||
|
GNU General Public License, you may choose any version ever published
|
||||||
|
by the Free Software Foundation.
|
||||||
|
|
||||||
|
If the Program specifies that a proxy can decide which future
|
||||||
|
versions of the GNU General Public License can be used, that proxy's
|
||||||
|
public statement of acceptance of a version permanently authorizes you
|
||||||
|
to choose that version for the Program.
|
||||||
|
|
||||||
|
Later license versions may give you additional or different
|
||||||
|
permissions. However, no additional obligations are imposed on any
|
||||||
|
author or copyright holder as a result of your choosing to follow a
|
||||||
|
later version.
|
||||||
|
|
||||||
|
15. Disclaimer of Warranty.
|
||||||
|
|
||||||
|
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||||
|
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||||
|
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||||
|
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||||
|
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||||
|
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||||
|
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||||
|
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||||
|
|
||||||
|
16. Limitation of Liability.
|
||||||
|
|
||||||
|
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||||
|
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||||
|
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||||
|
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||||
|
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||||
|
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||||
|
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||||
|
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||||
|
SUCH DAMAGES.
|
||||||
|
|
||||||
|
17. Interpretation of Sections 15 and 16.
|
||||||
|
|
||||||
|
If the disclaimer of warranty and limitation of liability provided
|
||||||
|
above cannot be given local legal effect according to their terms,
|
||||||
|
reviewing courts shall apply local law that most closely approximates
|
||||||
|
an absolute waiver of all civil liability in connection with the
|
||||||
|
Program, unless a warranty or assumption of liability accompanies a
|
||||||
|
copy of the Program in return for a fee.
|
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
How to Apply These Terms to Your New Programs
|
||||||
|
|
||||||
|
If you develop a new program, and you want it to be of the greatest
|
||||||
|
possible use to the public, the best way to achieve this is to make it
|
||||||
|
free software which everyone can redistribute and change under these terms.
|
||||||
|
|
||||||
|
To do so, attach the following notices to the program. It is safest
|
||||||
|
to attach them to the start of each source file to most effectively
|
||||||
|
state the exclusion of warranty; and each file should have at least
|
||||||
|
the "copyright" line and a pointer to where the full notice is found.
|
||||||
|
|
||||||
|
<one line to give the program's name and a brief idea of what it does.>
|
||||||
|
Copyright (C) <year> <name of author>
|
||||||
|
|
||||||
|
This program is free software: you can redistribute it and/or modify
|
||||||
|
it under the terms of the GNU General Public License as published by
|
||||||
|
the Free Software Foundation, either version 3 of the License, or
|
||||||
|
(at your option) any later version.
|
||||||
|
|
||||||
|
This program is distributed in the hope that it will be useful,
|
||||||
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
GNU General Public License for more details.
|
||||||
|
|
||||||
|
You should have received a copy of the GNU General Public License
|
||||||
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
Also add information on how to contact you by electronic and paper mail.
|
||||||
|
|
||||||
|
If the program does terminal interaction, make it output a short
|
||||||
|
notice like this when it starts in an interactive mode:
|
||||||
|
|
||||||
|
<program> Copyright (C) <year> <name of author>
|
||||||
|
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||||
|
This is free software, and you are welcome to redistribute it
|
||||||
|
under certain conditions; type `show c' for details.
|
||||||
|
|
||||||
|
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||||
|
parts of the General Public License. Of course, your program's commands
|
||||||
|
might be different; for a GUI interface, you would use an "about box".
|
||||||
|
|
||||||
|
You should also get your employer (if you work as a programmer) or school,
|
||||||
|
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||||
|
For more information on this, and how to apply and follow the GNU GPL, see
|
||||||
|
<https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
The GNU General Public License does not permit incorporating your program
|
||||||
|
into proprietary programs. If your program is a subroutine library, you
|
||||||
|
may consider it more useful to permit linking proprietary applications with
|
||||||
|
the library. If this is what you want to do, use the GNU Lesser General
|
||||||
|
Public License instead of this License. But first, please read
|
||||||
|
<https://www.gnu.org/licenses/why-not-lgpl.html>.
|
||||||
312
README.md
Normal file
|
|
@ -0,0 +1,312 @@
|
||||||
|
# QuickSearch
|
||||||
|
|
||||||
|
A fast local file indexer and search tool. QuickSearch walks your chosen
|
||||||
|
folders into a compact SQLite index (FTS5 full-text + zstd-compressed text
|
||||||
|
sidecar), keeps it fresh automatically with filesystem watchers and
|
||||||
|
periodic reindexing, and serves ranked search-as-you-type results in a
|
||||||
|
compact egui desktop app, or straight to your terminal.
|
||||||
|
|
||||||
|
## Build & run
|
||||||
|
|
||||||
|
Requirements: a Rust toolchain (edition 2021). The old WebKit/WebView
|
||||||
|
dependencies (`setup.sh`) are gone; the GUI renders with OpenGL via egui.
|
||||||
|
|
||||||
|
SQLite and zstd are compiled from bundled C sources, so a C toolchain is
|
||||||
|
required on every platform:
|
||||||
|
|
||||||
|
- Linux: working OpenGL 3.3 drivers; `xdg-desktop-portal` (present on all
|
||||||
|
mainstream desktops) provides the native folder picker. On minimal
|
||||||
|
images you may need `build-essential pkg-config libxkbcommon-dev`.
|
||||||
|
- Windows: Visual Studio 2022 Build Tools with the "Desktop development
|
||||||
|
with C++" workload (MSVC v143 plus a Windows SDK). For the GNU target
|
||||||
|
instead, `rustup target add x86_64-pc-windows-gnu` and a mingw-w64
|
||||||
|
toolchain. Note that Windows ships only a software OpenGL 1.1 driver, so
|
||||||
|
a bare VM or an RDP session without a vendor GPU driver cannot create a
|
||||||
|
context and the window will fail to open.
|
||||||
|
- macOS: Xcode command line tools.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo build --release -p quicksearch-gui # binaries: target/release/quicksearch{,-cli}
|
||||||
|
cargo run -p quicksearch-gui # or just run it
|
||||||
|
cargo test -p quicksearch-core # backend test suite
|
||||||
|
```
|
||||||
|
|
||||||
|
`run.sh` / `run.bat` wrap the same commands.
|
||||||
|
|
||||||
|
Two binaries are produced. `quicksearch` is the desktop app; on Windows it
|
||||||
|
is built as a window-subsystem app so no console appears behind it.
|
||||||
|
`quicksearch-cli` is terminal search — a console app, so pipes, redirection
|
||||||
|
and exit codes behave normally. On Unix `quicksearch` also does both, and
|
||||||
|
`quicksearch-cli` is simply the same tool under a clearer name.
|
||||||
|
|
||||||
|
## Install (Debian / Ubuntu)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./packaging/build-deb.sh
|
||||||
|
sudo apt install ./dist/quicksearch_0.1.0-1_amd64.deb
|
||||||
|
```
|
||||||
|
|
||||||
|
The script builds the release binary, strips it, and assembles a `.deb` with
|
||||||
|
`dpkg-deb`. It needs no `cargo-deb`, no `debhelper` and no SVG rasteriser —
|
||||||
|
only `dpkg-deb` and `desktop-file-utils`, both standard on Debian and Ubuntu.
|
||||||
|
Useful flags: `--no-build` to package a binary you already built, `--no-strip`
|
||||||
|
to keep debug symbols, `-o DIR` to write elsewhere. `DEB_REVISION` and
|
||||||
|
`DEB_MAINTAINER` override the packaging revision and maintainer.
|
||||||
|
|
||||||
|
The package installs:
|
||||||
|
|
||||||
|
| Path | Contents |
|
||||||
|
| --- | --- |
|
||||||
|
| `/usr/bin/quicksearch` | the desktop app, which also does terminal search |
|
||||||
|
| `/usr/bin/quicksearch-cli` | terminal search only |
|
||||||
|
| `/usr/share/applications/quicksearch.desktop` | menu entry, so QuickSearch appears in the app launcher |
|
||||||
|
| `/usr/share/icons/hicolor/{16,22,24,32,48,64,128,256}x*/apps/` | icons at each size |
|
||||||
|
| `/usr/share/icons/hicolor/scalable/apps/quicksearch.svg` | the source icon |
|
||||||
|
| `/usr/share/man/man1/quicksearch{,-cli}.1.gz` | `man quicksearch`; the `-cli` page is a `.so` stub pointing at it |
|
||||||
|
| `/usr/share/doc/quicksearch/` | copyright, changelog, README, `config_example.toml` |
|
||||||
|
|
||||||
|
Installing registers the menu entry and the icon: dpkg triggers owned by
|
||||||
|
`desktop-file-utils` and `hicolor-icon-theme` refresh both caches, so no
|
||||||
|
maintainer scripts are involved and `apt remove` reverses it cleanly.
|
||||||
|
|
||||||
|
No `config.toml` is installed. One placed next to the executable would put
|
||||||
|
every user into portable mode (see [Configuration](#configuration)); instead
|
||||||
|
the app writes `~/.config/quicksearch/config.toml` on first run.
|
||||||
|
|
||||||
|
### Icons
|
||||||
|
|
||||||
|
`crates/quicksearch-gui/assets/icons/` holds `quicksearch_icon.svg` and the
|
||||||
|
PNGs rasterised from it. The PNGs are committed rather than generated, so an
|
||||||
|
ordinary `cargo build` needs no image tooling — the 256px one is compiled into
|
||||||
|
the binary with `include_bytes!` and becomes the window icon. Editing the SVG
|
||||||
|
means re-rendering the PNGs; `packaging/build-deb.sh` documents how in a
|
||||||
|
comment at the top.
|
||||||
|
|
||||||
|
X11 takes the window icon from the embedded PNG. Wayland ignores it and
|
||||||
|
matches the app id (`quicksearch`) against the installed
|
||||||
|
`quicksearch.desktop`, so under Wayland the titlebar icon appears only once
|
||||||
|
the package is installed.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### GUI
|
||||||
|
|
||||||
|
`quicksearch` with no query arguments opens the app:
|
||||||
|
|
||||||
|
- **Search**: results appear as you type; every keystroke cancels the
|
||||||
|
previous search. One checkbox enables the two fuzzy passes. Sort by
|
||||||
|
rank, name, path, size, or modified; right-click a result to open it,
|
||||||
|
reveal it in the file manager, or build an ignore filter from it
|
||||||
|
(session-only by default, optionally persisted to the config). Matches
|
||||||
|
in file contents show highlighted snippets.
|
||||||
|
- **Manage Index**: full indexing status, Start/Stop/Automatic controls,
|
||||||
|
indexed folder list, full-text extension filters, ignore patterns, and
|
||||||
|
the indexing options.
|
||||||
|
- **Duplicates**: files sharing a content hash, grouped.
|
||||||
|
- **Logs**: the lines the app would have printed to a terminal — warnings
|
||||||
|
from indexing, folder watching and opening files, newest last, with a
|
||||||
|
filter box and Copy button. Launched from a desktop launcher (or on
|
||||||
|
Windows, where the app has no console at all) this is the only place
|
||||||
|
they are visible.
|
||||||
|
|
||||||
|
The bottom status bar always shows what the indexer is doing (phase,
|
||||||
|
percent, files/sec) or the total indexed file count when idle.
|
||||||
|
|
||||||
|
### Terminal
|
||||||
|
|
||||||
|
```sh
|
||||||
|
quicksearch report type:Document modified:">=2024-01-01"
|
||||||
|
quicksearch --long --limit 20 "quarterly budget"
|
||||||
|
quicksearch --fuzzy repot # tolerates typos
|
||||||
|
```
|
||||||
|
|
||||||
|
Prints rank-ordered paths (pipe-friendly); `--long` adds rank, size,
|
||||||
|
mtime, and highlighted snippets. `quicksearch --help` shows all flags.
|
||||||
|
|
||||||
|
On Windows use `quicksearch-cli` for all of the above — `quicksearch.exe`
|
||||||
|
opens the app, and any query given to it seeds the search box instead of
|
||||||
|
printing. Colour in `--long` output needs a console with virtual-terminal
|
||||||
|
processing; Windows Terminal has it, and older consoles get plain text.
|
||||||
|
|
||||||
|
### Query syntax
|
||||||
|
|
||||||
|
This section is the complete reference (the in-app "?" popup shows a
|
||||||
|
condensed version of the same rules). Everything that isn't a filter is
|
||||||
|
matched as one phrase, in order. Filters combine freely with the search
|
||||||
|
text:
|
||||||
|
|
||||||
|
| Syntax | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| `budget report` | names, contents, and paths containing the phrase `budget report` |
|
||||||
|
| `"exact phrase"` | quotes keep spaces, stars, and filter-like words literal; `""` escapes a quote |
|
||||||
|
| `bud*port` | `*` matches any run of characters (it stays on one line of content); `%` and `_` are always literal |
|
||||||
|
| `regex:"(foo|bar)\d+"` | regular expression matched against names, contents, and paths; case-insensitive by default (`(?-i:…)` overrides); quote patterns containing spaces or `( ) : = < > "` |
|
||||||
|
| `type:Audio` | one of Audio, Image, Video, Document, Text, Archive, Spreadsheet, Presentation, Folder |
|
||||||
|
| `modified:>=2024-01-01` | also `<`, `<=`, `>`, `=` (dates are `yyyy-mm-dd`; `mtime:` is an alias) |
|
||||||
|
| `path:/home/me/docs` | restrict to a folder and its subfolders (`folder:` and `includefolder:` are aliases); `*` is literal here |
|
||||||
|
| `path:C:\Users\me\docs` | the same on Windows — drive letters and backslashes need no quoting |
|
||||||
|
| `mime:application/pdf` | exact MIME type |
|
||||||
|
| `name:re*.txt` | filename contains (as a filter, unranked; `filename:` is an alias); unquoted `*` globs |
|
||||||
|
|
||||||
|
Unrecognized `key:value` text (like `12:30`) stays part of the search
|
||||||
|
phrase, and a half-typed quote never errors while you type. `AND`, `OR`
|
||||||
|
and parentheses are treated as plain words. A term of only stars matches
|
||||||
|
nothing, and a regex that could match the empty string is rejected rather
|
||||||
|
than matching every file. `regex:` bypasses the trigram index entirely
|
||||||
|
and combines with filters; alongside search text it acts as an extra
|
||||||
|
requirement on those results.
|
||||||
|
|
||||||
|
The search box highlights this syntax as you type: recognized filter
|
||||||
|
keywords in red, their arguments in blue, syntax characters (operators,
|
||||||
|
quotes, live wildcards) in green, on a tinted chip per complete filter.
|
||||||
|
An argument the engine would reject — unknown `type:` name, bad date,
|
||||||
|
invalid regex — switches to the error color immediately.
|
||||||
|
|
||||||
|
Results are ranked: exact filename matches (case-sensitive first), then
|
||||||
|
filename substrings, then full-text matches ordered by occurrence count,
|
||||||
|
then fuzzy filename/full-text matches when enabled, and last the files
|
||||||
|
matched somewhere else in their path. Later, weaker matches only ever
|
||||||
|
append to the bottom of the list. Wildcard terms rank through the same
|
||||||
|
tiers (an "exact" match means the whole name matches the pattern) but
|
||||||
|
skip the fuzzy passes; regex-only queries reuse the substring, full-text,
|
||||||
|
and path tiers. Path matching needs at least three characters, and terms
|
||||||
|
may span separators (`docs/report`). Full-text matching also needs at
|
||||||
|
least three characters of literal text (the trigram floor). The fuzzy
|
||||||
|
passes tolerate typos with a budget of one edit per three characters,
|
||||||
|
capped by `[search] fuzzy_max_edits` (default 2; 0 turns fuzzy off).
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
`config.toml` lives at `~/.config/quicksearch/config.toml` (Windows:
|
||||||
|
`%APPDATA%\quicksearch\config.toml`) and is created on first run; the
|
||||||
|
default index goes to `~/.local/share/quicksearch/index.sqlite`
|
||||||
|
(Windows: `%LOCALAPPDATA%\quicksearch\index.sqlite`). See
|
||||||
|
`config_example.toml` for every option.
|
||||||
|
|
||||||
|
Defaults follow the platform. The first indexing root is your home
|
||||||
|
directory or `%USERPROFILE%`; `include_hidden = false` skips dot-files
|
||||||
|
everywhere and additionally anything marked Hidden or System on Windows,
|
||||||
|
which is what keeps `AppData`, `$RECYCLE.BIN` and `System Volume
|
||||||
|
Information` out of the index; and ignore patterns are matched
|
||||||
|
case-insensitively on Windows and macOS, matching the filesystem.
|
||||||
|
|
||||||
|
**Portable mode**: a `config.toml` sitting next to the `quicksearch`
|
||||||
|
binary overrides the user config entirely, and relative paths inside any
|
||||||
|
config resolve against the config file's own directory, so a folder
|
||||||
|
containing the binary, its config, and its index can be moved wholesale.
|
||||||
|
|
||||||
|
The GUI edits the config live; external edits apply on next start.
|
||||||
|
|
||||||
|
## Engineering overview
|
||||||
|
|
||||||
|
Two crates:
|
||||||
|
|
||||||
|
```
|
||||||
|
crates/quicksearch-core library: indexing, storage, search
|
||||||
|
crates/quicksearch-gui binary "quicksearch": egui app + terminal mode
|
||||||
|
```
|
||||||
|
|
||||||
|
### Backend (`quicksearch-core`)
|
||||||
|
|
||||||
|
Synchronous Rust: `std::thread` + `mpsc` channels, no async runtime.
|
||||||
|
|
||||||
|
- **Storage** (`db/`): SQLite via rusqlite (bundled), WAL mode so the
|
||||||
|
single writer never blocks streaming read-only searches. `files` holds
|
||||||
|
metadata (name, path, size, mtime, hash, MIME/type bitmask, per-row
|
||||||
|
index state); `searchabletext` is a *contentless* FTS5 table (postings
|
||||||
|
only, configurable tokenizer, trigram by default); canonical extracted
|
||||||
|
text lives zstd-compressed in `documents_text`, which powers snippets,
|
||||||
|
occurrence ranking, and fuzzy full-text search. Schema changes wipe and
|
||||||
|
rebuild by policy; the indexer (`open_or_recreate`) is the only code
|
||||||
|
allowed to do that; every consumer uses `open_existing`, which treats
|
||||||
|
drift as an error, never data loss.
|
||||||
|
- **Indexing** (`indexing.rs`, `file_handling.rs`): full runs walk each
|
||||||
|
root (`filtered_walk` prunes hidden/ignored subtrees before descending),
|
||||||
|
classify files by mtime into insert/update/skip, batch-write metadata,
|
||||||
|
sweep stale rows, then extract content (plaintext, Office, PDF, audio
|
||||||
|
tags, EXIF; see `extract/`) for FTS. Files no larger than
|
||||||
|
`processing.hash_length` skip that second pass entirely: the head the walk
|
||||||
|
reads to hash them is already their whole content, so a plaintext body is
|
||||||
|
extracted in the same `read` and stored complete. Progress streams through
|
||||||
|
a polled `IndexingStatus`.
|
||||||
|
- **Coordinator** (`coordinator.rs`): the object binaries construct.
|
||||||
|
Owns the `IndexingService`, the debouncing filesystem watcher
|
||||||
|
(`watcher.rs`), and the mode state machine (Auto / Manual). Watcher
|
||||||
|
events become single-file transactions (`incremental.rs`) that keep
|
||||||
|
`files`, FTS, and the text sidecar consistent per commit; a full
|
||||||
|
reindex runs on a configurable interval. Incremental writes defer while
|
||||||
|
a full run is active, so there is exactly one writer at a time.
|
||||||
|
Registration follows what the platform's notification API can do:
|
||||||
|
inotify covers one directory per watch, so the roots are walked and each
|
||||||
|
surviving directory registered individually (skipping `.git`,
|
||||||
|
`node_modules` and hidden subtrees, which is what keeps the watch count
|
||||||
|
affordable), while `ReadDirectoryChangesW` covers a whole tree from one
|
||||||
|
handle and takes a single watch per root, filtering the events instead.
|
||||||
|
Either way a tree too large to watch degrades to periodic reindexing
|
||||||
|
rather than going silently stale.
|
||||||
|
- **Search** (`search/`): `SearchService` runs one worker thread; each
|
||||||
|
query is a *generation*. New queries interrupt the in-flight SQLite
|
||||||
|
statement (`InterruptHandle`) and stale generations stop cooperatively,
|
||||||
|
so typing never waits. The cascade streams rank-ordered batches: one
|
||||||
|
`files` scan classifies exact/case/substring filename matches (ranks
|
||||||
|
1–4) and, since a path contains its own name, sets aside full-path
|
||||||
|
matches from the same rows (ranks 9–10); one FTS phrase probe verified
|
||||||
|
against the decompressed text yields full-text ranks 5–6 ordered by
|
||||||
|
occurrence count; and the opt-in fuzzy passes run a bitap (Wu–Manber)
|
||||||
|
matcher over filenames (rank 7), document text (rank 8) and paths
|
||||||
|
(rank 11), with a configurable edit budget. The deferred path tiers
|
||||||
|
flush last, so weaker matches only ever append. All SQL is
|
||||||
|
parameterized; structured filters from the query language (`query/`)
|
||||||
|
are ANDed onto every pass.
|
||||||
|
- **Baloo compatibility** (`cli.rs`, `mime.rs`): read-only endpoints
|
||||||
|
(`status_for_path`, `list_failed`, `index_size_breakdown`, …) and a
|
||||||
|
Baloo-shaped type model, groundwork for a future `balooctl`-compatible
|
||||||
|
layer.
|
||||||
|
- **Logging** (`log.rs`): background reporting goes through `log_info!` /
|
||||||
|
`log_warn!` rather than `println!`/`eprintln!`. Each writes its line to
|
||||||
|
stderr *and* appends it to a bounded in-memory ring (newest 5000 lines,
|
||||||
|
with a count of what was dropped) that the GUI's Logs tab reads, so a
|
||||||
|
windowed run with no terminal still surfaces them. Command output —
|
||||||
|
search hits, usage, the error a command exits with — stays on stdio.
|
||||||
|
- **Platform differences** (`platform.rs`): the single home for `#[cfg]`.
|
||||||
|
Home directory lookup, what counts as a hidden entry (dot-prefix, plus
|
||||||
|
the Hidden/System attributes on Windows), network-filesystem detection
|
||||||
|
(`/proc/mounts` against `GetDriveTypeW`), path collation, and the
|
||||||
|
watch-registration strategy all live here, so the rest of the crate can
|
||||||
|
ask a question rather than test a target. Anything decidable from a
|
||||||
|
string alone is split out so its tests run on every platform.
|
||||||
|
|
||||||
|
### Frontend (`quicksearch-gui`)
|
||||||
|
|
||||||
|
Immediate-mode egui/eframe app, one UI thread:
|
||||||
|
|
||||||
|
```
|
||||||
|
UI thread ──SearchRequest──▶ search worker ──SearchUpdate (mpsc)──▶ drained per frame
|
||||||
|
UI thread ──commands──────▶ IndexCoordinator ──state──▶ polled per frame
|
||||||
|
core threads ─────────────▶ ctx.request_repaint() (wake the UI)
|
||||||
|
```
|
||||||
|
|
||||||
|
Modules map one-to-one onto what you see: `app.rs` (shell, status bar,
|
||||||
|
config routing), `search_tab.rs` (query strip, virtualized results table,
|
||||||
|
snippet highlighting via `LayoutJob` byte ranges, ignore dialog),
|
||||||
|
`manage_tab.rs` (status detail + `tracker.rs` rate estimation, roots and
|
||||||
|
filter editors), `duplicates_tab.rs`, `logs_tab.rs` (a virtualized view of
|
||||||
|
the core log ring), `options.rs` (draft-based settings
|
||||||
|
editor shared between the window and the Manage tab), `platform.rs`
|
||||||
|
(open / reveal-in-file-manager, and the Windows stdio setup a
|
||||||
|
window-subsystem process needs before anything prints), `cli.rs` (terminal
|
||||||
|
mode, shared with the `quicksearch-cli` binary). There is no
|
||||||
|
pagination: the table is virtualized, so a single scroll list capped at
|
||||||
|
`display_limit` renders in microseconds regardless of row count.
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
- `cargo test -p quicksearch-core`: unit + integration suites (cascade
|
||||||
|
ranking, cancellation, incremental indexing, coordinator modes, config
|
||||||
|
resolution, fuzzy matcher vs. brute-force oracle).
|
||||||
|
- `cargo test -p quicksearch-gui`: formatter/tracker/CLI-parsing units.
|
||||||
|
- `QSB_SNIPPET_PERF=1 cargo test --release -p quicksearch-core --test
|
||||||
|
snippet_perf -- --nocapture`: snippet pipeline benchmark.
|
||||||
|
- New extractors: implement `extract::Extractor` and register it in
|
||||||
|
`Registry::default_set()`. New cascade behavior: `search/cascade.rs`
|
||||||
|
documents the rank invariants that keep streamed results append-only.
|
||||||
|
|
@ -1,37 +1,112 @@
|
||||||
|
# QuickSearch configuration reference.
|
||||||
|
#
|
||||||
|
# The live config is auto-created at ~/.config/quicksearch/config.toml
|
||||||
|
# (Windows: %APPDATA%\quicksearch\config.toml). A config.toml placed next
|
||||||
|
# to the quicksearch binary overrides it entirely (portable mode).
|
||||||
|
# Relative paths resolve against the directory containing the config
|
||||||
|
# file, so a portable folder can be moved wholesale.
|
||||||
|
#
|
||||||
|
# Every key is optional; missing keys take the defaults shown here.
|
||||||
|
|
||||||
[paths]
|
[paths]
|
||||||
# One or more directory roots to index. Walked in order; duplicate and
|
# One or more directory roots to index. Walked in order; duplicate and
|
||||||
# nested roots are de-duplicated automatically.
|
# nested roots are de-duplicated automatically. `~` expands to home.
|
||||||
indexing_paths = ["/"]
|
indexing_paths = ["~"]
|
||||||
database_path = "QuickSearch.db"
|
# SQLite index location. Default: ~/.local/share/quicksearch/index.sqlite
|
||||||
|
# On Windows the default is %LOCALAPPDATA%\quicksearch\index.sqlite. Write
|
||||||
|
# Windows paths as TOML *literal* strings (single quotes) so the
|
||||||
|
# backslashes need no escaping, and keep the index out of a roaming
|
||||||
|
# profile — it is far too large to synchronise:
|
||||||
|
# database_path = 'C:\Users\you\AppData\Local\quicksearch\index.sqlite'
|
||||||
|
database_path = "~/.local/share/quicksearch/index.sqlite"
|
||||||
|
|
||||||
|
[indexing]
|
||||||
|
# Automatic mode: filesystem watchers apply changes as they happen and a
|
||||||
|
# full reindex runs every reindex_interval_minutes. The watcher catches
|
||||||
|
# changes as they happen, so this only needs to be often enough to cover
|
||||||
|
# whatever the watcher missed.
|
||||||
|
auto_index = true
|
||||||
|
reindex_interval_minutes = 1440
|
||||||
|
# Follow symbolic links during directory walks.
|
||||||
|
follow_symlinks = false
|
||||||
|
# Index hidden files and directories. That means dot-files everywhere, and
|
||||||
|
# additionally anything carrying the Hidden or System attribute on Windows
|
||||||
|
# (AppData, $RECYCLE.BIN, System Volume Information, pagefile.sys ...).
|
||||||
|
include_hidden = false
|
||||||
|
# Empty = extract text from every supported format. Non-empty = content
|
||||||
|
# indexing only for these extensions; other files are still listed for
|
||||||
|
# filename search. Entries are case-insensitive, leading dot optional.
|
||||||
|
content_extensions = []
|
||||||
|
# Excluded from the index entirely. A pattern without a separator matches
|
||||||
|
# any single path component (so ".git" prunes whole subtrees); patterns
|
||||||
|
# containing one match full paths. Glob syntax (*, ?, [..]). Matching is
|
||||||
|
# case-insensitive on Windows and macOS, case-sensitive elsewhere,
|
||||||
|
# following the filesystem.
|
||||||
|
#
|
||||||
|
# The Windows defaults add: "$RECYCLE.BIN", "System Volume Information",
|
||||||
|
# "pagefile.sys", "hiberfil.sys", "swapfile.sys", "Thumbs.db",
|
||||||
|
# "desktop.ini".
|
||||||
|
ignore_patterns = [".git", "node_modules", "*.tmp", ".venv", "venv"]
|
||||||
|
|
||||||
|
# Worth adding by hand if you index a whole Windows drive rather than just
|
||||||
|
# your profile. Neither is excluded by default, because the default root
|
||||||
|
# is your profile and a bare "Windows" pattern would also match a folder
|
||||||
|
# of your own with that name:
|
||||||
|
# 'C:\Windows' — system files, nothing you would search for
|
||||||
|
# 'C:\Windows\WinSxS' — a hardlink farm that floods the Duplicates tab
|
||||||
|
|
||||||
|
# Walker threads per root, keyed by the exact root string from
|
||||||
|
# indexing_paths. Absent or 0 = auto (4 on local storage, 16 on network
|
||||||
|
# mounts, detected per root). Applies at the start of the next run.
|
||||||
|
# root_workers = { "/media/share" = 24 }
|
||||||
|
|
||||||
[processing]
|
[processing]
|
||||||
# Amount of data in bytes read from start/end of files used to calculate hash
|
# Bytes read from the start of each file for its content hash, which is
|
||||||
|
# `sha256(size || first hash_length bytes)` and backs duplicate detection.
|
||||||
|
# Only the head is read: seeking to the end for a second block costs an
|
||||||
|
# extra round trip per file on network shares.
|
||||||
|
#
|
||||||
|
# Known limitation: files of identical size whose heads match will be
|
||||||
|
# reported as duplicates. In practice that means pre-allocated VM disk
|
||||||
|
# images: a fixed-size VHD stores its unique footer at the end of the
|
||||||
|
# file, and a freshly pre-allocated raw/qcow2/VMDK image is all zeros at
|
||||||
|
# the head until it is partitioned.
|
||||||
hash_length = 8192
|
hash_length = 8192
|
||||||
# Maximum text content to index per file (bytes)
|
# Maximum extracted text stored per file (bytes).
|
||||||
maximum_text_size = 262144
|
maximum_text_size = 262144
|
||||||
# Maximum file size to process for text extraction (bytes)
|
# Files larger than this skip text extraction entirely (bytes).
|
||||||
maximum_text_file_size = 2097152
|
maximum_text_file_size = 2097152
|
||||||
# Number of files to process in each batch (directory walk / inserts / text extraction batches)
|
# Files per batch during walks / inserts / extraction.
|
||||||
batch_size = 200
|
batch_size = 200
|
||||||
# Files per transaction for incremental UPDATE files + DELETE from searchabletext (FTS); larger = fewer commits, more RAM per chunk
|
# Files per transaction for incremental FTS updates.
|
||||||
fts_update_batch_size = 1000
|
fts_update_batch_size = 1000
|
||||||
# If true, run a fast shell-backed tree count before Phase 1 (enables % progress; Linux uses GNU find -printf '\n' | wc -l when available).
|
# FTS5 tokenizer: 'trigram' (substring matching, the default; gets
|
||||||
# If false, Phase 1 shows file counts without a percentage.
|
# remove_diacritics 1 appended), 'unicode61', 'porter', or a full FTS5
|
||||||
precount_files_for_progress = false
|
# option string. See https://www.sqlite.org/fts5.html#tokenizers
|
||||||
# If true, follow symbolic links during directory walks (indexing only; shell precount unchanged).
|
|
||||||
follow_symlinks = false
|
|
||||||
# If true, hidden files and directories will be indexed.
|
|
||||||
include_hidden = false
|
|
||||||
# FTS5 tokenization method (e.g., 'trigram', 'porter', 'unicode61')
|
|
||||||
# Look here for more information https://www.sqlite.org/fts5.html#tokenizers
|
|
||||||
tokenize = "trigram"
|
tokenize = "trigram"
|
||||||
# If true (default), extracted text is stored zstd-compressed in a sidecar
|
# Store extracted text (zstd-compressed) alongside the FTS index. Off:
|
||||||
# table so the GUI's search results can render snippet previews with the
|
# the index shrinks to roughly stock-Baloo size, but search loses snippet
|
||||||
# query terms highlighted. If false, the inverted FTS5 index is still
|
# previews, occurrence ranking, case verification, and fuzzy full-text.
|
||||||
# populated (so queries match the same files) but nothing is stored
|
|
||||||
# alongside; result rows carry no snippet. Turning this off drops the
|
|
||||||
# on-disk footprint to roughly what stock Baloo uses, useful for
|
|
||||||
# apples-to-apples comparisons and for users who never read snippet text.
|
|
||||||
# Changing this only affects files indexed *after* the change; existing
|
|
||||||
# sidecar rows are kept until re-indexed.
|
|
||||||
store_text_for_snippets = true
|
store_text_for_snippets = true
|
||||||
|
|
||||||
|
[ui]
|
||||||
|
# Zoom factor for the whole GUI: fonts, spacing, and widgets scale
|
||||||
|
# together (0.5 – 2.5). Ctrl +/- and Ctrl 0 adjust it temporarily at
|
||||||
|
# runtime; this value is the persistent baseline.
|
||||||
|
scale = 1.1
|
||||||
|
|
||||||
|
[search]
|
||||||
|
# Start with the fuzzy passes enabled.
|
||||||
|
fuzzy_default = false
|
||||||
|
# Ceiling on the fuzzy stages' typo budget. The allowance grows with the
|
||||||
|
# search term, one edit per three characters, up to this value, so 2
|
||||||
|
# means "1 edit for 3-5 character terms, 2 for anything longer". 0 turns
|
||||||
|
# the fuzzy stages off. Above 3 is allowed but not recommended: matches
|
||||||
|
# become dominated by coincidence and every fuzzy pass slows down.
|
||||||
|
fuzzy_max_edits = 2
|
||||||
|
# Hard cap on results per search (the GUI's scroll list length).
|
||||||
|
display_limit = 1000
|
||||||
|
# Results per streamed batch (latency/overhead knob, not a page size).
|
||||||
|
results_per_page = 100
|
||||||
|
# How long the GUI waits after the last keystroke before searching (ms).
|
||||||
|
debounce_ms = 150
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,10 @@
|
||||||
name = "quicksearch-core"
|
name = "quicksearch-core"
|
||||||
version.workspace = true
|
version.workspace = true
|
||||||
edition.workspace = true
|
edition.workspace = true
|
||||||
|
license.workspace = true
|
||||||
|
authors.workspace = true
|
||||||
|
repository.workspace = true
|
||||||
|
description = "Indexing, storage and search engine behind QuickSearch."
|
||||||
|
|
||||||
[lib]
|
[lib]
|
||||||
name = "quicksearch_core"
|
name = "quicksearch_core"
|
||||||
|
|
@ -17,10 +21,25 @@ serde = { version = "1.0", features = ["derive"] }
|
||||||
toml = "0.8"
|
toml = "0.8"
|
||||||
mime_guess = "2.0"
|
mime_guess = "2.0"
|
||||||
infer = "0.15"
|
infer = "0.15"
|
||||||
pdf-extract = "0.7"
|
pdf-extract = "0.12"
|
||||||
lopdf = "0.32"
|
lopdf = "0.32"
|
||||||
lofty = "0.19"
|
lofty = "0.19"
|
||||||
kamadak-exif = "0.5"
|
kamadak-exif = "0.5"
|
||||||
notify = "6.1"
|
notify = "6.1"
|
||||||
ctrlc = "3.4"
|
ctrlc = "3.4"
|
||||||
zstd = "0.13"
|
zstd = "0.13"
|
||||||
|
globset = "0.4"
|
||||||
|
regex = "1"
|
||||||
|
|
||||||
|
# `GetDriveTypeW` (a mapped drive letter is the only way to spot an SMB share
|
||||||
|
# that isn't written as UNC) plus the FILE_ATTRIBUTE_* constants for hidden
|
||||||
|
# detection. Pinned to 0.52 deliberately: walkdir → winapi-util already
|
||||||
|
# resolves exactly that version, so this adds no new crate compilations.
|
||||||
|
[target.'cfg(windows)'.dependencies]
|
||||||
|
# GetDriveTypeW and the FILE_ATTRIBUTE_* constants live in
|
||||||
|
# Win32_Storage_FileSystem; DRIVE_REMOTE, oddly, is filed under
|
||||||
|
# Win32_System_WindowsProgramming.
|
||||||
|
windows-sys = { version = "0.52", features = [
|
||||||
|
"Win32_Storage_FileSystem",
|
||||||
|
"Win32_System_WindowsProgramming",
|
||||||
|
] }
|
||||||
|
|
|
||||||
216
crates/quicksearch-core/examples/indexprobe.rs
Normal file
|
|
@ -0,0 +1,216 @@
|
||||||
|
//! End-to-end timing and syscall accounting for a full indexing run.
|
||||||
|
//!
|
||||||
|
//! [`walkprobe`](walkprobe.rs) covers phase 1 alone, without a database. This
|
||||||
|
//! covers the whole pipeline — parallel walk, `files` writes, and content
|
||||||
|
//! extraction — because the interesting redundancy lives *between* the two
|
||||||
|
//! phases: the walk reads a file's head to hash it and sniff its MIME, and
|
||||||
|
//! extraction then reopens the same file and reads it again.
|
||||||
|
//!
|
||||||
|
//! ```text
|
||||||
|
//! cargo build -p quicksearch-core --example indexprobe --release
|
||||||
|
//! ./target/release/examples/indexprobe gen /tmp/qs-bench
|
||||||
|
//! ./target/release/examples/indexprobe cold /tmp/qs-bench /tmp/qs-bench.db
|
||||||
|
//! ./target/release/examples/indexprobe warm /tmp/qs-bench /tmp/qs-bench.db
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! `cold` deletes the database first, so every file is new: the walk hashes
|
||||||
|
//! it and extraction reads it. `warm` re-runs over the existing database with
|
||||||
|
//! the tree untouched, which is the case that has to stay at one `stat` per
|
||||||
|
//! file — see [`crate::file_handling::classify_for_indexing`].
|
||||||
|
//!
|
||||||
|
//! For syscalls per file, trace a run and bucket by the tree's paths:
|
||||||
|
//!
|
||||||
|
//! ```text
|
||||||
|
//! strace -f -y -o /tmp/t.log \
|
||||||
|
//! -e trace=openat,statx,newfstatat,fstat,read,pread64,readlink,close,getdents64,lseek \
|
||||||
|
//! ./target/release/examples/indexprobe cold /tmp/qs-bench /tmp/qs-bench.db
|
||||||
|
//! grep -oP '^\d+ \K[a-z0-9_]+' <(grep '/tmp/qs-bench/' /tmp/t.log) | sort | uniq -c
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! Group by thread id instead (`grep -oP '^\d+ [a-z0-9_]+'`) to see the split
|
||||||
|
//! between the walk workers and the extraction thread.
|
||||||
|
//!
|
||||||
|
//! The run modes deliberately do no filesystem inspection of their own — no
|
||||||
|
//! progress walk, no size survey — so that every syscall the trace attributes
|
||||||
|
//! to the tree came from the indexer. The size histogram is printed by `gen`.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
use quicksearch_core::indexing::{IndexingService, IndexingStatus};
|
||||||
|
|
||||||
|
/// Files whose head the walk reads in full at the default 8 KiB
|
||||||
|
/// `hash_length`, i.e. the ones extraction never needs to reopen.
|
||||||
|
const SMALL_TEXT: usize = 800;
|
||||||
|
/// Text files past `hash_length`, which extraction must still read.
|
||||||
|
const LARGE_TEXT: usize = 100;
|
||||||
|
/// No extractor claims these, so extraction resolves them without touching
|
||||||
|
/// the disk. A control group: their cost must not move.
|
||||||
|
const BINARY: usize = 100;
|
||||||
|
|
||||||
|
const WORDS: &[&str] = &[
|
||||||
|
"alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta",
|
||||||
|
"quick", "brown", "fox", "jumps", "over", "lazy", "dog", "indexer",
|
||||||
|
"rust", "cargo", "sqlite", "baloo", "tokenizer", "trigram", "snippet",
|
||||||
|
"ocean", "forest", "mountain", "river", "valley", "bridge", "tunnel",
|
||||||
|
"morning", "afternoon", "evening", "midnight", "yesterday", "today",
|
||||||
|
];
|
||||||
|
|
||||||
|
/// Deterministic so two runs index byte-identical trees and their timings are
|
||||||
|
/// comparable. Plain LCG — this only has to spread, not to be random.
|
||||||
|
struct Rng(u64);
|
||||||
|
|
||||||
|
impl Rng {
|
||||||
|
fn next(&mut self) -> u64 {
|
||||||
|
self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
|
||||||
|
self.0 >> 33
|
||||||
|
}
|
||||||
|
|
||||||
|
fn in_range(&mut self, lo: usize, hi: usize) -> usize {
|
||||||
|
lo + (self.next() as usize) % (hi - lo)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
let mode = std::env::args().nth(1).unwrap_or_default();
|
||||||
|
let tree = PathBuf::from(std::env::args().nth(2).expect("usage: indexprobe <gen|cold|warm> <tree> [db]"));
|
||||||
|
|
||||||
|
match mode.as_str() {
|
||||||
|
"gen" => generate(&tree),
|
||||||
|
"cold" | "warm" => {
|
||||||
|
let db = PathBuf::from(std::env::args().nth(3).expect("usage: indexprobe <cold|warm> <tree> <db>"));
|
||||||
|
if mode == "cold" {
|
||||||
|
for suffix in ["", "-wal", "-shm"] {
|
||||||
|
let _ = std::fs::remove_file(format!("{}{}", db.display(), suffix));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
run(&mode, &tree, &db);
|
||||||
|
}
|
||||||
|
_ => {
|
||||||
|
eprintln!("usage: indexprobe <gen|cold|warm> <tree> [db]");
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build a tree with a size mix that separates the three code paths, and
|
||||||
|
/// report it so results are self-describing.
|
||||||
|
fn generate(tree: &Path) {
|
||||||
|
let _ = std::fs::remove_dir_all(tree);
|
||||||
|
std::fs::create_dir_all(tree).expect("create tree");
|
||||||
|
|
||||||
|
let mut rng = Rng(0x5eed);
|
||||||
|
let (mut small_bytes, mut large_bytes, mut bin_bytes) = (0usize, 0usize, 0usize);
|
||||||
|
|
||||||
|
// Spread across subdirectories so the walk does real directory work
|
||||||
|
// rather than one enormous readdir.
|
||||||
|
for i in 0..SMALL_TEXT {
|
||||||
|
let dir = tree.join(format!("src/mod{}", i % 40));
|
||||||
|
std::fs::create_dir_all(&dir).expect("mkdir");
|
||||||
|
let ext = ["txt", "md", "rs", "json"][i % 4];
|
||||||
|
let size = rng.in_range(200, 8 * 1024);
|
||||||
|
let body = prose(&mut rng, size);
|
||||||
|
small_bytes += body.len();
|
||||||
|
std::fs::write(dir.join(format!("f{}.{}", i, ext)), body).expect("write");
|
||||||
|
}
|
||||||
|
|
||||||
|
for i in 0..LARGE_TEXT {
|
||||||
|
let dir = tree.join(format!("docs/set{}", i % 10));
|
||||||
|
std::fs::create_dir_all(&dir).expect("mkdir");
|
||||||
|
let size = rng.in_range(8 * 1024 + 1, 200 * 1024);
|
||||||
|
let body = prose(&mut rng, size);
|
||||||
|
large_bytes += body.len();
|
||||||
|
std::fs::write(dir.join(format!("doc{}.md", i)), body).expect("write");
|
||||||
|
}
|
||||||
|
|
||||||
|
for i in 0..BINARY {
|
||||||
|
let dir = tree.join(format!("assets/set{}", i % 10));
|
||||||
|
std::fs::create_dir_all(&dir).expect("mkdir");
|
||||||
|
let n = rng.in_range(1024, 50 * 1024);
|
||||||
|
let blob: Vec<u8> = (0..n).map(|_| (rng.next() & 0xff) as u8).collect();
|
||||||
|
bin_bytes += blob.len();
|
||||||
|
std::fs::write(dir.join(format!("blob{}.bin", i)), blob).expect("write");
|
||||||
|
}
|
||||||
|
|
||||||
|
let total = SMALL_TEXT + LARGE_TEXT + BINARY;
|
||||||
|
eprintln!("generated {} files under {}", total, tree.display());
|
||||||
|
eprintln!(
|
||||||
|
" text <= 8 KiB : {:5} files, {:8.1} MiB (head covers the whole file)",
|
||||||
|
SMALL_TEXT,
|
||||||
|
small_bytes as f64 / (1024.0 * 1024.0)
|
||||||
|
);
|
||||||
|
eprintln!(
|
||||||
|
" text > 8 KiB : {:5} files, {:8.1} MiB (extraction must read it)",
|
||||||
|
LARGE_TEXT,
|
||||||
|
large_bytes as f64 / (1024.0 * 1024.0)
|
||||||
|
);
|
||||||
|
eprintln!(
|
||||||
|
" binary : {:5} files, {:8.1} MiB (no extractor; control group)",
|
||||||
|
BINARY,
|
||||||
|
bin_bytes as f64 / (1024.0 * 1024.0)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn prose(rng: &mut Rng, target: usize) -> String {
|
||||||
|
let mut s = String::with_capacity(target + 16);
|
||||||
|
while s.len() < target {
|
||||||
|
s.push_str(WORDS[rng.next() as usize % WORDS.len()]);
|
||||||
|
s.push(if rng.next() % 12 == 0 { '\n' } else { ' ' });
|
||||||
|
}
|
||||||
|
s.truncate(target);
|
||||||
|
s
|
||||||
|
}
|
||||||
|
|
||||||
|
fn run(mode: &str, tree: &Path, db: &Path) {
|
||||||
|
let config = Config::default();
|
||||||
|
|
||||||
|
// `run_indexing` writes this marker only on a successful finish, so it is
|
||||||
|
// the one unambiguous completion signal — polling the status enum races,
|
||||||
|
// because a small tree finishes between two polls and `Idle` then means
|
||||||
|
// both "not started" and "already done".
|
||||||
|
if db.exists() {
|
||||||
|
let conn = rusqlite::Connection::open(db).expect("open db");
|
||||||
|
conn.execute("DELETE FROM schema_info WHERE key = 'last_full_index'", [])
|
||||||
|
.expect("clear marker");
|
||||||
|
}
|
||||||
|
|
||||||
|
let service = IndexingService::new();
|
||||||
|
let start = Instant::now();
|
||||||
|
service
|
||||||
|
.start_indexing(
|
||||||
|
vec![tree.to_string_lossy().into_owned()],
|
||||||
|
db.to_string_lossy().into_owned(),
|
||||||
|
config,
|
||||||
|
)
|
||||||
|
.expect("start indexing");
|
||||||
|
|
||||||
|
let deadline = Instant::now() + Duration::from_secs(600);
|
||||||
|
let mut done = false;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
if let IndexingStatus::Error(e) = service.get_status() {
|
||||||
|
panic!("indexing failed: {}", e);
|
||||||
|
}
|
||||||
|
if db.exists() {
|
||||||
|
if let Ok(conn) = rusqlite::Connection::open(db) {
|
||||||
|
if quicksearch_core::db::repo::get_last_full_index(&conn).is_some() {
|
||||||
|
done = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(5));
|
||||||
|
}
|
||||||
|
let elapsed = start.elapsed();
|
||||||
|
assert!(done, "indexing did not finish within the timeout");
|
||||||
|
service.stop_indexing().expect("stop");
|
||||||
|
|
||||||
|
let total = SMALL_TEXT + LARGE_TEXT + BINARY;
|
||||||
|
eprintln!(
|
||||||
|
"{}: {:?} ({:.0} files/sec over {} files)",
|
||||||
|
mode,
|
||||||
|
elapsed,
|
||||||
|
total as f64 / elapsed.as_secs_f64(),
|
||||||
|
total
|
||||||
|
);
|
||||||
|
}
|
||||||
126
crates/quicksearch-core/examples/walkprobe.rs
Normal file
|
|
@ -0,0 +1,126 @@
|
||||||
|
//! Times and syscall-counts the phase-1 walk, without touching a database.
|
||||||
|
//!
|
||||||
|
//! ```text
|
||||||
|
//! cargo build -p quicksearch-core --example walkprobe --release
|
||||||
|
//! ./target/release/examples/walkprobe <root> parallel # the threaded walker
|
||||||
|
//! ./target/release/examples/walkprobe <root> serial # one thread, for comparison
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! Indexing a network share is bound by round trips, not bandwidth: every
|
||||||
|
//! metadata operation that misses the client cache costs one, and throughput
|
||||||
|
//! is round-trips-in-flight divided by latency. So the number that matters is
|
||||||
|
//! syscalls per file, which this makes directly visible:
|
||||||
|
//!
|
||||||
|
//! ```text
|
||||||
|
//! strace -f -c -e trace=openat,statx,newfstatat,readlink,getdents64,read,lseek,close \
|
||||||
|
//! ./target/release/examples/walkprobe <root> parallel
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! Expect roughly one `statx` per unchanged file, plus open/read/close for
|
||||||
|
//! files that are new or modified, and `readlink` only for resolving the roots
|
||||||
|
//! themselves — a per-file `readlink` count means a `canonicalize` has crept
|
||||||
|
//! back into the hot path.
|
||||||
|
//!
|
||||||
|
//! Those four syscalls also now cover the *whole* cost of a small text file:
|
||||||
|
//! the head read for the hash is the file's entire contents, so the walk
|
||||||
|
//! extracts its text there and the content pass never opens it again. That
|
||||||
|
//! work is CPU, not syscalls, so it shows up in files/sec here and not in the
|
||||||
|
//! trace. Use [`indexprobe`](indexprobe.rs) to see both phases together.
|
||||||
|
//!
|
||||||
|
//! Both modes report files/sec. Run each twice: the first pass warms the page
|
||||||
|
//! cache (or, on a share, the client's attribute cache), so the second is the
|
||||||
|
//! one to compare.
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::sync::atomic::AtomicBool;
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
use std::time::{Instant, UNIX_EPOCH};
|
||||||
|
|
||||||
|
use quicksearch_core::config::{Config, IgnoreSet};
|
||||||
|
use quicksearch_core::extract::Registry;
|
||||||
|
use quicksearch_core::file_handling::{
|
||||||
|
classify_for_indexing, filtered_walk, prepare_file_record, ExistingFileEntry, FileIndexAction,
|
||||||
|
UnreadableDirs,
|
||||||
|
};
|
||||||
|
use quicksearch_core::walk::walk_indexable_files;
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
let root = std::env::args().nth(1).unwrap();
|
||||||
|
let mode = std::env::args().nth(2).unwrap_or_else(|| "parallel".into());
|
||||||
|
let config = Config::default();
|
||||||
|
let existing: HashMap<String, ExistingFileEntry> = HashMap::new();
|
||||||
|
|
||||||
|
let start = Instant::now();
|
||||||
|
let (seen, prepared) = match mode.as_str() {
|
||||||
|
"serial" => serial(&root, &config, &existing),
|
||||||
|
_ => parallel(&root, &config, existing),
|
||||||
|
};
|
||||||
|
let elapsed = start.elapsed();
|
||||||
|
|
||||||
|
eprintln!(
|
||||||
|
"{mode}: {seen} files, {prepared} prepared in {:?} ({:.0} files/sec)",
|
||||||
|
elapsed,
|
||||||
|
seen as f64 / elapsed.as_secs_f64()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn serial(
|
||||||
|
root: &str,
|
||||||
|
config: &Config,
|
||||||
|
existing: &HashMap<String, ExistingFileEntry>,
|
||||||
|
) -> (usize, usize) {
|
||||||
|
let ignore = IgnoreSet::compile(&[]).unwrap();
|
||||||
|
let registry = Registry::default_set();
|
||||||
|
let (mut seen, mut prepared) = (0, 0);
|
||||||
|
for entry in filtered_walk(root, false, false, &ignore, &UnreadableDirs::default()) {
|
||||||
|
seen += 1;
|
||||||
|
// Same rule as the real walk: a name that is not valid UTF-8 cannot be
|
||||||
|
// stored in `files.path` and reopened by it, so it is skipped before
|
||||||
|
// anything tries to hash it. Counted as seen, never prepared.
|
||||||
|
let Some(path) = entry.path().to_str().map(str::to_owned) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let Ok(meta) = std::fs::metadata(entry.path()) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let Some(mtime) = meta
|
||||||
|
.modified()
|
||||||
|
.ok()
|
||||||
|
.and_then(|t| t.duration_since(UNIX_EPOCH).ok())
|
||||||
|
.map(|d| d.as_secs())
|
||||||
|
else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if classify_for_indexing(&path, mtime, existing) != FileIndexAction::Skip
|
||||||
|
&& prepare_file_record(&path, &meta, config, ®istry).is_some()
|
||||||
|
{
|
||||||
|
prepared += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(seen, prepared)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parallel(
|
||||||
|
root: &str,
|
||||||
|
config: &Config,
|
||||||
|
existing: HashMap<String, ExistingFileEntry>,
|
||||||
|
) -> (usize, usize) {
|
||||||
|
let (mut seen, mut prepared) = (0, 0);
|
||||||
|
for file in walk_indexable_files(
|
||||||
|
&[root.to_string()],
|
||||||
|
false,
|
||||||
|
false,
|
||||||
|
IgnoreSet::compile(&[]).unwrap(),
|
||||||
|
Arc::new(existing),
|
||||||
|
config.clone(),
|
||||||
|
Arc::new(Registry::default_set()),
|
||||||
|
Arc::new(Mutex::new(false)),
|
||||||
|
Arc::new(AtomicBool::new(false)),
|
||||||
|
4,
|
||||||
|
) {
|
||||||
|
seen += 1;
|
||||||
|
if file.record.is_some() {
|
||||||
|
prepared += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(seen, prepared)
|
||||||
|
}
|
||||||
|
|
@ -8,7 +8,7 @@
|
||||||
|
|
||||||
use rusqlite::{params, OptionalExtension};
|
use rusqlite::{params, OptionalExtension};
|
||||||
|
|
||||||
use crate::db::open_or_recreate;
|
use crate::db::open_existing;
|
||||||
use crate::db::repo::{STATE_DONE, STATE_FAILED, STATE_NA, STATE_PENDING};
|
use crate::db::repo::{STATE_DONE, STATE_FAILED, STATE_NA, STATE_PENDING};
|
||||||
|
|
||||||
/// Per-file indexing status, mirroring Baloo's multi-state reporting.
|
/// Per-file indexing status, mirroring Baloo's multi-state reporting.
|
||||||
|
|
@ -86,7 +86,7 @@ impl SizeReport {
|
||||||
/// Query the per-file indexing status. Returns `FileStatus` with
|
/// Query the per-file indexing status. Returns `FileStatus` with
|
||||||
/// `basic == NotIndexed` if the path isn't in the database.
|
/// `basic == NotIndexed` if the path isn't in the database.
|
||||||
pub fn status_for_path(db_path: &str, path: &str) -> Result<FileStatus, String> {
|
pub fn status_for_path(db_path: &str, path: &str) -> Result<FileStatus, String> {
|
||||||
let conn = open_or_recreate(db_path, "trigram")?;
|
let conn = open_existing(db_path, false)?;
|
||||||
let row: Option<(i64, i64, Option<String>)> = conn
|
let row: Option<(i64, i64, Option<String>)> = conn
|
||||||
.query_row(
|
.query_row(
|
||||||
"SELECT basic_state, content_state, failure_msg FROM files WHERE path = ?1",
|
"SELECT basic_state, content_state, failure_msg FROM files WHERE path = ?1",
|
||||||
|
|
@ -113,7 +113,7 @@ pub fn status_for_path(db_path: &str, path: &str) -> Result<FileStatus, String>
|
||||||
|
|
||||||
/// Return every file that failed content extraction, newest first.
|
/// Return every file that failed content extraction, newest first.
|
||||||
pub fn list_failed(db_path: &str, limit: Option<u32>) -> Result<Vec<FailedEntry>, String> {
|
pub fn list_failed(db_path: &str, limit: Option<u32>) -> Result<Vec<FailedEntry>, String> {
|
||||||
let conn = open_or_recreate(db_path, "trigram")?;
|
let conn = open_existing(db_path, false)?;
|
||||||
let limit_sql = match limit {
|
let limit_sql = match limit {
|
||||||
Some(n) => format!(" LIMIT {}", n),
|
Some(n) => format!(" LIMIT {}", n),
|
||||||
None => String::new(),
|
None => String::new(),
|
||||||
|
|
@ -147,7 +147,7 @@ pub fn index_size_breakdown(db_path: &str) -> Result<SizeReport, String> {
|
||||||
let file_size_bytes = std::fs::metadata(db_path)
|
let file_size_bytes = std::fs::metadata(db_path)
|
||||||
.map(|m| m.len())
|
.map(|m| m.len())
|
||||||
.unwrap_or(0);
|
.unwrap_or(0);
|
||||||
let conn = open_or_recreate(db_path, "trigram")?;
|
let conn = open_existing(db_path, false)?;
|
||||||
let count = |table: &str| -> Result<i64, String> {
|
let count = |table: &str| -> Result<i64, String> {
|
||||||
conn.query_row(&format!("SELECT COUNT(*) FROM {}", table), [], |r| r.get(0))
|
conn.query_row(&format!("SELECT COUNT(*) FROM {}", table), [], |r| r.get(0))
|
||||||
.map_err(|e| format!("count {}: {}", table, e))
|
.map_err(|e| format!("count {}: {}", table, e))
|
||||||
|
|
@ -181,7 +181,7 @@ pub fn index_size_breakdown(db_path: &str) -> Result<SizeReport, String> {
|
||||||
/// Used by the Baloo compat daemon to report the "Files waiting for content
|
/// Used by the Baloo compat daemon to report the "Files waiting for content
|
||||||
/// indexing" figure both to balooctl and to the LMDB mirror.
|
/// indexing" figure both to balooctl and to the LMDB mirror.
|
||||||
pub fn pending_content_count(db_path: &str) -> Result<i64, String> {
|
pub fn pending_content_count(db_path: &str) -> Result<i64, String> {
|
||||||
let conn = open_or_recreate(db_path, "trigram")?;
|
let conn = open_existing(db_path, false)?;
|
||||||
conn.query_row(
|
conn.query_row(
|
||||||
"SELECT COUNT(*) FROM files WHERE content_state = ?1",
|
"SELECT COUNT(*) FROM files WHERE content_state = ?1",
|
||||||
rusqlite::params![crate::db::repo::STATE_PENDING],
|
rusqlite::params![crate::db::repo::STATE_PENDING],
|
||||||
|
|
@ -190,10 +190,33 @@ pub fn pending_content_count(db_path: &str) -> Result<i64, String> {
|
||||||
.map_err(|e| format!("pending_content_count: {}", e))
|
.map_err(|e| format!("pending_content_count: {}", e))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Cheap aggregate counts for the GUI's idle status bar ("N files
|
||||||
|
/// indexed"). Callers cache the result; it's three COUNT scans, not
|
||||||
|
/// something to run per frame.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub struct IndexCounts {
|
||||||
|
pub files: i64,
|
||||||
|
pub content_done: i64,
|
||||||
|
pub content_pending: i64,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn index_counts(db_path: &str) -> Result<IndexCounts, String> {
|
||||||
|
let conn = open_existing(db_path, false)?;
|
||||||
|
let count = |sql: &str| -> Result<i64, String> {
|
||||||
|
conn.query_row(sql, [], |r| r.get(0))
|
||||||
|
.map_err(|e| format!("index_counts: {}", e))
|
||||||
|
};
|
||||||
|
Ok(IndexCounts {
|
||||||
|
files: count("SELECT COUNT(*) FROM files")?,
|
||||||
|
content_done: count("SELECT COUNT(*) FROM files WHERE content_state = 1")?,
|
||||||
|
content_pending: count("SELECT COUNT(*) FROM files WHERE content_state = 0")?,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
/// Remove a single file from the index. Returns whether a row was deleted.
|
/// Remove a single file from the index. Returns whether a row was deleted.
|
||||||
/// Keeps FTS/documents/properties in sync via the repo helpers.
|
/// Keeps FTS/documents/properties in sync via the repo helpers.
|
||||||
pub fn clear_path(db_path: &str, path: &str) -> Result<bool, String> {
|
pub fn clear_path(db_path: &str, path: &str) -> Result<bool, String> {
|
||||||
let mut conn = open_or_recreate(db_path, "trigram")?;
|
let mut conn = open_existing(db_path, true)?;
|
||||||
let tx = conn
|
let tx = conn
|
||||||
.transaction()
|
.transaction()
|
||||||
.map_err(|e| format!("clear_path begin tx: {}", e))?;
|
.map_err(|e| format!("clear_path begin tx: {}", e))?;
|
||||||
|
|
@ -206,6 +229,7 @@ pub fn clear_path(db_path: &str, path: &str) -> Result<bool, String> {
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use crate::db::open_or_recreate;
|
||||||
use crate::db::repo::{insert_file, set_content_done, set_content_failed, NewFile};
|
use crate::db::repo::{insert_file, set_content_done, set_content_failed, NewFile};
|
||||||
use crate::mime::FileType;
|
use crate::mime::FileType;
|
||||||
|
|
||||||
|
|
@ -374,4 +398,51 @@ mod tests {
|
||||||
|
|
||||||
std::fs::remove_file(&p).ok();
|
std::fs::remove_file(&p).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn clear_path_on_nondefault_tokenizer_db_removes_only_target() {
|
||||||
|
// Regression: clear_path used to open with a hardcoded "trigram", so on
|
||||||
|
// an index built with a non-default tokenizer the schema-mismatch wipe
|
||||||
|
// destroyed the WHOLE index instead of deleting one row. With
|
||||||
|
// open_existing it must delete only the target and leave the rest.
|
||||||
|
let p = tmp_path();
|
||||||
|
let dbp = p.to_str().unwrap();
|
||||||
|
{
|
||||||
|
let mut conn = open_or_recreate(dbp, "unicode61").unwrap();
|
||||||
|
let tx = conn.transaction().unwrap();
|
||||||
|
for (name, path) in [("a.txt", "/tmp/a.txt"), ("b.txt", "/tmp/b.txt")] {
|
||||||
|
insert_file(
|
||||||
|
&tx,
|
||||||
|
&NewFile {
|
||||||
|
name,
|
||||||
|
path,
|
||||||
|
parent: "/tmp",
|
||||||
|
size: 1,
|
||||||
|
mtime: 1,
|
||||||
|
inode: None,
|
||||||
|
device_id: None,
|
||||||
|
mime: Some("text/plain"),
|
||||||
|
ftype: FileType::TEXT,
|
||||||
|
hash: None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
.expect("unique path");
|
||||||
|
}
|
||||||
|
tx.commit().unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
assert!(clear_path(dbp, "/tmp/a.txt").unwrap());
|
||||||
|
// The other row must survive — proof we deleted one row, not wiped.
|
||||||
|
assert_eq!(
|
||||||
|
status_for_path(dbp, "/tmp/b.txt").unwrap().basic,
|
||||||
|
IndexState::Done
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
status_for_path(dbp, "/tmp/a.txt").unwrap().basic,
|
||||||
|
IndexState::NotIndexed
|
||||||
|
);
|
||||||
|
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
1019
crates/quicksearch-core/src/coordinator.rs
Normal file
|
|
@ -1,13 +1,15 @@
|
||||||
//! SQLite schema, on-disk open/recreate, and row-level repository helpers.
|
//! SQLite schema, on-disk open/recreate, and row-level repository helpers.
|
||||||
//!
|
//!
|
||||||
//! Policy: a single [`open::open_or_recreate`] is the only entry point. Any
|
//! Policy: the indexer (owner) opens via [`open::open_or_recreate`], which on
|
||||||
//! schema mismatch — wrong version, drifted tokenizer, absent `schema_info`
|
//! any schema mismatch — wrong version, drifted tokenizer, absent
|
||||||
//! — wipes the DB and rebuilds from [`schema::SCHEMA_CURRENT`]. There are
|
//! `schema_info` — wipes the DB and rebuilds from [`schema::SCHEMA_CURRENT`].
|
||||||
//! no in-place migrations by design; re-indexing is accepted as the cost
|
//! There are no in-place migrations by design; re-indexing is accepted as the
|
||||||
//! of avoiding migration-path complexity.
|
//! cost of avoiding migration-path complexity. *Consumers* (search, status,
|
||||||
|
//! size, `clear`) instead use [`open::open_existing`], which never creates or
|
||||||
|
//! wipes — a tokenizer difference or stale version is an error, not data loss.
|
||||||
|
|
||||||
pub mod open;
|
pub mod open;
|
||||||
pub mod repo;
|
pub mod repo;
|
||||||
pub mod schema;
|
pub mod schema;
|
||||||
|
|
||||||
pub use open::{open_or_recreate, CURRENT_SCHEMA_VERSION};
|
pub use open::{open_existing, open_or_recreate, CURRENT_SCHEMA_VERSION};
|
||||||
|
|
|
||||||
|
|
@ -15,9 +15,11 @@
|
||||||
|
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
||||||
use rusqlite::{params, Connection, OptionalExtension};
|
use rusqlite::{params, Connection, OpenFlags, OptionalExtension};
|
||||||
|
|
||||||
use super::schema::{effective_tokenizer, fts_create_sql, PRAGMAS_FAST, SCHEMA_CURRENT};
|
use super::schema::{
|
||||||
|
effective_tokenizer, fts_create_sql, PRAGMAS_FAST, PRAGMAS_READONLY, SCHEMA_CURRENT,
|
||||||
|
};
|
||||||
|
|
||||||
/// Bump this whenever [`SCHEMA_CURRENT`] or [`fts_create_sql`] changes in
|
/// Bump this whenever [`SCHEMA_CURRENT`] or [`fts_create_sql`] changes in
|
||||||
/// a way that makes an old DB unreadable by new code. Any such bump
|
/// a way that makes an old DB unreadable by new code. Any such bump
|
||||||
|
|
@ -34,6 +36,14 @@ pub const CURRENT_SCHEMA_VERSION: u32 = 3;
|
||||||
/// schema mismatch and triggers the wipe-and-recreate path.
|
/// schema mismatch and triggers the wipe-and-recreate path.
|
||||||
pub fn open_or_recreate(db_path: &str, tokenizer: &str) -> Result<Connection, String> {
|
pub fn open_or_recreate(db_path: &str, tokenizer: &str) -> Result<Connection, String> {
|
||||||
let path = Path::new(db_path).to_path_buf();
|
let path = Path::new(db_path).to_path_buf();
|
||||||
|
// The owner creates the directory too — a fresh install's default
|
||||||
|
// XDG data dir doesn't exist until first use.
|
||||||
|
if let Some(dir) = path.parent() {
|
||||||
|
if !dir.as_os_str().is_empty() {
|
||||||
|
std::fs::create_dir_all(dir)
|
||||||
|
.map_err(|e| format!("Failed to create database dir {}: {}", dir.display(), e))?;
|
||||||
|
}
|
||||||
|
}
|
||||||
let conn = Connection::open(db_path)
|
let conn = Connection::open(db_path)
|
||||||
.map_err(|e| format!("Failed to open database at {}: {}", db_path, e))?;
|
.map_err(|e| format!("Failed to open database at {}: {}", db_path, e))?;
|
||||||
conn.execute_batch(PRAGMAS_FAST)
|
conn.execute_batch(PRAGMAS_FAST)
|
||||||
|
|
@ -46,8 +56,8 @@ pub fn open_or_recreate(db_path: &str, tokenizer: &str) -> Result<Connection, St
|
||||||
// Schema is present but stale, or pre-existing rows belong to an
|
// Schema is present but stale, or pre-existing rows belong to an
|
||||||
// older layout, or the tokenizer drifted. Log once so the rebuild
|
// older layout, or the tokenizer drifted. Log once so the rebuild
|
||||||
// isn't silent, then wipe + recreate.
|
// isn't silent, then wipe + recreate.
|
||||||
eprintln!(
|
crate::log_warn!(
|
||||||
"QuickSearch: database at {} does not match current schema; rebuilding. \
|
"database at {} does not match current schema; rebuilding. \
|
||||||
Existing rows will be re-scanned on next indexing run.",
|
Existing rows will be re-scanned on next indexing run.",
|
||||||
db_path
|
db_path
|
||||||
);
|
);
|
||||||
|
|
@ -56,10 +66,47 @@ pub fn open_or_recreate(db_path: &str, tokenizer: &str) -> Result<Connection, St
|
||||||
Ok(conn)
|
Ok(conn)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// True iff the DB has `schema_info` with the current version *and* the
|
/// Open an *existing* index without ever recreating it. Verifies the schema
|
||||||
/// effective-tokenizer string this caller asked for. Anything else —
|
/// version matches this build; on any mismatch — missing file, no
|
||||||
/// missing table, wrong version, different tokenizer — returns false.
|
/// `schema_info`, wrong version — returns an error instead of wiping. The
|
||||||
fn db_matches_current(conn: &Connection, tokenizer: &str) -> Result<bool, String> {
|
/// on-disk FTS tokenizer is used as-is: a tokenizer difference is never a
|
||||||
|
/// reason to destroy a readable index.
|
||||||
|
///
|
||||||
|
/// `write == false` opens read-only; `write == true` opens read-write (for
|
||||||
|
/// row-level deletes like `clear`) but still never creates or wipes — there
|
||||||
|
/// is no `SQLITE_OPEN_CREATE`, so a missing file is a clean error.
|
||||||
|
///
|
||||||
|
/// Use this for every *consumer* (search, status, size, `clear`). Only the
|
||||||
|
/// indexer's own write path uses [`open_or_recreate`], which may wipe on a
|
||||||
|
/// genuine schema/tokenizer change it owns.
|
||||||
|
pub fn open_existing(db_path: &str, write: bool) -> Result<Connection, String> {
|
||||||
|
let flags = OpenFlags::SQLITE_OPEN_NO_MUTEX
|
||||||
|
| if write {
|
||||||
|
OpenFlags::SQLITE_OPEN_READ_WRITE
|
||||||
|
} else {
|
||||||
|
OpenFlags::SQLITE_OPEN_READ_ONLY
|
||||||
|
};
|
||||||
|
let conn = Connection::open_with_flags(db_path, flags)
|
||||||
|
.map_err(|e| format!("Failed to open database at {}: {}", db_path, e))?;
|
||||||
|
let pragmas = if write { PRAGMAS_FAST } else { PRAGMAS_READONLY };
|
||||||
|
conn.execute_batch(pragmas)
|
||||||
|
.map_err(|e| format!("Failed to apply pragmas: {}", e))?;
|
||||||
|
|
||||||
|
if !schema_version_current(&conn)? {
|
||||||
|
return Err(format!(
|
||||||
|
"index at {} is not a compatible QuickSearch index (schema v{} expected); \
|
||||||
|
refusing to modify it. Re-index to rebuild.",
|
||||||
|
db_path, CURRENT_SCHEMA_VERSION
|
||||||
|
));
|
||||||
|
}
|
||||||
|
Ok(conn)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// True iff the DB has a `schema_info` table whose `version` equals
|
||||||
|
/// [`CURRENT_SCHEMA_VERSION`]. Shared by the wipe decision
|
||||||
|
/// ([`db_matches_current`]) and the non-destructive [`open_existing`] path.
|
||||||
|
/// Deliberately ignores the tokenizer — that's only the owner's concern.
|
||||||
|
fn schema_version_current(conn: &Connection) -> Result<bool, String> {
|
||||||
let has_info: bool = conn
|
let has_info: bool = conn
|
||||||
.query_row(
|
.query_row(
|
||||||
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='schema_info'",
|
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='schema_info'",
|
||||||
|
|
@ -81,8 +128,14 @@ fn db_matches_current(conn: &Connection, tokenizer: &str) -> Result<bool, String
|
||||||
)
|
)
|
||||||
.optional()
|
.optional()
|
||||||
.map_err(|e| format!("read schema_info.version: {}", e))?;
|
.map_err(|e| format!("read schema_info.version: {}", e))?;
|
||||||
let version_ok = version.as_deref() == Some(&CURRENT_SCHEMA_VERSION.to_string());
|
Ok(version.as_deref() == Some(&CURRENT_SCHEMA_VERSION.to_string()))
|
||||||
if !version_ok {
|
}
|
||||||
|
|
||||||
|
/// True iff the DB has `schema_info` with the current version *and* the
|
||||||
|
/// effective-tokenizer string this caller asked for. Anything else —
|
||||||
|
/// missing table, wrong version, different tokenizer — returns false.
|
||||||
|
fn db_matches_current(conn: &Connection, tokenizer: &str) -> Result<bool, String> {
|
||||||
|
if !schema_version_current(conn)? {
|
||||||
return Ok(false);
|
return Ok(false);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -104,10 +157,22 @@ fn wipe_and_reopen(conn: Connection, path: &Path) -> Result<Connection, String>
|
||||||
drop(conn);
|
drop(conn);
|
||||||
// Primary file may already be absent (fresh open that just needed
|
// Primary file may already be absent (fresh open that just needed
|
||||||
// the table applied). Ignore NotFound; anything else is an error.
|
// the table applied). Ignore NotFound; anything else is an error.
|
||||||
match std::fs::remove_file(path) {
|
//
|
||||||
|
// `remove_file_retrying` matters on Windows, where a delete fails while
|
||||||
|
// *any* handle is open — most often an antivirus scanner reading the file
|
||||||
|
// in the moment after we closed it. Unix `unlink` never hits this, so the
|
||||||
|
// retry costs nothing there.
|
||||||
|
match crate::platform::remove_file_retrying(path) {
|
||||||
Ok(()) => {}
|
Ok(()) => {}
|
||||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
|
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
|
||||||
Err(e) => return Err(format!("Failed to remove old database: {}", e)),
|
Err(e) => {
|
||||||
|
return Err(format!(
|
||||||
|
"Failed to remove old database at {}: {}. \
|
||||||
|
Another QuickSearch instance may have the index open.",
|
||||||
|
path.display(),
|
||||||
|
e
|
||||||
|
))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
// Sidecars are optional — delete best-effort.
|
// Sidecars are optional — delete best-effort.
|
||||||
for suffix in ["-wal", "-shm", "-journal"] {
|
for suffix in ["-wal", "-shm", "-journal"] {
|
||||||
|
|
@ -116,7 +181,7 @@ fn wipe_and_reopen(conn: Connection, path: &Path) -> Result<Connection, String>
|
||||||
path.file_name().and_then(|s| s.to_str()).unwrap_or(""),
|
path.file_name().and_then(|s| s.to_str()).unwrap_or(""),
|
||||||
suffix
|
suffix
|
||||||
));
|
));
|
||||||
let _ = std::fs::remove_file(sidecar);
|
let _ = crate::platform::remove_file_retrying(&sidecar);
|
||||||
}
|
}
|
||||||
let conn = Connection::open(path)
|
let conn = Connection::open(path)
|
||||||
.map_err(|e| format!("Failed to reopen database after rebuild: {}", e))?;
|
.map_err(|e| format!("Failed to reopen database after rebuild: {}", e))?;
|
||||||
|
|
@ -326,4 +391,166 @@ mod tests {
|
||||||
drop(conn);
|
drop(conn);
|
||||||
std::fs::remove_file(&p).ok();
|
std::fs::remove_file(&p).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn open_existing_reads_nondefault_tokenizer_without_wiping() {
|
||||||
|
// The exact scenario that previously caused data loss: an index built
|
||||||
|
// with a non-default tokenizer, then opened by a *consumer* that only
|
||||||
|
// knows "trigram". `open_existing` must read it as-is and never wipe.
|
||||||
|
let p = tmp_db_path();
|
||||||
|
{
|
||||||
|
let conn = open_or_recreate(p.to_str().unwrap(), "unicode61").unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO files (name, path, parent, size, mtime) \
|
||||||
|
VALUES ('note', '/note.txt', '/', 0, 0)",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
// Seed the FTS index (rowid = the files row we just inserted) so a
|
||||||
|
// MATCH query can be exercised against the on-disk tokenizer.
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO searchabletext (rowid, name, text, properties) \
|
||||||
|
VALUES (last_insert_rowid(), 'note', 'hello world', '')",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
let conn = open_existing(p.to_str().unwrap(), false).unwrap();
|
||||||
|
let files: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
files, 1,
|
||||||
|
"open_existing must not wipe a non-default-tokenizer DB"
|
||||||
|
);
|
||||||
|
// The on-disk tokenizer is used as-is: a MATCH against the stored term
|
||||||
|
// returns the row.
|
||||||
|
let hits: i64 = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM searchabletext WHERE searchabletext MATCH 'hello'",
|
||||||
|
[],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(hits, 1);
|
||||||
|
// And the stored tokenizer is still the non-default one — proof we
|
||||||
|
// neither rewrote the FTS table nor reset schema_info.
|
||||||
|
let tok: String = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT value FROM schema_info WHERE key='tokenize'",
|
||||||
|
[],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(tok, "unicode61");
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn open_or_recreate_creates_missing_parent_dirs() {
|
||||||
|
// Fresh installs point at ~/.local/share/quicksearch/… which
|
||||||
|
// doesn't exist yet; the owner open must create it.
|
||||||
|
let mut dir = std::env::temp_dir();
|
||||||
|
dir.push(format!(
|
||||||
|
"qs-mkdir-{}-{}",
|
||||||
|
std::process::id(),
|
||||||
|
std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.unwrap()
|
||||||
|
.as_nanos()
|
||||||
|
));
|
||||||
|
let db = dir.join("nested/deeper/index.sqlite");
|
||||||
|
let conn = open_or_recreate(db.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
drop(conn);
|
||||||
|
assert!(db.exists());
|
||||||
|
std::fs::remove_dir_all(&dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn writable_opens_use_wal_and_it_persists() {
|
||||||
|
let p = tmp_db_path();
|
||||||
|
{
|
||||||
|
let conn = open_or_recreate(p.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
let mode: String = conn
|
||||||
|
.query_row("PRAGMA journal_mode", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(mode.to_lowercase(), "wal");
|
||||||
|
}
|
||||||
|
// WAL is persistent in the file: a later read-only consumer sees it
|
||||||
|
// without being able to (or needing to) set it.
|
||||||
|
let conn = open_existing(p.to_str().unwrap(), false).unwrap();
|
||||||
|
let mode: String = conn
|
||||||
|
.query_row("PRAGMA journal_mode", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(mode.to_lowercase(), "wal");
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn open_existing_errors_on_missing_file() {
|
||||||
|
let p = tmp_db_path();
|
||||||
|
assert!(!p.exists());
|
||||||
|
let res = open_existing(p.to_str().unwrap(), false);
|
||||||
|
assert!(res.is_err(), "missing file must error, not be created");
|
||||||
|
assert!(!p.exists(), "open_existing must not create the file");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn open_existing_errors_on_version_mismatch_without_wiping() {
|
||||||
|
// A DB from a prior schema version. A consumer opening it must get an
|
||||||
|
// error and leave the file untouched — the data is the owner's to
|
||||||
|
// rebuild, never a reader's to destroy.
|
||||||
|
let p = tmp_db_path();
|
||||||
|
{
|
||||||
|
let conn = Connection::open(&p).unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE schema_info (key TEXT PRIMARY KEY, value TEXT NOT NULL)",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO schema_info(key,value) VALUES('version','1')",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
conn.execute("CREATE TABLE files (id INTEGER PRIMARY KEY, name TEXT)", [])
|
||||||
|
.unwrap();
|
||||||
|
conn.execute("INSERT INTO files(name) VALUES('sentinel')", [])
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let res = open_existing(p.to_str().unwrap(), false);
|
||||||
|
assert!(res.is_err(), "stale schema version must error");
|
||||||
|
// Sentinel row still present → the file was not wiped.
|
||||||
|
let conn = Connection::open(&p).unwrap();
|
||||||
|
let n: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(n, 1, "open_existing must never delete on version mismatch");
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn open_existing_rw_allows_delete() {
|
||||||
|
let p = tmp_db_path();
|
||||||
|
{
|
||||||
|
let conn = open_or_recreate(p.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO files (name, path, parent, size, mtime) \
|
||||||
|
VALUES ('a', '/a', '/', 0, 0)",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let conn = open_existing(p.to_str().unwrap(), true).unwrap();
|
||||||
|
let removed = conn
|
||||||
|
.execute("DELETE FROM files WHERE path = '/a'", [])
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(removed, 1);
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -278,17 +278,41 @@ fn encode_properties_for_fts(props: &[(String, String)]) -> String {
|
||||||
buf
|
buf
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Flush and close a connection, restoring durable PRAGMAs. Call on clean
|
/// Flush the WAL into the main DB file and close. Call on clean shutdown so
|
||||||
/// shutdown so the next open sees a consistent DB.
|
/// the next open starts with an empty log. WAL mode itself is persistent in
|
||||||
|
/// the file — deliberately left on.
|
||||||
pub fn checkpoint_and_close(conn: Connection) {
|
pub fn checkpoint_and_close(conn: Connection) {
|
||||||
let _ = conn.execute_batch(
|
let _ = conn.execute_batch("PRAGMA wal_checkpoint(TRUNCATE);");
|
||||||
"PRAGMA journal_mode = DELETE; \
|
|
||||||
PRAGMA synchronous = FULL; \
|
|
||||||
PRAGMA wal_checkpoint(FULL);",
|
|
||||||
);
|
|
||||||
drop(conn);
|
drop(conn);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Read the `last_full_index` marker (unix seconds of the last *successful*
|
||||||
|
/// full indexing run) from `schema_info`. Absent key — fresh DB, or a DB
|
||||||
|
/// from before this marker existed — means "never".
|
||||||
|
pub fn get_last_full_index(conn: &Connection) -> Option<u64> {
|
||||||
|
conn.query_row(
|
||||||
|
"SELECT value FROM schema_info WHERE key = 'last_full_index'",
|
||||||
|
[],
|
||||||
|
|r| r.get::<_, String>(0),
|
||||||
|
)
|
||||||
|
.optional()
|
||||||
|
.ok()
|
||||||
|
.flatten()
|
||||||
|
.and_then(|v| v.parse().ok())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stamp `last_full_index` with `ts` (unix seconds). Called at the end of
|
||||||
|
/// every successful full indexing run; the coordinator reads it to schedule
|
||||||
|
/// periodic reindexing.
|
||||||
|
pub fn set_last_full_index(conn: &Connection, ts: u64) -> Result<(), String> {
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR REPLACE INTO schema_info(key, value) VALUES ('last_full_index', ?1)",
|
||||||
|
params![ts.to_string()],
|
||||||
|
)
|
||||||
|
.map_err(|e| format!("write last_full_index: {}", e))?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
@ -480,6 +504,59 @@ mod tests {
|
||||||
std::fs::remove_file(&p).ok();
|
std::fs::remove_file(&p).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn last_full_index_round_trip() {
|
||||||
|
let p = tmp_path();
|
||||||
|
let conn = open_or_recreate(p.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
assert_eq!(get_last_full_index(&conn), None, "fresh DB has no marker");
|
||||||
|
set_last_full_index(&conn, 1_700_000_123).unwrap();
|
||||||
|
assert_eq!(get_last_full_index(&conn), Some(1_700_000_123));
|
||||||
|
// Overwrite, not accumulate.
|
||||||
|
set_last_full_index(&conn, 1_700_000_999).unwrap();
|
||||||
|
assert_eq!(get_last_full_index(&conn), Some(1_700_000_999));
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn checkpoint_and_close_truncates_wal() {
|
||||||
|
let p = tmp_path();
|
||||||
|
let mut conn = open_or_recreate(p.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
{
|
||||||
|
let tx = conn.transaction().unwrap();
|
||||||
|
insert_file(
|
||||||
|
&tx,
|
||||||
|
&NewFile {
|
||||||
|
name: "w.txt",
|
||||||
|
path: "/tmp/w.txt",
|
||||||
|
parent: "/tmp",
|
||||||
|
size: 1,
|
||||||
|
mtime: 1,
|
||||||
|
inode: None,
|
||||||
|
device_id: None,
|
||||||
|
mime: None,
|
||||||
|
ftype: FileType::EMPTY,
|
||||||
|
hash: None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
tx.commit().unwrap();
|
||||||
|
}
|
||||||
|
checkpoint_and_close(conn);
|
||||||
|
// After a TRUNCATE checkpoint + close of the last connection the WAL
|
||||||
|
// sidecar is gone or empty; the row lives in the main file.
|
||||||
|
let wal = std::path::PathBuf::from(format!("{}-wal", p.display()));
|
||||||
|
let wal_len = std::fs::metadata(&wal).map(|m| m.len()).unwrap_or(0);
|
||||||
|
assert_eq!(wal_len, 0, "WAL should be truncated on clean close");
|
||||||
|
let conn = crate::db::open_existing(p.to_str().unwrap(), false).unwrap();
|
||||||
|
let n: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(n, 1);
|
||||||
|
drop(conn);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn set_content_failed_writes_failed_table() {
|
fn set_content_failed_writes_failed_table() {
|
||||||
let p = tmp_path();
|
let p = tmp_path();
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,32 @@
|
||||||
//! SQL strings for the current schema. Versioned; [`migrate`](super::migrate)
|
//! SQL strings for the current schema. Versioned; [`migrate`](super::migrate)
|
||||||
//! drives the upgrade path.
|
//! drives the upgrade path.
|
||||||
|
|
||||||
/// Pragmas applied on every connection open. Tuned for write throughput during
|
/// Pragmas applied on every writable connection open.
|
||||||
/// indexing; a clean shutdown re-enables journal_mode/synchronous via
|
///
|
||||||
|
/// WAL, not journal-off: auto-indexing writes continuously while searches
|
||||||
|
/// stream from their own read-only connections, and WAL is what lets those
|
||||||
|
/// readers proceed without ever blocking the writer (or vice versa).
|
||||||
|
/// `synchronous = NORMAL` under WAL risks only the last commit on power
|
||||||
|
/// loss — acceptable for an index that is re-derivable from disk. Only two
|
||||||
|
/// writers exist (full index runs and the coordinator's incremental
|
||||||
|
/// updates) and they're serialized by design; `busy_timeout` is a backstop,
|
||||||
|
/// not a coordination mechanism. A clean shutdown truncates the log via
|
||||||
/// [`super::repo::checkpoint_and_close`].
|
/// [`super::repo::checkpoint_and_close`].
|
||||||
pub const PRAGMAS_FAST: &str = "
|
pub const PRAGMAS_FAST: &str = "
|
||||||
PRAGMA journal_mode = OFF;
|
PRAGMA journal_mode = WAL;
|
||||||
PRAGMA synchronous = 0;
|
PRAGMA synchronous = NORMAL;
|
||||||
|
PRAGMA busy_timeout = 5000;
|
||||||
|
PRAGMA cache_size = 10000;
|
||||||
|
PRAGMA temp_store = MEMORY;
|
||||||
|
PRAGMA foreign_keys = ON;
|
||||||
|
";
|
||||||
|
|
||||||
|
/// Pragmas safe to apply on a read-only connection, where `journal_mode`
|
||||||
|
/// and `synchronous` can't be changed on the file. Used by
|
||||||
|
/// [`super::open::open_existing`] for read-only opens; write paths get the
|
||||||
|
/// full [`PRAGMAS_FAST`] set.
|
||||||
|
pub const PRAGMAS_READONLY: &str = "
|
||||||
|
PRAGMA busy_timeout = 5000;
|
||||||
PRAGMA cache_size = 10000;
|
PRAGMA cache_size = 10000;
|
||||||
PRAGMA temp_store = MEMORY;
|
PRAGMA temp_store = MEMORY;
|
||||||
PRAGMA foreign_keys = ON;
|
PRAGMA foreign_keys = ON;
|
||||||
|
|
|
||||||
|
|
@ -2,8 +2,15 @@
|
||||||
//!
|
//!
|
||||||
//! An [`Extractor`] decides whether it can handle a given MIME type and, if
|
//! An [`Extractor`] decides whether it can handle a given MIME type and, if
|
||||||
//! so, produces [`ExtractedContent`] for the file. The [`Registry`] picks the
|
//! so, produces [`ExtractedContent`] for the file. The [`Registry`] picks the
|
||||||
//! first registered extractor that accepts the MIME and runs it. Callers can
|
//! first registered extractor that accepts the MIME and runs it.
|
||||||
//! also fall back to an extension-based match for files with no detected MIME.
|
//!
|
||||||
|
//! Dispatch is by MIME only — a file with no detected type is recorded as
|
||||||
|
//! "not applicable" rather than guessed at again here. Extensions that
|
||||||
|
//! `mime_guess` misses or mistypes are corrected upstream instead, in
|
||||||
|
//! [`crate::mime::guess_mime_from_head`], so there is one place where "what is
|
||||||
|
//! this file" gets decided, and it is decided once: the walk sniffs the head it
|
||||||
|
//! already read, stores the answer, and nothing downstream reopens the file to
|
||||||
|
//! ask again.
|
||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
@ -70,6 +77,32 @@ pub trait Extractor: Send + Sync {
|
||||||
/// [`ExtractError`] to mark the file's content state as failed (so it
|
/// [`ExtractError`] to mark the file's content state as failed (so it
|
||||||
/// won't be retried every run).
|
/// won't be retried every run).
|
||||||
fn extract(&self, path: &Path) -> Result<ExtractedContent, ExtractError>;
|
fn extract(&self, path: &Path) -> Result<ExtractedContent, ExtractError>;
|
||||||
|
|
||||||
|
/// Extract from bytes the caller already holds, when those bytes are the
|
||||||
|
/// file's *entire* contents.
|
||||||
|
///
|
||||||
|
/// Indexing hashes the head of every new or changed file, so for anything
|
||||||
|
/// no larger than `hash_length` the whole file is already in memory by the
|
||||||
|
/// time the walk classifies it. An extractor that can work from that buffer
|
||||||
|
/// saves the content pass an open/read/close — several round trips per
|
||||||
|
/// file on a network share — and closes a consistency gap, because the
|
||||||
|
/// text then comes from the same `read` as the size, mtime and hash stored
|
||||||
|
/// alongside it.
|
||||||
|
///
|
||||||
|
/// The default is `None`: "I need the file on disk." Formats that seek,
|
||||||
|
/// or that read a central directory at the end of the file, must keep it.
|
||||||
|
/// Returning `Some(Err(_))` is a real extraction failure, recorded like
|
||||||
|
/// any other; returning `None` simply defers to [`Extractor::extract`].
|
||||||
|
///
|
||||||
|
/// `path` is passed only so failures name the same file the on-disk path
|
||||||
|
/// would — nothing here may open it.
|
||||||
|
fn extract_from_head(
|
||||||
|
&self,
|
||||||
|
_path: &Path,
|
||||||
|
_head: &[u8],
|
||||||
|
) -> Option<Result<ExtractedContent, ExtractError>> {
|
||||||
|
None
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// An ordered dispatch table of extractors. The first extractor whose
|
/// An ordered dispatch table of extractors. The first extractor whose
|
||||||
|
|
@ -105,6 +138,28 @@ impl Registry {
|
||||||
Ok(None)
|
Ok(None)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// [`Registry::extract`] for a file whose complete contents the caller
|
||||||
|
/// already holds. `Ok(None)` when no extractor claims the MIME, and
|
||||||
|
/// `None` when the one that does needs the file on disk after all —
|
||||||
|
/// both mean "leave this to the content pass".
|
||||||
|
///
|
||||||
|
/// Dispatch stays here rather than at the call site so there is still
|
||||||
|
/// exactly one place that decides what an extractor sees for a given MIME.
|
||||||
|
pub fn extract_complete_head(
|
||||||
|
&self,
|
||||||
|
path: &Path,
|
||||||
|
mime: &str,
|
||||||
|
head: &[u8],
|
||||||
|
) -> Option<Result<ExtractedContent, ExtractError>> {
|
||||||
|
let lower = mime.to_ascii_lowercase();
|
||||||
|
for e in &self.extractors {
|
||||||
|
if e.supports(&lower) {
|
||||||
|
return e.extract_from_head(path, head);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
/// The default set wired up for Set A: plaintext, office docs, PDF,
|
/// The default set wired up for Set A: plaintext, office docs, PDF,
|
||||||
/// audio tags, image EXIF.
|
/// audio tags, image EXIF.
|
||||||
pub fn default_set() -> Self {
|
pub fn default_set() -> Self {
|
||||||
|
|
@ -136,6 +191,38 @@ mod tests {
|
||||||
assert!(out.is_none());
|
assert!(out.is_none());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn complete_head_extraction_dispatches_only_to_extractors_that_opt_in() {
|
||||||
|
let r = Registry::default_set();
|
||||||
|
let p = Path::new("/tmp/whatever");
|
||||||
|
|
||||||
|
// Plaintext opts in, so a small text file never reaches the disk pass.
|
||||||
|
let out = r.extract_complete_head(p, "text/plain", b"hello");
|
||||||
|
assert!(matches!(out, Some(Ok(ref c)) if c.text == "hello"));
|
||||||
|
|
||||||
|
// A format that seeks or reads a trailer must not be handed a buffer.
|
||||||
|
// `None` here is what routes it back to the on-disk extractor.
|
||||||
|
assert!(r.extract_complete_head(p, "application/pdf", b"%PDF-1.4").is_none());
|
||||||
|
assert!(r.extract_complete_head(p, "image/png", b"\x89PNG").is_none());
|
||||||
|
|
||||||
|
// No extractor claims the MIME at all.
|
||||||
|
assert!(r.extract_complete_head(p, "application/x-nonesuch", b"..").is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn complete_head_extraction_matches_the_on_disk_dispatch() {
|
||||||
|
// Both entry points must pick the same extractor for a MIME, or a
|
||||||
|
// file's text would depend on which pass happened to handle it.
|
||||||
|
let r = Registry::default_set();
|
||||||
|
let p = Path::new("/tmp/whatever");
|
||||||
|
for mime in ["text/plain", "TEXT/PLAIN", "application/json", "application/x-sql"] {
|
||||||
|
assert!(
|
||||||
|
r.extract_complete_head(p, mime, b"x").is_some(),
|
||||||
|
"{} should extract from a head", mime
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn properties_sorted_is_deterministic() {
|
fn properties_sorted_is_deterministic() {
|
||||||
let c = ExtractedContent::with_text("hi")
|
let c = ExtractedContent::with_text("hi")
|
||||||
|
|
|
||||||
|
|
@ -6,12 +6,48 @@
|
||||||
//! failed. Properties (title, author, etc.) from the PDF `Info` dictionary
|
//! failed. Properties (title, author, etc.) from the PDF `Info` dictionary
|
||||||
//! are pulled via `lopdf` where available.
|
//! are pulled via `lopdf` where available.
|
||||||
|
|
||||||
|
use std::cell::Cell;
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
use std::sync::OnceLock;
|
||||||
|
|
||||||
use lopdf::{Document as LopdfDocument, Object};
|
use lopdf::{Document as LopdfDocument, Object};
|
||||||
|
|
||||||
use super::{ExtractError, ExtractedContent, Extractor};
|
use super::{ExtractError, ExtractedContent, Extractor};
|
||||||
|
|
||||||
|
thread_local! {
|
||||||
|
/// True while this thread is inside a contained `pdf_extract` call.
|
||||||
|
static SUPPRESS_PANIC_PRINT: Cell<bool> = const { Cell::new(false) };
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Chain a process panic hook (once) that swallows the default
|
||||||
|
/// "thread panicked at …" report while this thread is inside a *contained*
|
||||||
|
/// PDF extraction — those panics are expected on malformed PDFs, caught,
|
||||||
|
/// and recorded as the file's failure reason, so printing each one is pure
|
||||||
|
/// console spam. Panics anywhere else print exactly as before.
|
||||||
|
fn install_quiet_panic_hook() {
|
||||||
|
static INSTALLED: OnceLock<()> = OnceLock::new();
|
||||||
|
INSTALLED.get_or_init(|| {
|
||||||
|
let previous = std::panic::take_hook();
|
||||||
|
std::panic::set_hook(Box::new(move |info| {
|
||||||
|
if !SUPPRESS_PANIC_PRINT.with(|flag| flag.get()) {
|
||||||
|
previous(info);
|
||||||
|
}
|
||||||
|
}));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Human-readable message from a caught panic payload; lands in
|
||||||
|
/// `failed_files.reason`.
|
||||||
|
fn panic_message(payload: &(dyn std::any::Any + Send)) -> String {
|
||||||
|
if let Some(s) = payload.downcast_ref::<&str>() {
|
||||||
|
(*s).to_string()
|
||||||
|
} else if let Some(s) = payload.downcast_ref::<String>() {
|
||||||
|
s.clone()
|
||||||
|
} else {
|
||||||
|
"unknown panic".to_string()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct PdfExtractor;
|
pub struct PdfExtractor;
|
||||||
|
|
||||||
impl Extractor for PdfExtractor {
|
impl Extractor for PdfExtractor {
|
||||||
|
|
@ -20,10 +56,15 @@ impl Extractor for PdfExtractor {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn extract(&self, path: &Path) -> Result<ExtractedContent, ExtractError> {
|
fn extract(&self, path: &Path) -> Result<ExtractedContent, ExtractError> {
|
||||||
// Text. Catch panics from pdf_extract (some PDFs crash its parser).
|
// Text. Catch panics from pdf_extract (some PDFs crash its parser)
|
||||||
|
// and keep the default hook from spamming stderr about them.
|
||||||
|
install_quiet_panic_hook();
|
||||||
let path_buf = path.to_path_buf();
|
let path_buf = path.to_path_buf();
|
||||||
let text = std::panic::catch_unwind(move || pdf_extract::extract_text(&path_buf))
|
SUPPRESS_PANIC_PRINT.with(|flag| flag.set(true));
|
||||||
.map_err(|_| "pdf_extract panicked".to_string())?
|
let result = std::panic::catch_unwind(move || pdf_extract::extract_text(&path_buf));
|
||||||
|
SUPPRESS_PANIC_PRINT.with(|flag| flag.set(false));
|
||||||
|
let text = result
|
||||||
|
.map_err(|panic| format!("pdf_extract panicked: {}", panic_message(&*panic)))?
|
||||||
.map_err(|e| format!("pdf_extract: {}", e))?;
|
.map_err(|e| format!("pdf_extract: {}", e))?;
|
||||||
|
|
||||||
let mut out = ExtractedContent::with_text(text);
|
let mut out = ExtractedContent::with_text(text);
|
||||||
|
|
@ -69,6 +110,20 @@ fn object_to_string(obj: &Object) -> Option<String> {
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn contained_panics_are_caught_quietly_with_reason() {
|
||||||
|
install_quiet_panic_hook();
|
||||||
|
SUPPRESS_PANIC_PRINT.with(|flag| flag.set(true));
|
||||||
|
let result = std::panic::catch_unwind(|| panic!("synthetic pdf failure"));
|
||||||
|
SUPPRESS_PANIC_PRINT.with(|flag| flag.set(false));
|
||||||
|
let payload = result.expect_err("must panic");
|
||||||
|
assert_eq!(panic_message(&*payload), "synthetic pdf failure");
|
||||||
|
// Panics outside the suppression window keep printing: the flag is
|
||||||
|
// thread-local and cleared, so nothing here can silence other
|
||||||
|
// threads or later tests.
|
||||||
|
assert!(!SUPPRESS_PANIC_PRINT.with(|flag| flag.get()));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn supports_pdf_mime() {
|
fn supports_pdf_mime() {
|
||||||
assert!(PdfExtractor.supports("application/pdf"));
|
assert!(PdfExtractor.supports("application/pdf"));
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,21 @@
|
||||||
//! Read the file as UTF-8 text. Handles text/plain, text/x-*, application/json
|
//! Read the file as UTF-8 text. Handles text/plain, text/x-*, application/json
|
||||||
//! and most source-code MIMEs.
|
//! and most source-code MIMEs.
|
||||||
|
|
||||||
|
use std::fs::File;
|
||||||
|
use std::io::Read;
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
||||||
use super::{ExtractError, ExtractedContent, Extractor};
|
use super::{ExtractError, ExtractedContent, Extractor};
|
||||||
|
|
||||||
|
/// Decode bytes that are known to be a complete file. Shared by both entry
|
||||||
|
/// points so on-disk and already-in-memory extraction cannot drift apart.
|
||||||
|
fn decode(bytes: Vec<u8>, path: &Path) -> Result<ExtractedContent, ExtractError> {
|
||||||
|
match String::from_utf8(bytes) {
|
||||||
|
Ok(text) => Ok(ExtractedContent::with_text(text)),
|
||||||
|
Err(e) => Err(format!("plaintext read {}: {}", path.display(), e.utf8_error())),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct PlaintextExtractor;
|
pub struct PlaintextExtractor;
|
||||||
|
|
||||||
impl Extractor for PlaintextExtractor {
|
impl Extractor for PlaintextExtractor {
|
||||||
|
|
@ -22,13 +33,66 @@ impl Extractor for PlaintextExtractor {
|
||||||
| "application/toml"
|
| "application/toml"
|
||||||
| "application/yaml"
|
| "application/yaml"
|
||||||
| "application/x-yaml"
|
| "application/x-yaml"
|
||||||
|
// `.sql` resolves here rather than to `text/*`, so without it
|
||||||
|
// schema dumps are listed by name but never full-text indexed.
|
||||||
|
| "application/x-sql"
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Read the whole file, sized from the handle we just opened.
|
||||||
|
///
|
||||||
|
/// `std::fs::read_to_string` would cost two extra syscalls here: a
|
||||||
|
/// path-based `statx` to size its buffer, and a second `read` returning 0,
|
||||||
|
/// because "read to EOF" can only observe EOF that way — `read_to_end`
|
||||||
|
/// terminates on `Ok(0)` alone, so a short read does not end it. Sizing
|
||||||
|
/// the buffer ourselves lets the loop finish on `filled == size` and issue
|
||||||
|
/// exactly one `read` for a file that fits.
|
||||||
|
///
|
||||||
|
/// A file that shrank between the `fstat` and the `read` keeps its prefix
|
||||||
|
/// rather than failing. A file that grew is read up to the size we saw;
|
||||||
|
/// its mtime moved, so the next run reclassifies it as changed and
|
||||||
|
/// re-extracts (see [`crate::file_handling::classify_for_indexing`]).
|
||||||
|
/// Neither case was ever atomic — a concurrent writer can tear a file
|
||||||
|
/// across any read sequence, including `read_to_string`'s.
|
||||||
fn extract(&self, path: &Path) -> Result<ExtractedContent, ExtractError> {
|
fn extract(&self, path: &Path) -> Result<ExtractedContent, ExtractError> {
|
||||||
let text = std::fs::read_to_string(path)
|
let mut f = File::open(path)
|
||||||
.map_err(|e| format!("plaintext read {}: {}", path.display(), e))?;
|
.map_err(|e| format!("plaintext read {}: {}", path.display(), e))?;
|
||||||
Ok(ExtractedContent::with_text(text))
|
let size = f
|
||||||
|
.metadata()
|
||||||
|
.map_err(|e| format!("plaintext read {}: {}", path.display(), e))?
|
||||||
|
.len() as usize;
|
||||||
|
|
||||||
|
// procfs, sysfs and some FUSE mounts report zero for files that do
|
||||||
|
// have content, so a sized read would store nothing. Only these pay
|
||||||
|
// the read-to-EOF probe — which is what a genuinely empty file cost
|
||||||
|
// before anyway.
|
||||||
|
if size == 0 {
|
||||||
|
let mut buf = Vec::new();
|
||||||
|
f.read_to_end(&mut buf)
|
||||||
|
.map_err(|e| format!("plaintext read {}: {}", path.display(), e))?;
|
||||||
|
return decode(buf, path);
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut buf = vec![0u8; size];
|
||||||
|
let mut filled = 0;
|
||||||
|
while filled < size {
|
||||||
|
match f.read(&mut buf[filled..]) {
|
||||||
|
Ok(0) => break,
|
||||||
|
Ok(n) => filled += n,
|
||||||
|
Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => {}
|
||||||
|
Err(e) => return Err(format!("plaintext read {}: {}", path.display(), e)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
buf.truncate(filled);
|
||||||
|
decode(buf, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn extract_from_head(
|
||||||
|
&self,
|
||||||
|
path: &Path,
|
||||||
|
head: &[u8],
|
||||||
|
) -> Option<Result<ExtractedContent, ExtractError>> {
|
||||||
|
Some(decode(head.to_vec(), path))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -36,23 +100,115 @@ impl Extractor for PlaintextExtractor {
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
#[test]
|
fn tmp(tag: &str, body: &[u8]) -> std::path::PathBuf {
|
||||||
fn reads_utf8_file() {
|
|
||||||
let mut p = std::env::temp_dir();
|
let mut p = std::env::temp_dir();
|
||||||
p.push(format!(
|
p.push(format!(
|
||||||
"qs-plaintext-{}-{}.txt",
|
"qs-plaintext-{}-{}-{}.txt",
|
||||||
|
tag,
|
||||||
std::process::id(),
|
std::process::id(),
|
||||||
std::time::SystemTime::now()
|
std::time::SystemTime::now()
|
||||||
.duration_since(std::time::UNIX_EPOCH)
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.as_nanos()
|
.as_nanos()
|
||||||
));
|
));
|
||||||
std::fs::write(&p, "hello world").unwrap();
|
std::fs::write(&p, body).unwrap();
|
||||||
|
p
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn reads_utf8_file() {
|
||||||
|
let p = tmp("basic", b"hello world");
|
||||||
let c = PlaintextExtractor.extract(&p).unwrap();
|
let c = PlaintextExtractor.extract(&p).unwrap();
|
||||||
assert_eq!(c.text, "hello world");
|
assert_eq!(c.text, "hello world");
|
||||||
std::fs::remove_file(&p).ok();
|
std::fs::remove_file(&p).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn head_extraction_matches_reading_the_file() {
|
||||||
|
let p = tmp("agree", b"shared body with unicode: caf\xc3\xa9 \xe2\x9c\x93");
|
||||||
|
let from_disk = PlaintextExtractor.extract(&p).unwrap();
|
||||||
|
let bytes = std::fs::read(&p).unwrap();
|
||||||
|
let from_head = PlaintextExtractor.extract_from_head(&p, &bytes).unwrap().unwrap();
|
||||||
|
assert_eq!(from_disk.text, from_head.text);
|
||||||
|
assert_eq!(from_disk.properties, from_head.properties);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn both_paths_reject_invalid_utf8_and_name_the_file() {
|
||||||
|
let p = tmp("badutf8", &[0x68, 0x69, 0xff, 0xfe]);
|
||||||
|
let disk_err = PlaintextExtractor.extract(&p).unwrap_err();
|
||||||
|
let head_err = PlaintextExtractor
|
||||||
|
.extract_from_head(&p, &[0x68, 0x69, 0xff, 0xfe])
|
||||||
|
.unwrap()
|
||||||
|
.unwrap_err();
|
||||||
|
assert_eq!(disk_err, head_err, "one decode path, one message");
|
||||||
|
assert!(disk_err.contains("badutf8"), "the failure names the file: {}", disk_err);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn reads_a_file_larger_than_one_buffer_completely() {
|
||||||
|
// Past any plausible head window, so the read loop has to iterate if
|
||||||
|
// the kernel returns a short read.
|
||||||
|
let body = "abcdefgh".repeat(200 * 1024 / 8);
|
||||||
|
let p = tmp("large", body.as_bytes());
|
||||||
|
let c = PlaintextExtractor.extract(&p).unwrap();
|
||||||
|
assert_eq!(c.text.len(), body.len());
|
||||||
|
assert_eq!(c.text, body);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_file_extracts_to_empty_text() {
|
||||||
|
let p = tmp("empty", b"");
|
||||||
|
assert_eq!(PlaintextExtractor.extract(&p).unwrap().text, "");
|
||||||
|
assert_eq!(
|
||||||
|
PlaintextExtractor.extract_from_head(&p, &[]).unwrap().unwrap().text,
|
||||||
|
""
|
||||||
|
);
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A file whose reported size is a lie in the "there is more than this"
|
||||||
|
/// direction — the shape procfs and sysfs have. Sizing the buffer from
|
||||||
|
/// `st_size` alone would store nothing, so `extract` must fall back to
|
||||||
|
/// reading until EOF.
|
||||||
|
#[test]
|
||||||
|
fn a_file_reporting_zero_size_is_still_read_to_eof() {
|
||||||
|
let p = Path::new("/proc/self/status");
|
||||||
|
if !p.exists() {
|
||||||
|
return; // not Linux; the guard is only reachable there
|
||||||
|
}
|
||||||
|
assert_eq!(
|
||||||
|
std::fs::metadata(p).unwrap().len(),
|
||||||
|
0,
|
||||||
|
"precondition: procfs reports zero size"
|
||||||
|
);
|
||||||
|
let c = PlaintextExtractor.extract(p).unwrap();
|
||||||
|
assert!(
|
||||||
|
c.text.contains("Name:"),
|
||||||
|
"content must survive a zero st_size, got {} bytes",
|
||||||
|
c.text.len()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The same lie in the other direction, which the sized read handles by
|
||||||
|
/// keeping whatever was actually there.
|
||||||
|
#[test]
|
||||||
|
fn a_file_that_shrank_after_sizing_keeps_its_prefix() {
|
||||||
|
let p = tmp("shrink", &vec![b'x'; 4096]);
|
||||||
|
let f = File::options().write(true).open(&p).unwrap();
|
||||||
|
// Truncate behind `extract`'s back is not reproducible, so assert the
|
||||||
|
// property directly: a buffer sized larger than the file yields the
|
||||||
|
// file, not an error.
|
||||||
|
f.set_len(10).unwrap();
|
||||||
|
drop(f);
|
||||||
|
let c = PlaintextExtractor.extract(&p).unwrap();
|
||||||
|
assert_eq!(c.text, "xxxxxxxxxx", "a shrunk file reads short, not fatal");
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn supports_text_mimes() {
|
fn supports_text_mimes() {
|
||||||
let e = PlaintextExtractor;
|
let e = PlaintextExtractor;
|
||||||
|
|
|
||||||
486
crates/quicksearch-core/src/incremental.rs
Normal file
|
|
@ -0,0 +1,486 @@
|
||||||
|
//! Incremental single-path index updates, driven by watcher events.
|
||||||
|
//!
|
||||||
|
//! One [`FsEvent`] becomes one (or a few) small transactions: files row,
|
||||||
|
//! `documents_text`, and FTS are updated together, so the index is
|
||||||
|
//! consistent after every commit. The same filters as the full walk apply
|
||||||
|
//! ([`IgnoreSet`], hidden components, `content_extensions`, size caps) —
|
||||||
|
//! a watcher event for something the walker would have skipped is a no-op.
|
||||||
|
//!
|
||||||
|
//! Renames are handled as remove + re-add: they're rare, and rewriting
|
||||||
|
//! `path`/`parent` strings plus re-tokenizing the FTS `name` column in
|
||||||
|
//! place is more machinery than re-extracting one file.
|
||||||
|
//!
|
||||||
|
//! Scope note: the watcher only reports paths under the configured roots,
|
||||||
|
//! so no root containment check is repeated here.
|
||||||
|
|
||||||
|
use std::path::Path;
|
||||||
|
|
||||||
|
use rusqlite::{Connection, OptionalExtension};
|
||||||
|
|
||||||
|
use crate::config::{content_allowed, Config, IgnoreSet};
|
||||||
|
use crate::db::repo;
|
||||||
|
use crate::extract::Registry;
|
||||||
|
use crate::file_handling::{
|
||||||
|
db_key_for_missing_path, extract_and_store, filtered_walk, UnreadableDirs,
|
||||||
|
prepare_file_record_from_path, store_inline_text,
|
||||||
|
};
|
||||||
|
use crate::platform::path_has_hidden_component_under;
|
||||||
|
use crate::query::translator::like_subtree_pattern;
|
||||||
|
use crate::watcher::FsEvent;
|
||||||
|
|
||||||
|
/// Apply one filesystem event to the index. Missing files are treated as
|
||||||
|
/// no-ops (a Create followed by a quick delete resolves via the Remove
|
||||||
|
/// event); unchanged mtimes short-circuit without touching the DB.
|
||||||
|
pub fn apply_fs_event(
|
||||||
|
conn: &mut Connection,
|
||||||
|
event: &FsEvent,
|
||||||
|
config: &Config,
|
||||||
|
ignore: &IgnoreSet,
|
||||||
|
registry: &Registry,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
match event {
|
||||||
|
FsEvent::Create(p) | FsEvent::Modify(p) => upsert_path(conn, p, config, ignore, registry),
|
||||||
|
FsEvent::Remove(p) => remove_path(conn, p),
|
||||||
|
FsEvent::Rename { from, to } => {
|
||||||
|
remove_path(conn, from)?;
|
||||||
|
upsert_path(conn, to, config, ignore, registry)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn upsert_path(
|
||||||
|
conn: &mut Connection,
|
||||||
|
path: &Path,
|
||||||
|
config: &Config,
|
||||||
|
ignore: &IgnoreSet,
|
||||||
|
registry: &Registry,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
if ignore.matches_path(path) {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
// Measured from the innermost configured root: the walk never filters the
|
||||||
|
// root it was handed, so a root that is itself hidden (`~/.config/app`, or
|
||||||
|
// anything under `%LOCALAPPDATA%` on Windows) must not be rejected here —
|
||||||
|
// that disagreement is what makes the index churn every cycle.
|
||||||
|
if !config.indexing.include_hidden
|
||||||
|
&& path_has_hidden_component_under(path, &config.resolved_indexing_paths())
|
||||||
|
{
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let Ok(meta) = std::fs::metadata(path) else {
|
||||||
|
// Already gone again — the pending Remove event handles it.
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
if meta.is_dir() {
|
||||||
|
// A moved-in tree surfaces as one directory event; walk it with
|
||||||
|
// the same filters as a full run.
|
||||||
|
let Some(root) = path.to_str() else {
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
let entries: Vec<_> = filtered_walk(
|
||||||
|
root,
|
||||||
|
config.indexing.follow_symlinks,
|
||||||
|
config.indexing.include_hidden,
|
||||||
|
ignore,
|
||||||
|
&UnreadableDirs::default(),
|
||||||
|
)
|
||||||
|
.collect();
|
||||||
|
for entry in entries {
|
||||||
|
upsert_file(conn, entry.path(), config, registry)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
} else {
|
||||||
|
upsert_file(conn, path, config, registry)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn upsert_file(
|
||||||
|
conn: &mut Connection,
|
||||||
|
path: &Path,
|
||||||
|
config: &Config,
|
||||||
|
registry: &Registry,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
let Some(rec) = prepare_file_record_from_path(path, config, registry) else {
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
|
||||||
|
let tx = conn
|
||||||
|
.transaction()
|
||||||
|
.map_err(|e| format!("begin incremental tx: {}", e))?;
|
||||||
|
|
||||||
|
let existing: Option<(i64, i64)> = tx
|
||||||
|
.query_row(
|
||||||
|
"SELECT id, mtime FROM files WHERE path = ?1",
|
||||||
|
rusqlite::params![rec.path],
|
||||||
|
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||||
|
)
|
||||||
|
.optional()
|
||||||
|
.map_err(|e| format!("lookup {}: {}", rec.path, e))?;
|
||||||
|
|
||||||
|
let file_id = match existing {
|
||||||
|
Some((_, mtime)) if mtime.max(0) as u64 == rec.mtime => return Ok(()),
|
||||||
|
Some((id, _)) => {
|
||||||
|
repo::update_file_basic(
|
||||||
|
&tx,
|
||||||
|
&rec.path,
|
||||||
|
rec.size,
|
||||||
|
rec.mtime,
|
||||||
|
Some(&rec.hash),
|
||||||
|
rec.mime.as_deref(),
|
||||||
|
rec.ftype,
|
||||||
|
)?;
|
||||||
|
id
|
||||||
|
}
|
||||||
|
None => match repo::insert_file(&tx, &rec.as_new_file())? {
|
||||||
|
Some(id) => id,
|
||||||
|
// Lost a race with another writer on the same path; the row
|
||||||
|
// that won is current enough.
|
||||||
|
None => return Ok(()),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
if rec.size > config.processing.maximum_text_file_size
|
||||||
|
|| !content_allowed(Path::new(&rec.path), config)
|
||||||
|
{
|
||||||
|
repo::set_content_na(&tx, file_id)?;
|
||||||
|
} else if let Some(text) = rec.inline_text.as_deref() {
|
||||||
|
// Small enough that `prepare_file_record_from_path` already read the
|
||||||
|
// whole file; reopening it here would be the same bytes twice.
|
||||||
|
store_inline_text(&tx, file_id, &rec, text, config)?;
|
||||||
|
} else {
|
||||||
|
extract_and_store(
|
||||||
|
&tx,
|
||||||
|
file_id,
|
||||||
|
&rec.name,
|
||||||
|
&rec.path,
|
||||||
|
rec.mime.as_deref(),
|
||||||
|
registry,
|
||||||
|
config,
|
||||||
|
)?;
|
||||||
|
}
|
||||||
|
|
||||||
|
tx.commit().map_err(|e| format!("commit incremental tx: {}", e))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn remove_path(conn: &mut Connection, path: &Path) -> Result<(), String> {
|
||||||
|
// The insert side stores a canonicalized path, so the raw event spelling
|
||||||
|
// is not a usable key — but the file is already gone, so `canonicalize`
|
||||||
|
// cannot be called on it directly either.
|
||||||
|
let path_str = db_key_for_missing_path(path);
|
||||||
|
let tx = conn
|
||||||
|
.transaction()
|
||||||
|
.map_err(|e| format!("begin incremental tx: {}", e))?;
|
||||||
|
|
||||||
|
repo::delete_file_by_path(&tx, &path_str)?;
|
||||||
|
|
||||||
|
// Directory removals surface as one event for the directory itself —
|
||||||
|
// sweep everything indexed beneath it.
|
||||||
|
let subtree: Vec<String> = {
|
||||||
|
let mut stmt = tx
|
||||||
|
.prepare("SELECT path FROM files WHERE path LIKE ?1 ESCAPE '\\'")
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
let rows = stmt
|
||||||
|
.query_map(rusqlite::params![like_subtree_pattern(&path_str)], |r| {
|
||||||
|
r.get::<_, String>(0)
|
||||||
|
})
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
rows.collect::<Result<Vec<_>, _>>().map_err(|e| e.to_string())?
|
||||||
|
};
|
||||||
|
for p in &subtree {
|
||||||
|
repo::delete_file_by_path(&tx, p)?;
|
||||||
|
}
|
||||||
|
|
||||||
|
tx.commit().map_err(|e| format!("commit incremental tx: {}", e))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::db::open_or_recreate;
|
||||||
|
|
||||||
|
struct Fixture {
|
||||||
|
conn: Connection,
|
||||||
|
dir: std::path::PathBuf,
|
||||||
|
db: std::path::PathBuf,
|
||||||
|
config: Config,
|
||||||
|
ignore: IgnoreSet,
|
||||||
|
registry: Registry,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Fixture {
|
||||||
|
fn new() -> Fixture {
|
||||||
|
let stamp = format!(
|
||||||
|
"{}-{}",
|
||||||
|
std::process::id(),
|
||||||
|
std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.unwrap()
|
||||||
|
.as_nanos()
|
||||||
|
);
|
||||||
|
let dir = std::env::temp_dir().join(format!("qs-incr-{}", stamp));
|
||||||
|
std::fs::create_dir_all(&dir).unwrap();
|
||||||
|
let db = std::env::temp_dir().join(format!("qs-incr-{}.sqlite", stamp));
|
||||||
|
let conn = open_or_recreate(db.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
let config = Config::default();
|
||||||
|
let ignore = IgnoreSet::compile(&config.indexing.ignore_patterns).unwrap();
|
||||||
|
Fixture {
|
||||||
|
conn,
|
||||||
|
dir,
|
||||||
|
db,
|
||||||
|
config,
|
||||||
|
ignore,
|
||||||
|
registry: Registry::default_set(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn apply(&mut self, event: &FsEvent) {
|
||||||
|
apply_fs_event(&mut self.conn, event, &self.config, &self.ignore, &self.registry)
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write(&self, name: &str, content: &str) -> std::path::PathBuf {
|
||||||
|
let p = self.dir.join(name);
|
||||||
|
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
|
||||||
|
std::fs::write(&p, content).unwrap();
|
||||||
|
p
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The key the index actually stores. Must go through
|
||||||
|
/// `path_to_db_string`, or every lookup here misses the
|
||||||
|
/// `\\?\`-stripped spelling on Windows.
|
||||||
|
fn canonical(&self, p: &Path) -> String {
|
||||||
|
crate::file_handling::path_to_db_string(&p.canonicalize().unwrap())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn row(&self, path: &str) -> Option<(i64, i64, i64)> {
|
||||||
|
self.conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT id, mtime, content_state FROM files WHERE path = ?1",
|
||||||
|
rusqlite::params![path],
|
||||||
|
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
|
||||||
|
)
|
||||||
|
.optional()
|
||||||
|
.unwrap()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn counts(&self) -> (i64, i64, i64) {
|
||||||
|
let files = self
|
||||||
|
.conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
let fts = self
|
||||||
|
.conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM searchabletext", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
let texts = self
|
||||||
|
.conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM documents_text", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
(files, fts, texts)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn fts_hits(&self, term: &str) -> i64 {
|
||||||
|
self.conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM searchabletext WHERE searchabletext MATCH ?1",
|
||||||
|
rusqlite::params![format!("\"{}\"", term)],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for Fixture {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
std::fs::remove_dir_all(&self.dir).ok();
|
||||||
|
std::fs::remove_file(&self.db).ok();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn create_indexes_file_and_content() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
let p = f.write("hello.txt", "greetings earthling");
|
||||||
|
f.apply(&FsEvent::Create(p.clone()));
|
||||||
|
|
||||||
|
let canonical = f.canonical(&p);
|
||||||
|
let (_, _, content_state) = f.row(&canonical).expect("row exists");
|
||||||
|
assert_eq!(content_state, repo::STATE_DONE);
|
||||||
|
assert_eq!(f.counts(), (1, 1, 1), "files + FTS + text all written");
|
||||||
|
assert_eq!(f.fts_hits("earthling"), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn modify_with_same_mtime_is_noop_and_changed_mtime_reextracts() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
let p = f.write("doc.txt", "first version");
|
||||||
|
f.apply(&FsEvent::Create(p.clone()));
|
||||||
|
let canonical = f.canonical(&p);
|
||||||
|
let (id1, mtime1, _) = f.row(&canonical).unwrap();
|
||||||
|
|
||||||
|
// Same mtime → no-op (id unchanged, no re-extraction).
|
||||||
|
f.apply(&FsEvent::Modify(p.clone()));
|
||||||
|
let (id2, mtime2, _) = f.row(&canonical).unwrap();
|
||||||
|
assert_eq!((id1, mtime1), (id2, mtime2));
|
||||||
|
|
||||||
|
// Bump mtime and content → re-extracted, FTS follows.
|
||||||
|
std::fs::write(&p, "second edition entirely").unwrap();
|
||||||
|
let newer = std::time::SystemTime::now() + std::time::Duration::from_secs(5);
|
||||||
|
let file = std::fs::File::options().write(true).open(&p).unwrap();
|
||||||
|
file.set_modified(newer).unwrap();
|
||||||
|
drop(file);
|
||||||
|
f.apply(&FsEvent::Modify(p.clone()));
|
||||||
|
assert_eq!(f.fts_hits("edition"), 1);
|
||||||
|
assert_eq!(f.fts_hits("version"), 0, "stale tokens removed");
|
||||||
|
assert_eq!(f.counts(), (1, 1, 1), "still exactly one of everything");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn remove_file_cleans_all_tables() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
let p = f.write("bye.txt", "ephemeral text");
|
||||||
|
f.apply(&FsEvent::Create(p.clone()));
|
||||||
|
let canonical = f.canonical(&p);
|
||||||
|
std::fs::remove_file(&p).unwrap();
|
||||||
|
f.apply(&FsEvent::Remove(canonical.clone().into()));
|
||||||
|
assert!(f.row(&canonical).is_none());
|
||||||
|
assert_eq!(f.counts(), (0, 0, 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn directory_create_and_remove_walks_subtree() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
f.write("tree/a.txt", "alpha content");
|
||||||
|
f.write("tree/nested/b.txt", "beta content");
|
||||||
|
f.write("tree/.hidden.txt", "should not index");
|
||||||
|
f.write("tree/junk.tmp", "ignored pattern");
|
||||||
|
let tree = f.dir.join("tree");
|
||||||
|
f.apply(&FsEvent::Create(tree.clone()));
|
||||||
|
assert_eq!(f.counts().0, 2, "hidden + ignored excluded");
|
||||||
|
|
||||||
|
let canonical_tree = f.canonical(&tree);
|
||||||
|
std::fs::remove_dir_all(&tree).unwrap();
|
||||||
|
f.apply(&FsEvent::Remove(canonical_tree.into()));
|
||||||
|
assert_eq!(f.counts(), (0, 0, 0), "subtree swept");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn rename_moves_the_row() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
let from = f.write("old-name.txt", "movable feast");
|
||||||
|
f.apply(&FsEvent::Create(from.clone()));
|
||||||
|
let canonical_from = f.canonical(&from);
|
||||||
|
|
||||||
|
let to = f.dir.join("new-name.txt");
|
||||||
|
std::fs::rename(&from, &to).unwrap();
|
||||||
|
f.apply(&FsEvent::Rename {
|
||||||
|
from: canonical_from.clone().into(),
|
||||||
|
to: to.clone(),
|
||||||
|
});
|
||||||
|
|
||||||
|
assert!(f.row(&canonical_from).is_none());
|
||||||
|
let canonical_to = f.canonical(&to);
|
||||||
|
assert!(f.row(&canonical_to).is_some());
|
||||||
|
assert_eq!(f.counts(), (1, 1, 1));
|
||||||
|
assert_eq!(f.fts_hits("feast"), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ignored_and_hidden_events_are_noops() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
let ignored = f.write("junk.tmp", "x");
|
||||||
|
let hidden = f.write(".secret", "x");
|
||||||
|
f.apply(&FsEvent::Create(ignored));
|
||||||
|
f.apply(&FsEvent::Create(hidden));
|
||||||
|
// Missing file too.
|
||||||
|
f.apply(&FsEvent::Create(f.dir.join("never-existed.txt")));
|
||||||
|
assert_eq!(f.counts(), (0, 0, 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn content_extension_filter_gates_extraction() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
f.config.indexing.content_extensions = vec!["md".into()];
|
||||||
|
let txt = f.write("listed-only.txt", "text body here");
|
||||||
|
f.apply(&FsEvent::Create(txt.clone()));
|
||||||
|
|
||||||
|
let canonical = f.canonical(&txt);
|
||||||
|
let (_, _, content_state) = f.row(&canonical).expect("row listed");
|
||||||
|
assert_eq!(
|
||||||
|
content_state,
|
||||||
|
repo::STATE_NA,
|
||||||
|
"filename indexed, content skipped"
|
||||||
|
);
|
||||||
|
assert_eq!(f.counts(), (1, 0, 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A Remove event whose path is spelled differently from the stored key
|
||||||
|
/// must still delete the row. `dir/./f.txt` and `dir/f.txt` are the same
|
||||||
|
/// file; only the canonicalized spelling is in the index.
|
||||||
|
#[test]
|
||||||
|
fn remove_with_a_non_canonical_spelling_still_deletes() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
let p = f.write("sub/gone.txt", "vanishing text");
|
||||||
|
f.apply(&FsEvent::Create(p.clone()));
|
||||||
|
assert_eq!(f.counts(), (1, 1, 1));
|
||||||
|
|
||||||
|
std::fs::remove_file(&p).unwrap();
|
||||||
|
// Same file, spelled with a redundant `.` component.
|
||||||
|
let odd = f.dir.join("sub").join(".").join("gone.txt");
|
||||||
|
f.apply(&FsEvent::Remove(odd));
|
||||||
|
assert_eq!(f.counts(), (0, 0, 0), "row removed despite the spelling");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The subtree sweep must not take siblings whose names merely share a
|
||||||
|
/// string prefix — `tree2` is not inside `tree`.
|
||||||
|
#[test]
|
||||||
|
fn subtree_sweep_spares_prefix_siblings() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
f.write("tree/a.txt", "alpha content");
|
||||||
|
f.write("tree2/b.txt", "beta content");
|
||||||
|
let tree = f.dir.join("tree");
|
||||||
|
f.apply(&FsEvent::Create(tree.clone()));
|
||||||
|
f.apply(&FsEvent::Create(f.dir.join("tree2")));
|
||||||
|
assert_eq!(f.counts().0, 2);
|
||||||
|
|
||||||
|
let canonical_tree = f.canonical(&tree);
|
||||||
|
std::fs::remove_dir_all(&tree).unwrap();
|
||||||
|
f.apply(&FsEvent::Remove(canonical_tree.into()));
|
||||||
|
|
||||||
|
assert_eq!(f.counts().0, 1, "only tree/ was swept");
|
||||||
|
let survivor = f.canonical(&f.dir.join("tree2").join("b.txt"));
|
||||||
|
assert!(f.row(&survivor).is_some(), "tree2 untouched");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A directory whose name contains a LIKE metacharacter must be swept
|
||||||
|
/// literally, not as a wildcard.
|
||||||
|
#[test]
|
||||||
|
fn subtree_sweep_treats_like_metacharacters_literally() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
f.write("a_b/inside.txt", "underscore dir");
|
||||||
|
f.write("axb/other.txt", "wildcard bait");
|
||||||
|
f.apply(&FsEvent::Create(f.dir.join("a_b")));
|
||||||
|
f.apply(&FsEvent::Create(f.dir.join("axb")));
|
||||||
|
assert_eq!(f.counts().0, 2);
|
||||||
|
|
||||||
|
let target = f.dir.join("a_b");
|
||||||
|
let canonical = f.canonical(&target);
|
||||||
|
std::fs::remove_dir_all(&target).unwrap();
|
||||||
|
f.apply(&FsEvent::Remove(canonical.into()));
|
||||||
|
|
||||||
|
assert_eq!(f.counts().0, 1, "`_` must not match `x`");
|
||||||
|
let survivor = f.canonical(&f.dir.join("axb").join("other.txt"));
|
||||||
|
assert!(f.row(&survivor).is_some());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn oversize_files_get_content_na() {
|
||||||
|
let mut f = Fixture::new();
|
||||||
|
f.config.processing.maximum_text_file_size = 4;
|
||||||
|
let p = f.write("big.txt", "way more than four bytes");
|
||||||
|
f.apply(&FsEvent::Create(p.clone()));
|
||||||
|
let canonical = f.canonical(&p);
|
||||||
|
let (_, _, content_state) = f.row(&canonical).unwrap();
|
||||||
|
assert_eq!(content_state, repo::STATE_NA);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,13 +1,18 @@
|
||||||
pub mod cli;
|
pub mod cli;
|
||||||
pub mod config;
|
pub mod config;
|
||||||
|
pub mod coordinator;
|
||||||
pub mod db;
|
pub mod db;
|
||||||
pub mod document_extraction;
|
pub mod document_extraction;
|
||||||
pub mod extract;
|
pub mod extract;
|
||||||
pub mod file_handling;
|
pub mod file_handling;
|
||||||
|
pub mod incremental;
|
||||||
pub mod indexing;
|
pub mod indexing;
|
||||||
|
pub mod log;
|
||||||
pub mod mime;
|
pub mod mime;
|
||||||
|
pub mod platform;
|
||||||
pub mod query;
|
pub mod query;
|
||||||
pub mod search_sql;
|
pub mod search;
|
||||||
pub mod shutdown;
|
pub mod shutdown;
|
||||||
pub mod snippet;
|
pub mod snippet;
|
||||||
|
pub mod walk;
|
||||||
pub mod watcher;
|
pub mod watcher;
|
||||||
|
|
|
||||||
241
crates/quicksearch-core/src/log.rs
Normal file
|
|
@ -0,0 +1,241 @@
|
||||||
|
//! The process log: every line that would go to the terminal, kept in
|
||||||
|
//! memory so a windowed run can show it.
|
||||||
|
//!
|
||||||
|
//! Launched from a desktop launcher — or on Windows, where the GUI binary is
|
||||||
|
//! built for the window subsystem and has no console at all — the process has
|
||||||
|
//! nowhere to print. The warnings the walker, indexer and watcher emit are
|
||||||
|
//! exactly the ones a user needs when something looks wrong, and they were
|
||||||
|
//! going nowhere.
|
||||||
|
//!
|
||||||
|
//! So background reporting goes through [`log_info!`] and [`log_warn!`]
|
||||||
|
//! instead of `println!`/`eprintln!`: each writes the same line to stderr
|
||||||
|
//! *and* appends it to a bounded ring the GUI's Logs tab reads. A terminal
|
||||||
|
//! run looks exactly as it did; a windowed run gains the tab.
|
||||||
|
//!
|
||||||
|
//! Command output — search hits from `quicksearch-cli`, usage text, the
|
||||||
|
//! errors a command exits with — is not logged. That is a program's answer to
|
||||||
|
//! what it was asked, not a background event, and it belongs on stdout.
|
||||||
|
|
||||||
|
use std::collections::VecDeque;
|
||||||
|
use std::io::Write;
|
||||||
|
use std::sync::{LazyLock, Mutex, MutexGuard};
|
||||||
|
use std::time::{SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
|
/// Lines retained before the oldest are dropped.
|
||||||
|
///
|
||||||
|
/// A run over a tree full of unreadable files can log per file, so this is
|
||||||
|
/// bounded rather than complete: the newest few thousand lines are what
|
||||||
|
/// diagnosing anything actually needs, and the ring holds the count it threw
|
||||||
|
/// away so the tab can say so instead of quietly lying.
|
||||||
|
pub const CAPACITY: usize = 5_000;
|
||||||
|
|
||||||
|
/// How loud a line is. The GUI colors by this; stderr gets the `Warning:`
|
||||||
|
/// prefix that the same messages carried when they were `eprintln!`s.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum Level {
|
||||||
|
Info,
|
||||||
|
Warn,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One recorded line.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct LogLine {
|
||||||
|
/// Unix seconds when it was recorded.
|
||||||
|
pub at: u64,
|
||||||
|
pub level: Level,
|
||||||
|
/// Exactly the text written to stderr, prefix included.
|
||||||
|
pub text: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record an informational line: `println!`-style formatting.
|
||||||
|
#[macro_export]
|
||||||
|
macro_rules! log_info {
|
||||||
|
($($arg:tt)*) => {
|
||||||
|
$crate::log::record($crate::log::Level::Info, ::std::format!($($arg)*))
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record a warning. The stored and printed text gains a `Warning: ` prefix,
|
||||||
|
/// so call sites pass the message alone.
|
||||||
|
#[macro_export]
|
||||||
|
macro_rules! log_warn {
|
||||||
|
($($arg:tt)*) => {
|
||||||
|
$crate::log::record($crate::log::Level::Warn, ::std::format!($($arg)*))
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Write `message` to stderr and to the ring.
|
||||||
|
///
|
||||||
|
/// Prefer the [`log_info!`] / [`log_warn!`] macros; this is what they call.
|
||||||
|
///
|
||||||
|
/// A failed stderr write is ignored rather than propagated: `eprintln!`
|
||||||
|
/// *panics* when the handle is unwritable, which on a process launched
|
||||||
|
/// without stdio would take down whichever background thread happened to
|
||||||
|
/// report something. Losing the terminal copy is acceptable — that is
|
||||||
|
/// precisely the case where the in-memory copy is the one that matters.
|
||||||
|
pub fn record(level: Level, message: String) {
|
||||||
|
let text = match level {
|
||||||
|
Level::Warn => format!("Warning: {}", message),
|
||||||
|
Level::Info => message,
|
||||||
|
};
|
||||||
|
let _ = writeln!(std::io::stderr(), "{}", text);
|
||||||
|
lock().push(level, text);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every retained line, oldest first.
|
||||||
|
pub fn snapshot() -> Vec<LogLine> {
|
||||||
|
lock().lines.iter().cloned().collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How many lines have been recorded since the process started, including
|
||||||
|
/// ones since dropped. Only ever grows, so a poll of this is the cheap way
|
||||||
|
/// to ask "anything new?" without copying the ring.
|
||||||
|
pub fn recorded() -> u64 {
|
||||||
|
lock().recorded
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How many lines the ring has evicted since the last [`clear`].
|
||||||
|
pub fn dropped() -> u64 {
|
||||||
|
lock().dropped
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Forget every retained line. [`recorded`] keeps counting.
|
||||||
|
pub fn clear() {
|
||||||
|
lock().clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
static LOG: LazyLock<Mutex<Ring>> = LazyLock::new(|| Mutex::new(Ring::new(CAPACITY)));
|
||||||
|
|
||||||
|
/// Logging must not turn one panic into a cascade of them: a thread that
|
||||||
|
/// died mid-push would otherwise poison the lock and take down every later
|
||||||
|
/// logger. The ring is a `VecDeque` of owned strings, so the worst a poisoned
|
||||||
|
/// guard can hold is a line that was half-added.
|
||||||
|
fn lock() -> MutexGuard<'static, Ring> {
|
||||||
|
LOG.lock().unwrap_or_else(|e| e.into_inner())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The bounded line buffer. Split from the global so it can be tested on its
|
||||||
|
/// own instance — every other test in the process shares the global one.
|
||||||
|
struct Ring {
|
||||||
|
lines: VecDeque<LogLine>,
|
||||||
|
capacity: usize,
|
||||||
|
recorded: u64,
|
||||||
|
dropped: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Ring {
|
||||||
|
fn new(capacity: usize) -> Ring {
|
||||||
|
Ring {
|
||||||
|
lines: VecDeque::new(),
|
||||||
|
capacity: capacity.max(1),
|
||||||
|
recorded: 0,
|
||||||
|
dropped: 0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn push(&mut self, level: Level, text: String) {
|
||||||
|
while self.lines.len() >= self.capacity {
|
||||||
|
self.lines.pop_front();
|
||||||
|
self.dropped += 1;
|
||||||
|
}
|
||||||
|
self.lines.push_back(LogLine {
|
||||||
|
at: now_unix(),
|
||||||
|
level,
|
||||||
|
text,
|
||||||
|
});
|
||||||
|
self.recorded += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn clear(&mut self) {
|
||||||
|
self.lines.clear();
|
||||||
|
self.dropped = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn now_unix() -> u64 {
|
||||||
|
SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.map(|d| d.as_secs())
|
||||||
|
.unwrap_or(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn texts(ring: &Ring) -> Vec<&str> {
|
||||||
|
ring.lines.iter().map(|l| l.text.as_str()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn lines_come_back_oldest_first() {
|
||||||
|
let mut ring = Ring::new(8);
|
||||||
|
ring.push(Level::Info, "one".into());
|
||||||
|
ring.push(Level::Warn, "two".into());
|
||||||
|
assert_eq!(texts(&ring), vec!["one", "two"]);
|
||||||
|
assert_eq!(ring.lines[1].level, Level::Warn);
|
||||||
|
assert_eq!(ring.recorded, 2);
|
||||||
|
assert_eq!(ring.dropped, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_oldest_lines_are_dropped_at_capacity() {
|
||||||
|
let mut ring = Ring::new(3);
|
||||||
|
for i in 0..5 {
|
||||||
|
ring.push(Level::Info, format!("line {}", i));
|
||||||
|
}
|
||||||
|
assert_eq!(
|
||||||
|
texts(&ring),
|
||||||
|
vec!["line 2", "line 3", "line 4"],
|
||||||
|
"only the newest `capacity` lines survive"
|
||||||
|
);
|
||||||
|
assert_eq!(ring.dropped, 2, "and the count of the lost ones is kept");
|
||||||
|
assert_eq!(ring.recorded, 5, "recorded counts everything ever pushed");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A zero capacity would spin the eviction loop forever on the first
|
||||||
|
/// push; hand-configuring one is not possible today, but the ring should
|
||||||
|
/// not depend on that staying true.
|
||||||
|
#[test]
|
||||||
|
fn a_zero_capacity_still_holds_one_line() {
|
||||||
|
let mut ring = Ring::new(0);
|
||||||
|
ring.push(Level::Info, "kept".into());
|
||||||
|
assert_eq!(texts(&ring), vec!["kept"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn clearing_empties_the_ring_but_not_the_total() {
|
||||||
|
let mut ring = Ring::new(2);
|
||||||
|
for i in 0..4 {
|
||||||
|
ring.push(Level::Info, format!("line {}", i));
|
||||||
|
}
|
||||||
|
ring.clear();
|
||||||
|
assert!(ring.lines.is_empty());
|
||||||
|
assert_eq!(ring.dropped, 0, "dropped counts against what is shown");
|
||||||
|
assert_eq!(ring.recorded, 4, "the running total survives a clear");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Through the global: the macros must land in the snapshot, and a
|
||||||
|
/// warning must carry the prefix its terminal line has. Written to
|
||||||
|
/// tolerate lines from tests running in parallel in this process.
|
||||||
|
#[test]
|
||||||
|
fn recorded_lines_reach_the_snapshot() {
|
||||||
|
let before = recorded();
|
||||||
|
crate::log_info!("test-marker info {}", 1);
|
||||||
|
crate::log_warn!("test-marker warn {}", 2);
|
||||||
|
assert!(recorded() >= before + 2);
|
||||||
|
|
||||||
|
let lines = snapshot();
|
||||||
|
let info = lines.iter().find(|l| l.text == "test-marker info 1");
|
||||||
|
let warn = lines
|
||||||
|
.iter()
|
||||||
|
.find(|l| l.text == "Warning: test-marker warn 2");
|
||||||
|
assert_eq!(info.map(|l| l.level), Some(Level::Info));
|
||||||
|
assert_eq!(
|
||||||
|
warn.map(|l| l.level),
|
||||||
|
Some(Level::Warn),
|
||||||
|
"a warning is stored with the prefix it printed with"
|
||||||
|
);
|
||||||
|
assert!(info.unwrap().at > 0, "timestamped when recorded");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,11 +1,16 @@
|
||||||
//! MIME type guessing and `FileType` bitmask classification.
|
//! MIME type guessing and `FileType` bitmask classification.
|
||||||
//!
|
//!
|
||||||
//! Two stages:
|
//! Two stages:
|
||||||
//! 1. [`guess_mime`] infers a MIME type from a path — extension first via
|
//! 1. [`guess_mime_from_head`] infers a MIME type — extension first via
|
||||||
//! `mime_guess`, falling back to magic-byte sniffing via `infer` for files
|
//! `mime_guess`, falling back to magic-byte sniffing via `infer` for files
|
||||||
//! whose extension is missing or ambiguous.
|
//! whose extension is missing or ambiguous.
|
||||||
//! 2. [`mime_to_type`] maps a MIME string to a [`FileType`] bitmask so a single
|
//! 2. [`mime_to_type`] maps a MIME string to a [`FileType`] bitmask so a single
|
||||||
//! file can belong to multiple categories (e.g. a `.docx` is Document|Text).
|
//! file can belong to multiple categories (e.g. a `.docx` is Document|Text).
|
||||||
|
//!
|
||||||
|
//! The magic bytes are always ones the caller already holds. Indexing reads
|
||||||
|
//! the head of every new or changed file to hash it, and those are the same
|
||||||
|
//! bytes `infer` wants, so there is no path-based variant that goes back to
|
||||||
|
//! disk for them — that was a second open/read/close per undetectable file.
|
||||||
|
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
||||||
|
|
@ -65,26 +70,72 @@ impl std::ops::BitOrAssign for FileType {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Guess a MIME type for a path on disk.
|
/// Extensions `mime_guess` gets wrong or does not know, and what they really
|
||||||
|
/// are.
|
||||||
///
|
///
|
||||||
/// Tries extension-based lookup via `mime_guess` first (cheap, no I/O). If
|
/// Consulted *before* `mime_guess`, because for these the table is not a
|
||||||
/// that returns nothing or a generic `application/octet-stream`, and the file
|
/// fallback but a correction. Everything here is plain text that would
|
||||||
/// is readable, falls back to `infer` magic-byte detection (reads a small
|
/// otherwise get no content indexing at all:
|
||||||
/// prefix of the file).
|
|
||||||
///
|
///
|
||||||
/// Returns `None` if no guess can be made.
|
/// - `.ps1`/`.psm1`/`.psd1` and `.url` are simply absent from `mime_guess`,
|
||||||
pub fn guess_mime(path: &Path) -> Option<String> {
|
/// and `infer` only knows binary magic, so they end up with no MIME — and
|
||||||
|
/// [`crate::extract::Registry`] has no extractor to offer, so the file is
|
||||||
|
/// marked "not applicable". PowerShell is the most common script type on a
|
||||||
|
/// Windows machine.
|
||||||
|
/// - `.bat` maps to `application/x-msdownload`, i.e. an executable. It is a
|
||||||
|
/// text file, and the plaintext extractor rightly refuses the executable
|
||||||
|
/// type. (`.cmd` already resolves to `text/plain`; it is listed so the pair
|
||||||
|
/// cannot drift.)
|
||||||
|
///
|
||||||
|
/// Platform-neutral on purpose: a `.ps1` copied to a Linux box should classify
|
||||||
|
/// the same way.
|
||||||
|
const EXTENSION_OVERRIDES: &[(&str, &str)] = &[
|
||||||
|
("bat", "text/plain"),
|
||||||
|
("cmd", "text/plain"),
|
||||||
|
("inf", "text/plain"),
|
||||||
|
("ps1", "text/plain"),
|
||||||
|
("psd1", "text/plain"),
|
||||||
|
("psm1", "text/plain"),
|
||||||
|
("url", "text/plain"),
|
||||||
|
];
|
||||||
|
|
||||||
|
/// Look up [`EXTENSION_OVERRIDES`] for `path`. Extension comparison is
|
||||||
|
/// ASCII-case-insensitive, which matters more on Windows where `REPORT.BAT` is
|
||||||
|
/// as common as the lowercase spelling.
|
||||||
|
fn extension_override(path: &Path) -> Option<&'static str> {
|
||||||
|
let ext = path.extension()?.to_str()?.to_ascii_lowercase();
|
||||||
|
EXTENSION_OVERRIDES
|
||||||
|
.iter()
|
||||||
|
.find(|(e, _)| *e == ext)
|
||||||
|
.map(|(_, mime)| *mime)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Infer a MIME type from a path plus the file's leading bytes.
|
||||||
|
///
|
||||||
|
/// Extension first — an override table, then `mime_guess` — and magic bytes
|
||||||
|
/// only when those come up empty or say `application/octet-stream`.
|
||||||
|
///
|
||||||
|
/// `head` is whatever the caller already read; indexing passes the same buffer
|
||||||
|
/// it hashes. It bounds magic-byte detection, so a caller that supplies fewer
|
||||||
|
/// than 262 bytes (`infer`'s longest signature) can get `None` where a longer
|
||||||
|
/// head would have matched. The indexer's `hash_length` defaults to 8 KiB —
|
||||||
|
/// exactly what `infer` itself reads from a path — so at default config this
|
||||||
|
/// is as good as opening the file, and strictly cheaper.
|
||||||
|
///
|
||||||
|
/// A `None` result is a real answer, not a "don't know": the content pass
|
||||||
|
/// stores it and does not re-derive it (see
|
||||||
|
/// [`crate::file_handling::extract_and_store`]).
|
||||||
|
pub fn guess_mime_from_head(path: &Path, head: &[u8]) -> Option<String> {
|
||||||
|
if let Some(m) = extension_override(path) {
|
||||||
|
return Some(m.to_string());
|
||||||
|
}
|
||||||
if let Some(g) = mime_guess::from_path(path).first() {
|
if let Some(g) = mime_guess::from_path(path).first() {
|
||||||
let s = g.essence_str();
|
let s = g.essence_str();
|
||||||
if !s.is_empty() && s != "application/octet-stream" {
|
if !s.is_empty() && s != "application/octet-stream" {
|
||||||
return Some(s.to_string());
|
return Some(s.to_string());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// Magic-byte fallback. `infer::get_from_path` handles errors by returning None.
|
infer::get(head).map(|t| t.mime_type().to_string())
|
||||||
if let Ok(Some(t)) = infer::get_from_path(path) {
|
|
||||||
return Some(t.mime_type().to_string());
|
|
||||||
}
|
|
||||||
None
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Map a MIME string to a [`FileType`] bitmask. Ported from Baloo's
|
/// Map a MIME string to a [`FileType`] bitmask. Ported from Baloo's
|
||||||
|
|
@ -131,7 +182,13 @@ pub fn mime_to_type(mime: &str) -> FileType {
|
||||||
| "vnd.oasis.opendocument.presentation" => {
|
| "vnd.oasis.opendocument.presentation" => {
|
||||||
t |= FileType::DOCUMENT | FileType::PRESENTATION;
|
t |= FileType::DOCUMENT | FileType::PRESENTATION;
|
||||||
}
|
}
|
||||||
// Archives
|
// Outlook saved messages and compiled HTML help are documents; both
|
||||||
|
// are ordinary things to find in a Windows home directory.
|
||||||
|
"vnd.ms-outlook" | "vnd.ms-htmlhelp" => {
|
||||||
|
t |= FileType::DOCUMENT;
|
||||||
|
}
|
||||||
|
// Archives. The Windows installer/cabinet formats are containers in
|
||||||
|
// exactly the same sense as the rest of this list.
|
||||||
"zip"
|
"zip"
|
||||||
| "x-tar"
|
| "x-tar"
|
||||||
| "x-7z-compressed"
|
| "x-7z-compressed"
|
||||||
|
|
@ -142,7 +199,9 @@ pub fn mime_to_type(mime: &str) -> FileType {
|
||||||
| "x-bzip2"
|
| "x-bzip2"
|
||||||
| "x-xz"
|
| "x-xz"
|
||||||
| "vnd.debian.binary-package"
|
| "vnd.debian.binary-package"
|
||||||
| "x-rpm" => {
|
| "x-rpm"
|
||||||
|
| "vnd.ms-cab-compressed"
|
||||||
|
| "x-msi" => {
|
||||||
t |= FileType::ARCHIVE;
|
t |= FileType::ARCHIVE;
|
||||||
}
|
}
|
||||||
// application/xml is structured text
|
// application/xml is structured text
|
||||||
|
|
@ -228,11 +287,137 @@ mod tests {
|
||||||
assert_eq!(FileType::from_name("Weird"), FileType::EMPTY);
|
assert_eq!(FileType::from_name("Weird"), FileType::EMPTY);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Extension resolution happens before magic bytes are consulted, so an
|
||||||
|
/// empty head is enough to exercise it.
|
||||||
#[test]
|
#[test]
|
||||||
fn guess_mime_by_extension() {
|
fn guess_mime_by_extension() {
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
assert_eq!(guess_mime(&PathBuf::from("a.txt")).as_deref(), Some("text/plain"));
|
let by_ext = |n: &str| guess_mime_from_head(&PathBuf::from(n), b"").unwrap_or_default();
|
||||||
assert_eq!(guess_mime(&PathBuf::from("a.png")).as_deref(), Some("image/png"));
|
assert_eq!(by_ext("a.txt"), "text/plain");
|
||||||
assert_eq!(guess_mime(&PathBuf::from("a.mp3")).as_deref(), Some("audio/mpeg"));
|
assert_eq!(by_ext("a.png"), "image/png");
|
||||||
|
assert_eq!(by_ext("a.mp3"), "audio/mpeg");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every override must land on a type the plaintext extractor accepts —
|
||||||
|
/// the point of the table is that these files get their contents indexed.
|
||||||
|
#[test]
|
||||||
|
fn windows_script_types_reach_the_plaintext_extractor() {
|
||||||
|
use crate::extract::{plaintext::PlaintextExtractor, Extractor};
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
for name in [
|
||||||
|
"deploy.ps1",
|
||||||
|
"Module.psm1",
|
||||||
|
"Module.psd1",
|
||||||
|
"build.bat",
|
||||||
|
"build.cmd",
|
||||||
|
"driver.inf",
|
||||||
|
"bookmark.url",
|
||||||
|
] {
|
||||||
|
let mime = guess_mime_from_head(&PathBuf::from(name), b"")
|
||||||
|
.unwrap_or_else(|| panic!("{} has no MIME", name));
|
||||||
|
assert!(
|
||||||
|
PlaintextExtractor.supports(&mime),
|
||||||
|
"{} -> {} is not extractable as text",
|
||||||
|
name,
|
||||||
|
mime
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extension_overrides_are_case_insensitive() {
|
||||||
|
use std::path::PathBuf;
|
||||||
|
// Uppercase extensions are ordinary on Windows.
|
||||||
|
assert_eq!(
|
||||||
|
guess_mime_from_head(&PathBuf::from("DEPLOY.PS1"), b"").as_deref(),
|
||||||
|
Some("text/plain")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guess_mime_from_head(&PathBuf::from("Build.Bat"), b"").as_deref(),
|
||||||
|
Some("text/plain")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The override table must win over the file's actual content: a `.ps1`
|
||||||
|
/// holding something `infer` would recognise is still a script.
|
||||||
|
#[test]
|
||||||
|
fn extension_overrides_beat_magic_bytes() {
|
||||||
|
use std::path::PathBuf;
|
||||||
|
assert_eq!(
|
||||||
|
guess_mime_from_head(&PathBuf::from("a.ps1"), b"Write-Host hi").as_deref(),
|
||||||
|
Some("text/plain")
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
guess_mime_from_head(&PathBuf::from("a.ps1"), b"%PDF-1.7").as_deref(),
|
||||||
|
Some("text/plain")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sql_dumps_are_extractable() {
|
||||||
|
use crate::extract::{plaintext::PlaintextExtractor, Extractor};
|
||||||
|
use std::path::PathBuf;
|
||||||
|
let mime = guess_mime_from_head(&PathBuf::from("schema.sql"), b"").unwrap();
|
||||||
|
assert!(PlaintextExtractor.supports(&mime), "{}", mime);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The content pass trusts the MIME the walk stored, including `None`, and
|
||||||
|
/// never reopens the file to second-guess it. That is only sound if a
|
||||||
|
/// `hash_length`-sized head is enough to recognise a format from its magic
|
||||||
|
/// bytes — `infer`'s longest signature is 262 bytes and the default head is
|
||||||
|
/// 8 KiB, so it is by a wide margin. This pins that for extensionless
|
||||||
|
/// files, where magic bytes are the only signal there is.
|
||||||
|
#[test]
|
||||||
|
fn a_default_sized_head_is_enough_for_magic_byte_detection() {
|
||||||
|
use std::path::PathBuf;
|
||||||
|
let head_bytes = crate::config::ProcessingConfig::default().hash_length;
|
||||||
|
|
||||||
|
let samples: &[(&str, &[u8], &str)] = &[
|
||||||
|
("png", &[0x89, b'P', b'N', b'G', 0x0d, 0x0a, 0x1a, 0x0a], "image/png"),
|
||||||
|
("gif", b"GIF89a", "image/gif"),
|
||||||
|
("pdf", b"%PDF-1.7", "application/pdf"),
|
||||||
|
("zip", &[0x50, 0x4b, 0x03, 0x04], "application/zip"),
|
||||||
|
("gz", &[0x1f, 0x8b, 0x08], "application/gzip"),
|
||||||
|
];
|
||||||
|
|
||||||
|
for (tag, magic, expected) in samples {
|
||||||
|
// No extension at all, so nothing but the bytes can answer.
|
||||||
|
let path = PathBuf::from(format!("/tmp/qs-sniff-{}", tag));
|
||||||
|
let mut body = magic.to_vec();
|
||||||
|
body.resize(head_bytes, 0);
|
||||||
|
assert_eq!(
|
||||||
|
guess_mime_from_head(&path, &body).as_deref(),
|
||||||
|
Some(*expected),
|
||||||
|
"{} must be detectable from a default-sized head",
|
||||||
|
tag
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The other side of that bound: starve the head below `infer`'s longest
|
||||||
|
/// signature and detection legitimately degrades. Documented behaviour of
|
||||||
|
/// a non-default `hash_length`, not a bug — but it must stay a `None`
|
||||||
|
/// rather than a wrong guess.
|
||||||
|
#[test]
|
||||||
|
fn a_head_shorter_than_the_signature_declines_rather_than_guessing() {
|
||||||
|
use std::path::PathBuf;
|
||||||
|
let path = PathBuf::from("/tmp/qs-sniff-truncated");
|
||||||
|
assert_eq!(guess_mime_from_head(&path, b"").as_deref(), None);
|
||||||
|
assert_eq!(guess_mime_from_head(&path, &[0x89]).as_deref(), None);
|
||||||
|
// Enough bytes, and it resolves.
|
||||||
|
assert_eq!(
|
||||||
|
guess_mime_from_head(&path, &[0x89, b'P', b'N', b'G', 0x0d, 0x0a, 0x1a, 0x0a])
|
||||||
|
.as_deref(),
|
||||||
|
Some("image/png")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn windows_container_and_document_types_classify() {
|
||||||
|
assert!(mime_to_type("application/vnd.ms-cab-compressed").contains(FileType::ARCHIVE));
|
||||||
|
assert!(mime_to_type("application/x-msi").contains(FileType::ARCHIVE));
|
||||||
|
assert!(mime_to_type("application/vnd.ms-outlook").contains(FileType::DOCUMENT));
|
||||||
|
assert!(mime_to_type("application/vnd.ms-htmlhelp").contains(FileType::DOCUMENT));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
421
crates/quicksearch-core/src/platform.rs
Normal file
|
|
@ -0,0 +1,421 @@
|
||||||
|
//! The one place `#[cfg]` lives.
|
||||||
|
//!
|
||||||
|
//! Everything here answers "what does this platform do differently", so no
|
||||||
|
//! other module has to ask. Two rules keep it honest:
|
||||||
|
//!
|
||||||
|
//! - Every function is defined for every target. Callers never wrap a call
|
||||||
|
//! site in `#[cfg]`; if a platform has nothing to do, its arm is the
|
||||||
|
//! trivial one.
|
||||||
|
//! - Anything that can be decided from a string rather than a syscall is
|
||||||
|
//! split out and made testable everywhere ([`is_unc_string`],
|
||||||
|
//! [`PATH_COLLATION`]), because the test suite runs on Linux.
|
||||||
|
|
||||||
|
use std::ffi::OsString;
|
||||||
|
use std::path::{Component, Path, PathBuf};
|
||||||
|
|
||||||
|
/// The user's home directory.
|
||||||
|
///
|
||||||
|
/// On Windows `%USERPROFILE%` is checked **first**. Git Bash and MSYS2 export
|
||||||
|
/// `HOME` as a POSIX path (`/c/Users/me`) that no Win32 API can open, and
|
||||||
|
/// preferring it would point the config file, the index, and the default
|
||||||
|
/// indexing root at a directory that does not exist.
|
||||||
|
pub fn home_dir() -> Option<OsString> {
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
if let Some(profile) = std::env::var_os("USERPROFILE") {
|
||||||
|
return Some(profile);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::env::var_os("HOME").or_else(|| std::env::var_os("USERPROFILE"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a directory entry counts as hidden.
|
||||||
|
///
|
||||||
|
/// Unix: a leading dot. Windows: a leading dot **or** `FILE_ATTRIBUTE_HIDDEN`
|
||||||
|
/// / `FILE_ATTRIBUTE_SYSTEM` — without which `include_hidden = false` hides
|
||||||
|
/// nothing on Windows, and `$RECYCLE.BIN`, `System Volume Information`,
|
||||||
|
/// `pagefile.sys` and `AppData` all get indexed.
|
||||||
|
///
|
||||||
|
/// `meta` is a closure because on Unix it is never called: the walkers
|
||||||
|
/// deliberately avoid `metadata()`, which would cost an extra `lstat` per
|
||||||
|
/// entry and a full round trip on a network share. On Windows the cost is
|
||||||
|
/// zero anyway — both `std::fs::DirEntry::metadata` and
|
||||||
|
/// `walkdir::DirEntry::metadata` hand back data already cached from
|
||||||
|
/// `FindNextFileW`.
|
||||||
|
pub fn entry_is_hidden<F>(name: &str, meta: F) -> bool
|
||||||
|
where
|
||||||
|
F: FnOnce() -> Option<std::fs::Metadata>,
|
||||||
|
{
|
||||||
|
if name.starts_with('.') {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
use std::os::windows::fs::MetadataExt;
|
||||||
|
use windows_sys::Win32::Storage::FileSystem::{
|
||||||
|
FILE_ATTRIBUTE_HIDDEN, FILE_ATTRIBUTE_SYSTEM,
|
||||||
|
};
|
||||||
|
if let Some(m) = meta() {
|
||||||
|
return m.file_attributes() & (FILE_ATTRIBUTE_HIDDEN | FILE_ATTRIBUTE_SYSTEM) != 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
{
|
||||||
|
let _ = meta;
|
||||||
|
}
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether `path` has a hidden component *below* the root that contains it.
|
||||||
|
///
|
||||||
|
/// Components at or above a root are exempt, because the walkers exempt their
|
||||||
|
/// root too (depth 0 is always kept — users explicitly chose their roots).
|
||||||
|
/// The two must agree: if they disagree, a full run indexes a file that the
|
||||||
|
/// watcher then refuses to update, and the index churns on every cycle.
|
||||||
|
///
|
||||||
|
/// That is a latent bug on Unix (`~/.config/app` as a root) and a certainty on
|
||||||
|
/// Windows, where `AppData` carries `FILE_ATTRIBUTE_HIDDEN` and
|
||||||
|
/// `std::env::temp_dir()` lives underneath it.
|
||||||
|
///
|
||||||
|
/// `roots` are matched by whole path components, so `/a/bc` is not treated as
|
||||||
|
/// living under `/a/b`. A path under no known root is checked in full.
|
||||||
|
pub fn path_has_hidden_component_under(path: &Path, roots: &[PathBuf]) -> bool {
|
||||||
|
// Innermost containing root wins: with both `/data` and `/data/.cache`
|
||||||
|
// configured, a file under the latter is only judged below `.cache`.
|
||||||
|
let base = roots
|
||||||
|
.iter()
|
||||||
|
.filter(|r| path.starts_with(r))
|
||||||
|
.max_by_key(|r| r.components().count());
|
||||||
|
|
||||||
|
let (mut current, tail) = match base {
|
||||||
|
Some(root) => match path.strip_prefix(root) {
|
||||||
|
Ok(tail) => (root.clone(), tail),
|
||||||
|
Err(_) => (PathBuf::new(), path),
|
||||||
|
},
|
||||||
|
None => (PathBuf::new(), path),
|
||||||
|
};
|
||||||
|
|
||||||
|
// Rebuild the absolute path as we descend: a bare tail component cannot
|
||||||
|
// be stat'd on its own, and the attribute check needs a real path.
|
||||||
|
for component in tail.components() {
|
||||||
|
current.push(component);
|
||||||
|
if let Component::Normal(name) = component {
|
||||||
|
let name = name.to_string_lossy();
|
||||||
|
if entry_is_hidden(&name, || std::fs::metadata(¤t).ok()) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether `s` names a UNC path, in either spelling.
|
||||||
|
///
|
||||||
|
/// Split out from [`is_network_path`] so the string half is testable on every
|
||||||
|
/// platform, and written with explicit parentheses — the precedence of `&&`
|
||||||
|
/// against `||` is exactly the kind of thing that silently disables the
|
||||||
|
/// network thread pool.
|
||||||
|
///
|
||||||
|
/// Only *called* on Windows; compiled everywhere so its tests run everywhere,
|
||||||
|
/// which is the point of splitting it out.
|
||||||
|
#[cfg_attr(not(windows), allow(dead_code))]
|
||||||
|
pub(crate) fn is_unc_string(s: &str) -> bool {
|
||||||
|
s.starts_with(r"\\?\UNC\") || (s.starts_with(r"\\") && !s.starts_with(r"\\?\"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Filesystem types whose operations are network round trips.
|
||||||
|
#[cfg(target_os = "linux")]
|
||||||
|
const NETWORK_FS_TYPES: [&str; 8] = [
|
||||||
|
"cifs", "smb3", "smbfs", "nfs", "nfs4", "afs", "fuse.sshfs", "9p",
|
||||||
|
];
|
||||||
|
|
||||||
|
/// Whether `path` lives on a network filesystem.
|
||||||
|
///
|
||||||
|
/// Reads `/proc/mounts` and takes the longest mount point that is a prefix of
|
||||||
|
/// `path` — the innermost mount is the one that actually serves it.
|
||||||
|
#[cfg(target_os = "linux")]
|
||||||
|
pub(crate) fn is_network_path(path: &Path) -> bool {
|
||||||
|
let Ok(mounts) = std::fs::read_to_string("/proc/mounts") else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
let target = std::fs::canonicalize(path).unwrap_or_else(|_| path.to_path_buf());
|
||||||
|
|
||||||
|
let mut best: Option<(usize, bool)> = None;
|
||||||
|
for line in mounts.lines() {
|
||||||
|
let mut fields = line.split_whitespace();
|
||||||
|
let (Some(_dev), Some(point), Some(fstype)) = (fields.next(), fields.next(), fields.next())
|
||||||
|
else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
// `/proc/mounts` octal-escapes spaces and a few other characters.
|
||||||
|
let point = point.replace("\\040", " ");
|
||||||
|
let point = Path::new(&point);
|
||||||
|
if !target.starts_with(point) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let depth = point.components().count();
|
||||||
|
let is_network = NETWORK_FS_TYPES.contains(&fstype);
|
||||||
|
if best.is_none_or(|(d, _)| depth > d) {
|
||||||
|
best = Some((depth, is_network));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
best.is_some_and(|(_, is_network)| is_network)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether `path` is served by a network redirector.
|
||||||
|
///
|
||||||
|
/// UNC needs no syscall. A *mapped drive letter* does: `Z:\` backed by an SMB
|
||||||
|
/// share is indistinguishable from a local disk by string inspection, and it
|
||||||
|
/// is the common case — asking `GetDriveTypeW` is the only way to tell. Left
|
||||||
|
/// undetected it walks with `LOCAL_THREADS` instead of `NETWORK_THREADS`,
|
||||||
|
/// which is the exact failure the threading design exists to prevent.
|
||||||
|
#[cfg(windows)]
|
||||||
|
pub(crate) fn is_network_path(path: &Path) -> bool {
|
||||||
|
use std::os::windows::ffi::OsStrExt;
|
||||||
|
use windows_sys::Win32::Storage::FileSystem::GetDriveTypeW;
|
||||||
|
use windows_sys::Win32::System::WindowsProgramming::DRIVE_REMOTE;
|
||||||
|
|
||||||
|
let s = path.to_string_lossy();
|
||||||
|
if is_unc_string(&s) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetDriveTypeW wants a root ("Z:\"), not an arbitrary path.
|
||||||
|
let Some(root) = path.components().next() else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
let Component::Prefix(prefix) = root else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
let mut wide: Vec<u16> = prefix.as_os_str().encode_wide().collect();
|
||||||
|
wide.push(b'\\' as u16);
|
||||||
|
wide.push(0);
|
||||||
|
unsafe { GetDriveTypeW(wide.as_ptr()) == DRIVE_REMOTE }
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(any(target_os = "linux", windows)))]
|
||||||
|
pub(crate) fn is_network_path(_path: &Path) -> bool {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether the filesystem-notification backend covers a whole tree from one
|
||||||
|
/// watch on its root.
|
||||||
|
///
|
||||||
|
/// `false` (inotify): one watch descriptor covers exactly one directory's
|
||||||
|
/// entries, so the caller must walk the tree and register every directory
|
||||||
|
/// itself — which is what lets it skip `.git`, `node_modules` and hidden
|
||||||
|
/// subtrees instead of spending a scarce descriptor on each.
|
||||||
|
///
|
||||||
|
/// `true` (`ReadDirectoryChangesW`): one handle covers the subtree, and
|
||||||
|
/// directories created later are included automatically. Registering
|
||||||
|
/// per-directory here would be actively harmful rather than merely wasteful —
|
||||||
|
/// notify allocates a 16 KiB buffer *inline per watch* plus a directory
|
||||||
|
/// handle, so a large tree would ask for gigabytes of buffers and tens of
|
||||||
|
/// thousands of handles. The pruning moves to the event path instead.
|
||||||
|
///
|
||||||
|
/// macOS FSEvents is also natively recursive, but it is left on the
|
||||||
|
/// per-directory path here because that path works there and is the one under
|
||||||
|
/// test.
|
||||||
|
pub const WATCH_ROOTS_RECURSIVELY: bool = cfg!(windows);
|
||||||
|
|
||||||
|
/// SQLite collation for comparing stored path strings.
|
||||||
|
///
|
||||||
|
/// Windows filesystems are case-insensitive, and SQLite's `LIKE` already folds
|
||||||
|
/// ASCII case by default. A path filter that compares one half with `=` and the
|
||||||
|
/// other with `LIKE` would otherwise disagree with itself. `NOCASE` folds ASCII
|
||||||
|
/// only, which matches what `LIKE` does — non-ASCII paths stay case-sensitive
|
||||||
|
/// on both sides, consistently.
|
||||||
|
pub const PATH_COLLATION: &str = if cfg!(windows) { "NOCASE" } else { "BINARY" };
|
||||||
|
|
||||||
|
/// How long to keep retrying a delete that fails because something else holds
|
||||||
|
/// the file open.
|
||||||
|
#[cfg(windows)]
|
||||||
|
const REMOVE_RETRY_BUDGET: std::time::Duration = std::time::Duration::from_millis(500);
|
||||||
|
|
||||||
|
/// `fs::remove_file`, retried briefly on Windows.
|
||||||
|
///
|
||||||
|
/// Unix `unlink` succeeds even with the file open, so this is a single call
|
||||||
|
/// there. Windows returns a sharing violation while *any* handle is open —
|
||||||
|
/// most often an antivirus scanner reading the file microseconds after we
|
||||||
|
/// closed it. The retry turns a spurious hard failure into a short pause.
|
||||||
|
pub fn remove_file_retrying(path: &Path) -> std::io::Result<()> {
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
{
|
||||||
|
std::fs::remove_file(path)
|
||||||
|
}
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
let deadline = std::time::Instant::now() + REMOVE_RETRY_BUDGET;
|
||||||
|
loop {
|
||||||
|
match std::fs::remove_file(path) {
|
||||||
|
Ok(()) => return Ok(()),
|
||||||
|
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Err(e),
|
||||||
|
Err(e) => {
|
||||||
|
if std::time::Instant::now() >= deadline {
|
||||||
|
return Err(e);
|
||||||
|
}
|
||||||
|
std::thread::sleep(std::time::Duration::from_millis(25));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Deny read access to `dir`, for tests that exercise the unreadable-directory
|
||||||
|
/// guards.
|
||||||
|
///
|
||||||
|
/// Exposed (hidden) rather than duplicated per test module because
|
||||||
|
/// `tests/full_index.rs` is a separate crate and needs it too. Windows uses
|
||||||
|
/// `icacls`: a deny ACE binds even the owner until the paired
|
||||||
|
/// [`restore_read`] rewrites it, and neither call needs elevation.
|
||||||
|
#[doc(hidden)]
|
||||||
|
pub fn deny_read(dir: &Path) -> std::io::Result<()> {
|
||||||
|
#[cfg(unix)]
|
||||||
|
{
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
std::fs::set_permissions(dir, std::fs::Permissions::from_mode(0o000))
|
||||||
|
}
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
icacls(dir, &["/deny", &format!("{}:(OI)(CI)(RD)", current_user()?)])
|
||||||
|
}
|
||||||
|
#[cfg(not(any(unix, windows)))]
|
||||||
|
{
|
||||||
|
let _ = dir;
|
||||||
|
Err(std::io::Error::new(
|
||||||
|
std::io::ErrorKind::Unsupported,
|
||||||
|
"deny_read is not supported on this target",
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Undo [`deny_read`] so the directory can be cleaned up.
|
||||||
|
#[doc(hidden)]
|
||||||
|
pub fn restore_read(dir: &Path) -> std::io::Result<()> {
|
||||||
|
#[cfg(unix)]
|
||||||
|
{
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
std::fs::set_permissions(dir, std::fs::Permissions::from_mode(0o755))
|
||||||
|
}
|
||||||
|
#[cfg(windows)]
|
||||||
|
{
|
||||||
|
icacls(dir, &["/remove:d", ¤t_user()?])
|
||||||
|
}
|
||||||
|
#[cfg(not(any(unix, windows)))]
|
||||||
|
{
|
||||||
|
let _ = dir;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(windows)]
|
||||||
|
fn current_user() -> std::io::Result<String> {
|
||||||
|
match (std::env::var("USERDOMAIN"), std::env::var("USERNAME")) {
|
||||||
|
(Ok(domain), Ok(user)) => Ok(format!("{}\\{}", domain, user)),
|
||||||
|
(_, Ok(user)) => Ok(user),
|
||||||
|
_ => Err(std::io::Error::other("USERNAME is not set")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(windows)]
|
||||||
|
fn icacls(dir: &Path, args: &[&str]) -> std::io::Result<()> {
|
||||||
|
let out = std::process::Command::new("icacls")
|
||||||
|
.arg(dir)
|
||||||
|
.args(args)
|
||||||
|
.output()?;
|
||||||
|
if out.status.success() {
|
||||||
|
Ok(())
|
||||||
|
} else {
|
||||||
|
Err(std::io::Error::other(format!(
|
||||||
|
"icacls {}: {}",
|
||||||
|
dir.display(),
|
||||||
|
String::from_utf8_lossy(&out.stderr).trim()
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unc_spellings() {
|
||||||
|
assert!(is_unc_string(r"\\server\share"));
|
||||||
|
assert!(is_unc_string(r"\\server\share\dir\file.txt"));
|
||||||
|
assert!(is_unc_string(r"\\?\UNC\server\share"));
|
||||||
|
// A verbatim *drive* path is local, not a share. This is the case the
|
||||||
|
// original `&&`/`||` precedence got wrong.
|
||||||
|
assert!(!is_unc_string(r"\\?\C:\Users\me"));
|
||||||
|
assert!(!is_unc_string(r"C:\Users\me"));
|
||||||
|
assert!(!is_unc_string("/home/me"));
|
||||||
|
assert!(!is_unc_string(""));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn collation_matches_like_case_folding() {
|
||||||
|
// LIKE folds ASCII case on every platform; the `=` half of a path
|
||||||
|
// filter has to agree with it, which is what this constant is for.
|
||||||
|
assert_eq!(PATH_COLLATION, if cfg!(windows) { "NOCASE" } else { "BINARY" });
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn dotfiles_are_hidden_without_consulting_metadata() {
|
||||||
|
let mut called = false;
|
||||||
|
assert!(entry_is_hidden(".git", || {
|
||||||
|
called = true;
|
||||||
|
None
|
||||||
|
}));
|
||||||
|
assert!(!called, "a dot prefix must short-circuit before any stat");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ordinary_names_are_not_hidden() {
|
||||||
|
assert!(!entry_is_hidden("Documents", || None));
|
||||||
|
assert!(!entry_is_hidden("report.txt", || None));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn hidden_components_are_measured_from_the_innermost_root() {
|
||||||
|
let root = PathBuf::from(format!("{}.config", sep_prefix()));
|
||||||
|
let roots = vec![root.clone()];
|
||||||
|
|
||||||
|
// The root itself is hidden, but it was chosen explicitly — the walk
|
||||||
|
// keeps it, so the watcher must too.
|
||||||
|
assert!(!path_has_hidden_component_under(&root, &roots));
|
||||||
|
assert!(!path_has_hidden_component_under(&root.join("app.conf"), &roots));
|
||||||
|
|
||||||
|
// A dot *below* the root still counts.
|
||||||
|
assert!(path_has_hidden_component_under(
|
||||||
|
&root.join(".secret").join("x"),
|
||||||
|
&roots
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_path_under_no_root_is_checked_in_full() {
|
||||||
|
let roots = vec![PathBuf::from(format!("{}srv", sep_prefix()))];
|
||||||
|
let stray = PathBuf::from(format!("{}home{}me{}.ssh", sep_prefix(), SEP, SEP));
|
||||||
|
assert!(path_has_hidden_component_under(&stray, &roots));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sibling_roots_do_not_capture_each_other() {
|
||||||
|
// `/a/bc` does not live under `/a/b`, so the `.x` below it is judged,
|
||||||
|
// not exempted.
|
||||||
|
let roots = vec![PathBuf::from(format!("{}a{}b", sep_prefix(), SEP))];
|
||||||
|
let other = PathBuf::from(format!("{}a{}bc{}.x", sep_prefix(), SEP, SEP));
|
||||||
|
assert!(path_has_hidden_component_under(&other, &roots));
|
||||||
|
}
|
||||||
|
|
||||||
|
const SEP: char = std::path::MAIN_SEPARATOR;
|
||||||
|
|
||||||
|
/// An absolute-path prefix for the running platform, so these tests read
|
||||||
|
/// the same on both.
|
||||||
|
fn sep_prefix() -> String {
|
||||||
|
if cfg!(windows) {
|
||||||
|
r"C:\".to_string()
|
||||||
|
} else {
|
||||||
|
"/".to_string()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -32,7 +32,32 @@ impl std::fmt::Display for LexError {
|
||||||
|
|
||||||
impl std::error::Error for LexError {}
|
impl std::error::Error for LexError {}
|
||||||
|
|
||||||
|
/// Whether the `:` at `colon` is the one in a drive letter rather than a
|
||||||
|
/// property operator.
|
||||||
|
///
|
||||||
|
/// True only when the word so far is exactly one ASCII letter *and* a path
|
||||||
|
/// separator follows, which is narrow enough to leave `12:30`, `a:b` and
|
||||||
|
/// `type:Audio` tokenizing exactly as before.
|
||||||
|
fn is_drive_letter_colon(bytes: &[u8], start: usize, colon: usize) -> bool {
|
||||||
|
colon == start + 1
|
||||||
|
&& bytes[start].is_ascii_alphabetic()
|
||||||
|
&& matches!(bytes.get(colon + 1), Some(b'\\') | Some(b'/'))
|
||||||
|
}
|
||||||
|
|
||||||
pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError> {
|
pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError> {
|
||||||
|
let (tokens, err) = tokenize_spanned(input);
|
||||||
|
match err {
|
||||||
|
Some(e) => Err(e),
|
||||||
|
None => Ok(tokens.into_iter().map(|(t, _)| t).collect()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`tokenize`], but each token carries its byte range in `input`, and a
|
||||||
|
/// trailing error (unterminated quote, invalid UTF-8) is returned alongside
|
||||||
|
/// the tokens lexed before it instead of discarding them. `Quoted` spans
|
||||||
|
/// include both quote characters. This is what the GUI's syntax highlighter
|
||||||
|
/// runs on: it must color the intact prefix of a half-typed query.
|
||||||
|
pub fn tokenize_spanned(input: &str) -> (Vec<(Token, std::ops::Range<usize>)>, Option<LexError>) {
|
||||||
let bytes = input.as_bytes();
|
let bytes = input.as_bytes();
|
||||||
let mut i = 0usize;
|
let mut i = 0usize;
|
||||||
let mut out = Vec::new();
|
let mut out = Vec::new();
|
||||||
|
|
@ -45,62 +70,72 @@ pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError> {
|
||||||
}
|
}
|
||||||
match b {
|
match b {
|
||||||
b'(' => {
|
b'(' => {
|
||||||
out.push(Token::LParen);
|
out.push((Token::LParen, i..i + 1));
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
b')' => {
|
b')' => {
|
||||||
out.push(Token::RParen);
|
out.push((Token::RParen, i..i + 1));
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
b':' => {
|
b':' => {
|
||||||
out.push(Token::Op(Op::Contains));
|
out.push((Token::Op(Op::Contains), i..i + 1));
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
b'=' => {
|
b'=' => {
|
||||||
out.push(Token::Op(Op::Eq));
|
out.push((Token::Op(Op::Eq), i..i + 1));
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
b'<' => {
|
b'<' => {
|
||||||
if bytes.get(i + 1) == Some(&b'=') {
|
if bytes.get(i + 1) == Some(&b'=') {
|
||||||
out.push(Token::Op(Op::Le));
|
out.push((Token::Op(Op::Le), i..i + 2));
|
||||||
i += 2;
|
i += 2;
|
||||||
} else {
|
} else {
|
||||||
out.push(Token::Op(Op::Lt));
|
out.push((Token::Op(Op::Lt), i..i + 1));
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
b'>' => {
|
b'>' => {
|
||||||
if bytes.get(i + 1) == Some(&b'=') {
|
if bytes.get(i + 1) == Some(&b'=') {
|
||||||
out.push(Token::Op(Op::Ge));
|
out.push((Token::Op(Op::Ge), i..i + 2));
|
||||||
i += 2;
|
i += 2;
|
||||||
} else {
|
} else {
|
||||||
out.push(Token::Op(Op::Gt));
|
out.push((Token::Op(Op::Gt), i..i + 1));
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
b'"' => {
|
b'"' => {
|
||||||
// Double-quoted phrase. Supports doubled-quote escape `""`.
|
// Double-quoted phrase. Supports doubled-quote escape `""`.
|
||||||
|
//
|
||||||
|
// Copied as a UTF-8 slice, not byte by byte: `bytes[j] as char`
|
||||||
|
// decodes Latin-1, so `"José"` came back as `José` and matched
|
||||||
|
// nothing. Quoting is also how people write paths containing
|
||||||
|
// spaces, which makes this the more visible of the two.
|
||||||
let mut j = i + 1;
|
let mut j = i + 1;
|
||||||
let mut buf = String::new();
|
let mut buf = String::new();
|
||||||
|
let mut segment_start = j;
|
||||||
while j < bytes.len() {
|
while j < bytes.len() {
|
||||||
if bytes[j] == b'"' {
|
if bytes[j] == b'"' {
|
||||||
|
buf.push_str(&input[segment_start..j]);
|
||||||
if bytes.get(j + 1) == Some(&b'"') {
|
if bytes.get(j + 1) == Some(&b'"') {
|
||||||
buf.push('"');
|
buf.push('"');
|
||||||
j += 2;
|
j += 2;
|
||||||
|
segment_start = j;
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
buf.push(bytes[j] as char);
|
|
||||||
j += 1;
|
j += 1;
|
||||||
}
|
}
|
||||||
if j >= bytes.len() {
|
if j >= bytes.len() {
|
||||||
return Err(LexError {
|
return (
|
||||||
|
out,
|
||||||
|
Some(LexError {
|
||||||
message: "unterminated quoted phrase".into(),
|
message: "unterminated quoted phrase".into(),
|
||||||
offset: i,
|
offset: i,
|
||||||
});
|
}),
|
||||||
|
);
|
||||||
}
|
}
|
||||||
out.push(Token::Quoted(buf));
|
out.push((Token::Quoted(buf), i..j + 1));
|
||||||
i = j + 1;
|
i = j + 1;
|
||||||
}
|
}
|
||||||
_ => {
|
_ => {
|
||||||
|
|
@ -108,6 +143,14 @@ pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError> {
|
||||||
let start = i;
|
let start = i;
|
||||||
while i < bytes.len() {
|
while i < bytes.len() {
|
||||||
let c = bytes[i];
|
let c = bytes[i];
|
||||||
|
if c == b':' && is_drive_letter_colon(bytes, start, i) {
|
||||||
|
// `C:\Users\me` is one word, not `C` `:` `\Users\me`.
|
||||||
|
// Without this, `path:C:\Users\me` parses as the
|
||||||
|
// filter `path` = `C` and silently matches nothing —
|
||||||
|
// the first thing a Windows user types.
|
||||||
|
i += 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
if c.is_ascii_whitespace()
|
if c.is_ascii_whitespace()
|
||||||
|| matches!(c, b'(' | b')' | b':' | b'=' | b'<' | b'>' | b'"')
|
|| matches!(c, b'(' | b')' | b':' | b'=' | b'<' | b'>' | b'"')
|
||||||
{
|
{
|
||||||
|
|
@ -115,22 +158,29 @@ pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError> {
|
||||||
}
|
}
|
||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
let word = std::str::from_utf8(&bytes[start..i])
|
let word = match std::str::from_utf8(&bytes[start..i]) {
|
||||||
.map_err(|e| LexError {
|
Ok(w) => w.to_string(),
|
||||||
|
Err(e) => {
|
||||||
|
return (
|
||||||
|
out,
|
||||||
|
Some(LexError {
|
||||||
message: format!("invalid UTF-8 in word: {}", e),
|
message: format!("invalid UTF-8 in word: {}", e),
|
||||||
offset: start,
|
offset: start,
|
||||||
})?
|
}),
|
||||||
.to_string();
|
)
|
||||||
match word.as_str() {
|
|
||||||
"AND" => out.push(Token::And),
|
|
||||||
"OR" => out.push(Token::Or),
|
|
||||||
_ => out.push(Token::Word(word)),
|
|
||||||
}
|
}
|
||||||
|
};
|
||||||
|
let tok = match word.as_str() {
|
||||||
|
"AND" => Token::And,
|
||||||
|
"OR" => Token::Or,
|
||||||
|
_ => Token::Word(word),
|
||||||
|
};
|
||||||
|
out.push((tok, start..i));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
Ok(out)
|
(out, None)
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|
@ -211,4 +261,153 @@ mod tests {
|
||||||
fn unterminated_quote_is_error() {
|
fn unterminated_quote_is_error() {
|
||||||
assert!(tokenize(r#""oops"#).is_err());
|
assert!(tokenize(r#""oops"#).is_err());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn non_ascii_survives_a_quoted_phrase() {
|
||||||
|
// Byte-wise copying decoded this as Latin-1 (`José`), so the phrase
|
||||||
|
// never matched anything.
|
||||||
|
let t = tokenize(r#""C:\Users\José\docs""#).unwrap();
|
||||||
|
assert_eq!(t, vec![Token::Quoted(r"C:\Users\José\docs".into())]);
|
||||||
|
|
||||||
|
// ...including around a doubled-quote escape, which splits the copy.
|
||||||
|
let t = tokenize(r#""ü""ö""#).unwrap();
|
||||||
|
assert_eq!(t, vec![Token::Quoted(r#"ü"ö"#.into())]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_drive_letter_colon_does_not_split_the_word() {
|
||||||
|
let t = tokenize(r"path:C:\Users\me\docs").unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
t,
|
||||||
|
vec![
|
||||||
|
Token::Word("path".into()),
|
||||||
|
Token::Op(Op::Contains),
|
||||||
|
Token::Word(r"C:\Users\me\docs".into()),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
|
||||||
|
// Forward slashes are equally valid on Windows.
|
||||||
|
let t = tokenize("path:D:/data").unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
t,
|
||||||
|
vec![
|
||||||
|
Token::Word("path".into()),
|
||||||
|
Token::Op(Op::Contains),
|
||||||
|
Token::Word("D:/data".into()),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn spans_cover_every_token_shape() {
|
||||||
|
let input = r#"type:Audio "a b" (x) size<=5"#;
|
||||||
|
let (toks, err) = tokenize_spanned(input);
|
||||||
|
assert!(err.is_none());
|
||||||
|
let spanned: Vec<(&str, Token)> = toks
|
||||||
|
.iter()
|
||||||
|
.map(|(t, r)| (&input[r.clone()], t.clone()))
|
||||||
|
.collect();
|
||||||
|
assert_eq!(
|
||||||
|
spanned,
|
||||||
|
vec![
|
||||||
|
("type", Token::Word("type".into())),
|
||||||
|
(":", Token::Op(Op::Contains)),
|
||||||
|
("Audio", Token::Word("Audio".into())),
|
||||||
|
(r#""a b""#, Token::Quoted("a b".into())),
|
||||||
|
("(", Token::LParen),
|
||||||
|
("x", Token::Word("x".into())),
|
||||||
|
(")", Token::RParen),
|
||||||
|
("size", Token::Word("size".into())),
|
||||||
|
("<=", Token::Op(Op::Le)),
|
||||||
|
("5", Token::Word("5".into())),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_span_includes_quotes_and_escapes() {
|
||||||
|
let input = r#"x "a""b" y"#;
|
||||||
|
let (toks, err) = tokenize_spanned(input);
|
||||||
|
assert!(err.is_none());
|
||||||
|
assert_eq!(toks[1].0, Token::Quoted(r#"a"b"#.into()));
|
||||||
|
assert_eq!(&input[toks[1].1.clone()], r#""a""b""#);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn spans_are_byte_offsets_around_non_ascii() {
|
||||||
|
let input = r#"José "café" naïve"#;
|
||||||
|
let (toks, err) = tokenize_spanned(input);
|
||||||
|
assert!(err.is_none());
|
||||||
|
assert_eq!(&input[toks[0].1.clone()], "José");
|
||||||
|
assert_eq!(&input[toks[1].1.clone()], r#""café""#);
|
||||||
|
assert_eq!(&input[toks[2].1.clone()], "naïve");
|
||||||
|
assert_eq!(toks[2].1.end, input.len());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unterminated_quote_keeps_prefix_tokens() {
|
||||||
|
let (toks, err) = tokenize_spanned(r#"type:Audio "oops"#);
|
||||||
|
let err = err.expect("should report the unterminated quote");
|
||||||
|
assert_eq!(err.offset, 11, "offset of the opening quote");
|
||||||
|
assert_eq!(
|
||||||
|
toks.iter().map(|(t, _)| t.clone()).collect::<Vec<_>>(),
|
||||||
|
vec![
|
||||||
|
Token::Word("type".into()),
|
||||||
|
Token::Op(Op::Contains),
|
||||||
|
Token::Word("Audio".into()),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn stars_stay_inside_words() {
|
||||||
|
let (toks, err) = tokenize_spanned("foo*bar *");
|
||||||
|
assert!(err.is_none());
|
||||||
|
assert_eq!(
|
||||||
|
toks.iter().map(|(t, _)| t.clone()).collect::<Vec<_>>(),
|
||||||
|
vec![Token::Word("foo*bar".into()), Token::Word("*".into())]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tokenize_matches_span_stripped_tokenize_spanned() {
|
||||||
|
for input in ["a AND (b:c)", r#""q" x>=2"#, "path:C:\\U foo*"] {
|
||||||
|
let plain = tokenize(input).unwrap();
|
||||||
|
let (spanned, err) = tokenize_spanned(input);
|
||||||
|
assert!(err.is_none());
|
||||||
|
let stripped: Vec<Token> = spanned.into_iter().map(|(t, _)| t).collect();
|
||||||
|
assert_eq!(plain, stripped, "input {:?}", input);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_drive_letter_rule_stays_narrow() {
|
||||||
|
// Two digits before the colon: still a time, not a drive.
|
||||||
|
assert_eq!(
|
||||||
|
tokenize("12:30").unwrap(),
|
||||||
|
vec![
|
||||||
|
Token::Word("12".into()),
|
||||||
|
Token::Op(Op::Contains),
|
||||||
|
Token::Word("30".into()),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
// One letter, but no separator after the colon.
|
||||||
|
assert_eq!(
|
||||||
|
tokenize("a:b").unwrap(),
|
||||||
|
vec![
|
||||||
|
Token::Word("a".into()),
|
||||||
|
Token::Op(Op::Contains),
|
||||||
|
Token::Word("b".into()),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
// A separator, but the key is longer than one character.
|
||||||
|
assert_eq!(
|
||||||
|
tokenize("type:/Audio").unwrap(),
|
||||||
|
vec![
|
||||||
|
Token::Word("type".into()),
|
||||||
|
Token::Op(Op::Contains),
|
||||||
|
Token::Word("/Audio".into()),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,12 @@
|
||||||
pub mod ast;
|
pub mod ast;
|
||||||
pub mod lexer;
|
pub mod lexer;
|
||||||
pub mod parser;
|
pub mod parser;
|
||||||
|
pub mod pattern;
|
||||||
|
pub mod split;
|
||||||
pub mod translator;
|
pub mod translator;
|
||||||
|
|
||||||
pub use ast::{Op, Term};
|
pub use ast::{Op, Term};
|
||||||
pub use translator::{parse_and_build, SqlQuery};
|
pub use lexer::tokenize_spanned;
|
||||||
|
pub use pattern::{RegexQuery, TermPattern};
|
||||||
|
pub use split::{split_for_cascade, CascadeQuery};
|
||||||
|
pub use translator::{parse_and_build, SqlQuery, TranslateError};
|
||||||
|
|
|
||||||
533
crates/quicksearch-core/src/query/pattern.rs
Normal file
|
|
@ -0,0 +1,533 @@
|
||||||
|
//! Compiled matchers for the cascade term and the `regex:` keyword.
|
||||||
|
//!
|
||||||
|
//! A term with an unquoted `*` compiles to a small regex (every literal
|
||||||
|
//! chunk escaped, stars joined with `.*`), so wildcards and `regex:` share
|
||||||
|
//! one linear-time matching engine. Terms without stars stay on the
|
||||||
|
//! [`Literal`](TermPattern::Literal) path, which reproduces the cascade's
|
||||||
|
//! original `==`/`find`/`count_occurrences` semantics byte for byte.
|
||||||
|
//!
|
||||||
|
//! `.` never matches `\n`, so a star cannot span lines of extracted text —
|
||||||
|
//! a `*` bridging a whole document would produce absurd match ranges and
|
||||||
|
//! page-sized snippets. Names and paths contain no newlines, so the rule
|
||||||
|
//! only shows up in content matching.
|
||||||
|
|
||||||
|
use std::ops::Range;
|
||||||
|
|
||||||
|
use regex::{Regex, RegexBuilder};
|
||||||
|
|
||||||
|
use super::translator::TranslateError;
|
||||||
|
use crate::snippet;
|
||||||
|
|
||||||
|
/// Compile-time memory cap for user-supplied and derived regexes. Keeps a
|
||||||
|
/// hostile pattern (`a{1000000}{1000}` and friends) from ballooning the
|
||||||
|
/// compiled program; matching itself is linear-time by construction.
|
||||||
|
const REGEX_SIZE_LIMIT: usize = 4 << 20;
|
||||||
|
|
||||||
|
/// Occurrence counts saturate here, matching `count_frac` in the cascade.
|
||||||
|
const COUNT_CAP: usize = 1000;
|
||||||
|
|
||||||
|
/// The cascade term, compiled once at split time.
|
||||||
|
#[derive(Debug, Clone, Default)]
|
||||||
|
pub enum TermPattern {
|
||||||
|
/// No matchable content: an empty term, or only stars (`*`, `**`).
|
||||||
|
/// Matches nothing — a bare `*` must not become a scan of everything.
|
||||||
|
#[default]
|
||||||
|
Empty,
|
||||||
|
/// A star-free term. Kept as plain string operations, not a regex, so
|
||||||
|
/// existing queries keep their exact semantics (and speed).
|
||||||
|
Literal(LiteralPattern),
|
||||||
|
/// A term with at least one active wildcard.
|
||||||
|
Wildcard(WildcardPattern),
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct LiteralPattern {
|
||||||
|
text: String,
|
||||||
|
folded: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct WildcardPattern {
|
||||||
|
/// Literal chunks between stars, in order. Never empty, and no chunk
|
||||||
|
/// is empty: edge stars are folded into the compiled regexes, doubled
|
||||||
|
/// stars collapse.
|
||||||
|
segments: Vec<String>,
|
||||||
|
/// Unanchored search regexes with non-greedy joins — leftmost-shortest
|
||||||
|
/// match, which is what a snippet window wants.
|
||||||
|
search_cs: Regex,
|
||||||
|
search_ci: Regex,
|
||||||
|
/// Anchored (`^…$`) regexes for whole-field matching (rank tiers 1/2).
|
||||||
|
anchored_cs: Regex,
|
||||||
|
anchored_ci: Regex,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One piece of the search phrase as split out of the token stream.
|
||||||
|
/// `glob` is true only for plain unquoted words — quoted phrases and
|
||||||
|
/// reassembled `key:value` text keep their stars literal.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct TermPart {
|
||||||
|
pub text: String,
|
||||||
|
pub glob: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A chunk stream: literal text interleaved with active stars.
|
||||||
|
enum Chunk {
|
||||||
|
Lit(String),
|
||||||
|
Star,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TermPattern {
|
||||||
|
/// Compile the joined term parts. Parts are joined with a single space,
|
||||||
|
/// exactly like the display term (`parts.join(" ")`).
|
||||||
|
pub fn build(parts: &[TermPart]) -> Result<TermPattern, TranslateError> {
|
||||||
|
let mut chunks: Vec<Chunk> = Vec::new();
|
||||||
|
let push_lit = |chunks: &mut Vec<Chunk>, s: &str| {
|
||||||
|
if s.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if let Some(Chunk::Lit(prev)) = chunks.last_mut() {
|
||||||
|
prev.push_str(s);
|
||||||
|
} else {
|
||||||
|
chunks.push(Chunk::Lit(s.to_string()));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for (idx, part) in parts.iter().enumerate() {
|
||||||
|
if idx > 0 {
|
||||||
|
push_lit(&mut chunks, " ");
|
||||||
|
}
|
||||||
|
if part.glob {
|
||||||
|
let mut first = true;
|
||||||
|
for piece in part.text.split('*') {
|
||||||
|
if !first && !matches!(chunks.last(), Some(Chunk::Star)) {
|
||||||
|
chunks.push(Chunk::Star);
|
||||||
|
}
|
||||||
|
first = false;
|
||||||
|
push_lit(&mut chunks, piece);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
push_lit(&mut chunks, &part.text);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let leading = matches!(chunks.first(), Some(Chunk::Star));
|
||||||
|
let trailing = chunks.len() > 1 && matches!(chunks.last(), Some(Chunk::Star));
|
||||||
|
let has_star = chunks.iter().any(|c| matches!(c, Chunk::Star));
|
||||||
|
let segments: Vec<String> = chunks
|
||||||
|
.into_iter()
|
||||||
|
.filter_map(|c| match c {
|
||||||
|
Chunk::Lit(s) => Some(s),
|
||||||
|
Chunk::Star => None,
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
if segments.is_empty() {
|
||||||
|
// "" or stars only.
|
||||||
|
return Ok(TermPattern::Empty);
|
||||||
|
}
|
||||||
|
if !has_star {
|
||||||
|
let text = segments.into_iter().next().unwrap();
|
||||||
|
let folded = text.to_ascii_lowercase();
|
||||||
|
return Ok(TermPattern::Literal(LiteralPattern { text, folded }));
|
||||||
|
}
|
||||||
|
|
||||||
|
let escaped: Vec<String> = segments.iter().map(|s| regex::escape(s)).collect();
|
||||||
|
let compile = |src: &str, ci: bool| -> Result<Regex, TranslateError> {
|
||||||
|
RegexBuilder::new(src)
|
||||||
|
.case_insensitive(ci)
|
||||||
|
.size_limit(REGEX_SIZE_LIMIT)
|
||||||
|
.build()
|
||||||
|
.map_err(|e| TranslateError::BadRegex(e.to_string()))
|
||||||
|
};
|
||||||
|
// Edge stars are dropped from the search form — under substring
|
||||||
|
// semantics a leading/trailing `.*?` adds nothing.
|
||||||
|
let search_src = escaped.join(".*?");
|
||||||
|
// The anchored form keeps them: `*foo` must whole-match "myfoo".
|
||||||
|
let anchored_src = format!(
|
||||||
|
"^{}{}{}$",
|
||||||
|
if leading { ".*" } else { "" },
|
||||||
|
escaped.join(".*"),
|
||||||
|
if trailing { ".*" } else { "" },
|
||||||
|
);
|
||||||
|
Ok(TermPattern::Wildcard(WildcardPattern {
|
||||||
|
search_cs: compile(&search_src, false)?,
|
||||||
|
search_ci: compile(&search_src, true)?,
|
||||||
|
anchored_cs: compile(&anchored_src, false)?,
|
||||||
|
anchored_ci: compile(&anchored_src, true)?,
|
||||||
|
segments,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn is_empty(&self) -> bool {
|
||||||
|
matches!(self, TermPattern::Empty)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn is_wildcard(&self) -> bool {
|
||||||
|
matches!(self, TermPattern::Wildcard(_))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The literal text, when the term has no wildcard. SQL builders branch
|
||||||
|
/// on this to keep the original single-`LIKE`/phrase-`MATCH` shapes.
|
||||||
|
pub fn literal(&self) -> Option<&str> {
|
||||||
|
match self {
|
||||||
|
TermPattern::Literal(l) => Some(&l.text),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Literal chunks between wildcards (the whole term when literal).
|
||||||
|
pub fn segments(&self) -> &[String] {
|
||||||
|
match self {
|
||||||
|
TermPattern::Empty => &[],
|
||||||
|
TermPattern::Literal(l) => std::slice::from_ref(&l.text),
|
||||||
|
TermPattern::Wildcard(w) => &w.segments,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Characters of literal (non-star) content — the trigram floor and
|
||||||
|
/// path-tier switch count these.
|
||||||
|
pub fn literal_char_count(&self) -> usize {
|
||||||
|
self.segments().iter().map(|s| s.chars().count()).sum()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Does the pattern match the entire field?
|
||||||
|
pub fn whole_match(&self, text: &str, case_insensitive: bool) -> bool {
|
||||||
|
match self {
|
||||||
|
TermPattern::Empty => false,
|
||||||
|
TermPattern::Literal(l) => {
|
||||||
|
if case_insensitive {
|
||||||
|
text.eq_ignore_ascii_case(&l.text)
|
||||||
|
} else {
|
||||||
|
text == l.text
|
||||||
|
}
|
||||||
|
}
|
||||||
|
TermPattern::Wildcard(w) => {
|
||||||
|
let re = if case_insensitive {
|
||||||
|
&w.anchored_ci
|
||||||
|
} else {
|
||||||
|
&w.anchored_cs
|
||||||
|
};
|
||||||
|
re.is_match(text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Leftmost match as a byte range. Literal folding is ASCII-only and
|
||||||
|
/// byte-length preserving, so folded offsets are valid in the original —
|
||||||
|
/// the same invariant the cascade has always relied on.
|
||||||
|
pub fn find_first(&self, text: &str, case_insensitive: bool) -> Option<Range<usize>> {
|
||||||
|
match self {
|
||||||
|
TermPattern::Empty => None,
|
||||||
|
TermPattern::Literal(l) => {
|
||||||
|
let pos = if case_insensitive {
|
||||||
|
text.to_ascii_lowercase().find(&l.folded)?
|
||||||
|
} else {
|
||||||
|
text.find(&l.text)?
|
||||||
|
};
|
||||||
|
Some(pos..pos + l.text.len())
|
||||||
|
}
|
||||||
|
TermPattern::Wildcard(w) => {
|
||||||
|
let re = if case_insensitive {
|
||||||
|
&w.search_ci
|
||||||
|
} else {
|
||||||
|
&w.search_cs
|
||||||
|
};
|
||||||
|
re.find(text).map(|m| m.range())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Non-overlapping occurrence count, capped at 1000 (the cascade's
|
||||||
|
/// `count_frac` saturates there anyway).
|
||||||
|
pub fn count(&self, text: &str, case_insensitive: bool) -> usize {
|
||||||
|
match self {
|
||||||
|
TermPattern::Empty => 0,
|
||||||
|
TermPattern::Literal(l) => {
|
||||||
|
snippet::count_occurrences(text, &l.text, !case_insensitive)
|
||||||
|
}
|
||||||
|
TermPattern::Wildcard(w) => {
|
||||||
|
let re = if case_insensitive {
|
||||||
|
&w.search_ci
|
||||||
|
} else {
|
||||||
|
&w.search_cs
|
||||||
|
};
|
||||||
|
re.find_iter(text).take(COUNT_CAP).count()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A compiled `regex:` query. Case-insensitive by default (override with an
|
||||||
|
/// inline `(?-i:…)`); `multi_line` makes `^`/`$` per-line over extracted
|
||||||
|
/// text, which is what they mean in a search box.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct RegexQuery {
|
||||||
|
pub source: String,
|
||||||
|
re: Regex,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RegexQuery {
|
||||||
|
pub fn new(source: &str) -> Result<RegexQuery, TranslateError> {
|
||||||
|
let re = RegexBuilder::new(source)
|
||||||
|
.case_insensitive(true)
|
||||||
|
.multi_line(true)
|
||||||
|
.size_limit(REGEX_SIZE_LIMIT)
|
||||||
|
.build()
|
||||||
|
.map_err(|e| TranslateError::BadRegex(e.to_string()))?;
|
||||||
|
// The regex analog of the bare-`*` rule, but loud: the user typed an
|
||||||
|
// explicit keyword, so tell them instead of matching every file.
|
||||||
|
if re.is_match("") {
|
||||||
|
return Err(TranslateError::BadRegex(format!(
|
||||||
|
"'{}' can match the empty string and would match every file",
|
||||||
|
source
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Ok(RegexQuery {
|
||||||
|
source: source.to_string(),
|
||||||
|
re,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn is_match(&self, text: &str) -> bool {
|
||||||
|
self.re.is_match(text)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn find_first(&self, text: &str) -> Option<Range<usize>> {
|
||||||
|
self.re.find(text).map(|m| m.range())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Non-overlapping occurrence count, capped at 1000.
|
||||||
|
pub fn count(&self, text: &str) -> usize {
|
||||||
|
self.re.find_iter(text).take(COUNT_CAP).count()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Cap a match range at `max_len` bytes (aligned back to a char boundary)
|
||||||
|
/// before handing it to `snippet::window_around`. A greedy user regex can
|
||||||
|
/// legitimately match megabytes of a minified file; the snippet window
|
||||||
|
/// wants the start of that, not all of it.
|
||||||
|
pub fn clamp_match_range(text: &str, range: Range<usize>, max_len: usize) -> Range<usize> {
|
||||||
|
let mut end = range.end.min(range.start + max_len);
|
||||||
|
while end > range.start && !text.is_char_boundary(end) {
|
||||||
|
end -= 1;
|
||||||
|
}
|
||||||
|
range.start..end
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn part(text: &str, glob: bool) -> TermPart {
|
||||||
|
TermPart {
|
||||||
|
text: text.into(),
|
||||||
|
glob,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn wildcard(parts: &[TermPart]) -> WildcardPattern {
|
||||||
|
match TermPattern::build(parts).unwrap() {
|
||||||
|
TermPattern::Wildcard(w) => w,
|
||||||
|
other => panic!("expected wildcard, got {:?}", other),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn starless_parts_build_a_literal() {
|
||||||
|
let p = TermPattern::build(&[part("hello", false), part("world", true)]).unwrap();
|
||||||
|
assert_eq!(p.literal(), Some("hello world"));
|
||||||
|
assert!(!p.is_wildcard());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn empty_and_star_only_terms_match_nothing() {
|
||||||
|
for parts in [
|
||||||
|
vec![],
|
||||||
|
vec![part("", false)],
|
||||||
|
vec![part("*", true)],
|
||||||
|
vec![part("**", true)],
|
||||||
|
] {
|
||||||
|
let p = TermPattern::build(&parts).unwrap();
|
||||||
|
assert!(p.is_empty(), "{:?}", parts);
|
||||||
|
assert!(!p.whole_match("anything", true));
|
||||||
|
assert!(p.find_first("anything", true).is_none());
|
||||||
|
assert_eq!(p.count("anything", true), 0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_star_stays_literal() {
|
||||||
|
// A quoted "*" arrives with glob = false.
|
||||||
|
let p = TermPattern::build(&[part("a*b", false)]).unwrap();
|
||||||
|
assert_eq!(p.literal(), Some("a*b"));
|
||||||
|
assert!(p.find_first("xa*by", false).is_some());
|
||||||
|
assert!(p.find_first("aXb", false).is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn segment_shapes() {
|
||||||
|
// Edge stars vanish into the anchors: `*foo` whole-matches any
|
||||||
|
// suffix `foo`, `foo*` any prefix.
|
||||||
|
let p = TermPattern::build(&[part("*foo", true)]).unwrap();
|
||||||
|
assert_eq!(p.segments(), ["foo"]);
|
||||||
|
assert!(p.whole_match("myfoo", false));
|
||||||
|
assert!(!p.whole_match("foomy", false));
|
||||||
|
|
||||||
|
let p = TermPattern::build(&[part("foo*", true)]).unwrap();
|
||||||
|
assert_eq!(p.segments(), ["foo"]);
|
||||||
|
assert!(p.whole_match("foomy", false));
|
||||||
|
assert!(!p.whole_match("myfoo", false));
|
||||||
|
|
||||||
|
let w = wildcard(&[part("f*o*o", true)]);
|
||||||
|
assert_eq!(w.segments, ["f", "o", "o"]);
|
||||||
|
|
||||||
|
// Doubled stars collapse.
|
||||||
|
let w = wildcard(&[part("f**o", true)]);
|
||||||
|
assert_eq!(w.segments, ["f", "o"]);
|
||||||
|
|
||||||
|
// The implicit joining space is literal content.
|
||||||
|
let w = wildcard(&[part("a*", true), part("b", false)]);
|
||||||
|
assert_eq!(w.segments, ["a", " b"]);
|
||||||
|
|
||||||
|
// `* *` — the joining space between two stars is interior literal
|
||||||
|
// content, so this is a real (if odd) pattern, not Empty.
|
||||||
|
let p = TermPattern::build(&[part("*", true), part("*", true)]).unwrap();
|
||||||
|
assert_eq!(p.segments(), [" "]);
|
||||||
|
assert!(p.whole_match("a b", false));
|
||||||
|
assert!(!p.whole_match("ab", false));
|
||||||
|
|
||||||
|
let p = TermPattern::build(&[part("*x", true), part("y*", true)]).unwrap();
|
||||||
|
assert_eq!(p.segments(), ["x y"]);
|
||||||
|
assert!(p.whole_match("ax yb", false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn whole_match_uses_anchors() {
|
||||||
|
let p = TermPattern::build(&[part("*.txt", true)]).unwrap();
|
||||||
|
assert!(p.whole_match("notes.txt", false));
|
||||||
|
assert!(p.whole_match("NOTES.TXT", true));
|
||||||
|
assert!(!p.whole_match("NOTES.TXT", false));
|
||||||
|
assert!(!p.whole_match("notes.txt.bak", false));
|
||||||
|
|
||||||
|
let p = TermPattern::build(&[part("rep*rt", true)]).unwrap();
|
||||||
|
assert!(p.whole_match("report", false));
|
||||||
|
assert!(!p.whole_match("report2024", false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn find_first_is_leftmost_shortest() {
|
||||||
|
let p = TermPattern::build(&[part("a*b", true)]).unwrap();
|
||||||
|
// Leftmost-first with a lazy join: starts at 0, ends at the first b.
|
||||||
|
assert_eq!(p.find_first("aXXbYYb", false), Some(0..4));
|
||||||
|
// Case-insensitive variant.
|
||||||
|
assert_eq!(p.find_first("AXXB", true), Some(0..4));
|
||||||
|
assert_eq!(p.find_first("AXXB", false), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn star_does_not_cross_newlines() {
|
||||||
|
let p = TermPattern::build(&[part("foo*bar", true)]).unwrap();
|
||||||
|
assert!(p.find_first("foo bar", false).is_some());
|
||||||
|
assert!(p.find_first("foo\nbar", false).is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn utf8_boundaries_in_segments_and_haystacks() {
|
||||||
|
let p = TermPattern::build(&[part("café*menu", true)]).unwrap();
|
||||||
|
let hay = "le café du menu";
|
||||||
|
let r = p.find_first(hay, false).unwrap();
|
||||||
|
assert_eq!(&hay[r], "café du menu");
|
||||||
|
// Case-insensitive over non-ASCII haystack: offsets stay valid.
|
||||||
|
let hay = "LE CAFÉ DU MENU";
|
||||||
|
let r = p.find_first(hay, true).unwrap();
|
||||||
|
assert!(hay.is_char_boundary(r.start) && hay.is_char_boundary(r.end));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn count_is_nonoverlapping_and_capped() {
|
||||||
|
let p = TermPattern::build(&[part("a*b", true)]).unwrap();
|
||||||
|
assert_eq!(p.count("ab ab ab", false), 3);
|
||||||
|
let many = "ab ".repeat(2000);
|
||||||
|
assert_eq!(p.count(&many, false), 1000);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn literal_parity_with_string_ops() {
|
||||||
|
let p = TermPattern::build(&[part("Report", false)]).unwrap();
|
||||||
|
assert!(p.whole_match("Report", false));
|
||||||
|
assert!(!p.whole_match("report", false));
|
||||||
|
assert!(p.whole_match("report", true));
|
||||||
|
assert_eq!(p.find_first("my Report.pdf", false), Some(3..9));
|
||||||
|
assert_eq!(p.find_first("my report.pdf", true), Some(3..9));
|
||||||
|
assert_eq!(p.find_first("my report.pdf", false), None);
|
||||||
|
assert_eq!(p.count("report Report", false), 1);
|
||||||
|
assert_eq!(p.count("report Report", true), 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn regex_defaults_case_insensitive_with_optout() {
|
||||||
|
let r = RegexQuery::new("foo\\d+").unwrap();
|
||||||
|
assert!(r.is_match("FOO123"));
|
||||||
|
let r = RegexQuery::new("(?-i:FOO)\\d+").unwrap();
|
||||||
|
assert!(r.is_match("FOO1"));
|
||||||
|
assert!(!r.is_match("foo1"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn regex_multiline_anchors() {
|
||||||
|
let r = RegexQuery::new("^total:").unwrap();
|
||||||
|
assert!(r.is_match("line one\ntotal: 5"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn invalid_regex_is_an_error_not_a_panic() {
|
||||||
|
for src in ["[", "(", "a{2,1}", "(?P<)"] {
|
||||||
|
assert!(
|
||||||
|
matches!(RegexQuery::new(src), Err(TranslateError::BadRegex(_))),
|
||||||
|
"{:?}",
|
||||||
|
src
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn empty_matchable_regexes_are_rejected() {
|
||||||
|
for src in ["", ".*", "a*", "x|", "()", "(a+)*"] {
|
||||||
|
assert!(
|
||||||
|
matches!(RegexQuery::new(src), Err(TranslateError::BadRegex(_))),
|
||||||
|
"{:?} should be rejected",
|
||||||
|
src
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn hostile_regexes_fail_fast_or_run_linear() {
|
||||||
|
// Deep nesting: rejected cleanly by the parser's nest limit.
|
||||||
|
assert!(RegexQuery::new(&"(".repeat(2000)).is_err());
|
||||||
|
// Huge counted repetition: rejected by size_limit, not compiled.
|
||||||
|
assert!(RegexQuery::new("a{1000000}{1000}").is_err());
|
||||||
|
// Classic backtracking bomb: the linear engine answers immediately
|
||||||
|
// (a backtracker would take exponential time here).
|
||||||
|
let r = RegexQuery::new("(a+)+$").unwrap();
|
||||||
|
let hay = format!("{}b", "a".repeat(10_000));
|
||||||
|
assert!(!r.is_match(&hay));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn find_and_count_on_regex() {
|
||||||
|
let r = RegexQuery::new("b[aeiou]d").unwrap();
|
||||||
|
let hay = "bad bed bodkin";
|
||||||
|
assert_eq!(r.find_first(hay), Some(0..3));
|
||||||
|
assert_eq!(r.count(hay), 3);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn clamp_respects_char_boundaries() {
|
||||||
|
let text = "aééééb";
|
||||||
|
let r = clamp_match_range(text, 0..text.len(), 4);
|
||||||
|
assert!(text.is_char_boundary(r.end));
|
||||||
|
assert!(r.end <= 4);
|
||||||
|
// No-op when already short enough.
|
||||||
|
assert_eq!(clamp_match_range(text, 1..3, 100), 1..3);
|
||||||
|
}
|
||||||
|
}
|
||||||
457
crates/quicksearch-core/src/query/split.rs
Normal file
|
|
@ -0,0 +1,457 @@
|
||||||
|
//! Split raw search-box input into (cascade term, structured filters).
|
||||||
|
//!
|
||||||
|
//! The ranked search cascade has no boolean logic by design: everything
|
||||||
|
//! that isn't a recognized `key:value` filter joins the *term* — the single
|
||||||
|
//! phrase the cascade matches through its filename/full-text/fuzzy stages.
|
||||||
|
//! Recognized filters (`type:`, `modified:`, `path:`, `mime:`, `name:`)
|
||||||
|
//! become parameterized SQL fragments ANDed onto every cascade stage.
|
||||||
|
//!
|
||||||
|
//! Robustness rules for search-as-you-type:
|
||||||
|
//! - A lex error (e.g. a half-typed quote) degrades to "whole input is the
|
||||||
|
//! term" — incremental typing must never surface an error.
|
||||||
|
//! - An *unrecognized* `key:value` (like `12:30`) is reassembled verbatim
|
||||||
|
//! into the term.
|
||||||
|
//! - A recognized key whose value doesn't translate (bad date, unknown type
|
||||||
|
//! name) is a real [`TranslateError`] — the caller shows it inline.
|
||||||
|
//! - `AND`/`OR`/parens are not operators here; the words pass through into
|
||||||
|
//! the term, parens are dropped.
|
||||||
|
|
||||||
|
use super::ast::Op;
|
||||||
|
use super::lexer::{tokenize, Token};
|
||||||
|
use super::pattern::{RegexQuery, TermPart, TermPattern};
|
||||||
|
use super::translator::{build_filter, is_filter_key, TranslateError};
|
||||||
|
|
||||||
|
/// The cascade's parsed input: one term string plus composable filter SQL.
|
||||||
|
#[derive(Debug, Clone, Default)]
|
||||||
|
pub struct CascadeQuery {
|
||||||
|
/// The ranked search phrase; may be empty when the input was
|
||||||
|
/// filter-only or regex-only.
|
||||||
|
pub term: String,
|
||||||
|
/// `term` compiled for matching: literal, wildcard, or empty.
|
||||||
|
pub pattern: TermPattern,
|
||||||
|
/// A `regex:` filter, matched in Rust against name, path and content —
|
||||||
|
/// never part of the SQL.
|
||||||
|
pub regex: Option<RegexQuery>,
|
||||||
|
/// Zero or more ` AND (...)` fragments with anonymous `?` placeholders
|
||||||
|
/// over alias `f`; appended verbatim to every stage's WHERE clause.
|
||||||
|
pub filter_sql: String,
|
||||||
|
pub filter_params: Vec<rusqlite::types::Value>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CascadeQuery {
|
||||||
|
/// Nothing to rank on: no term pattern and no regex. (Filters alone
|
||||||
|
/// don't drive a search.)
|
||||||
|
pub fn is_empty(&self) -> bool {
|
||||||
|
self.pattern.is_empty() && self.regex.is_none()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn term_only(term: &str) -> CascadeQuery {
|
||||||
|
let term = term.trim().to_string();
|
||||||
|
// Un-lexable input is searched verbatim — stars are not wildcards
|
||||||
|
// here, mirroring the "whole input is the term" degrade rule.
|
||||||
|
let pattern = TermPattern::build(&[TermPart {
|
||||||
|
text: term.clone(),
|
||||||
|
glob: false,
|
||||||
|
}])
|
||||||
|
.expect("literal patterns always compile");
|
||||||
|
CascadeQuery {
|
||||||
|
term,
|
||||||
|
pattern,
|
||||||
|
..CascadeQuery::default()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn op_str(op: Op) -> &'static str {
|
||||||
|
match op {
|
||||||
|
Op::Contains => ":",
|
||||||
|
Op::Eq => "=",
|
||||||
|
Op::Lt => "<",
|
||||||
|
Op::Le => "<=",
|
||||||
|
Op::Gt => ">",
|
||||||
|
Op::Ge => ">=",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn split_for_cascade(input: &str) -> Result<CascadeQuery, TranslateError> {
|
||||||
|
// NUL bytes can't occur in filenames or extracted text, but they do
|
||||||
|
// break SQLite text binding and the FTS5 query parser — strip them.
|
||||||
|
let input = input.replace('\0', "");
|
||||||
|
let input = input.as_str();
|
||||||
|
let tokens = match tokenize(input) {
|
||||||
|
Ok(t) => t,
|
||||||
|
// Half-typed input (unterminated quote, invalid word): the whole
|
||||||
|
// raw string is the term. Never an error mid-keystroke.
|
||||||
|
Err(_) => return Ok(CascadeQuery::term_only(input)),
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut out = CascadeQuery::default();
|
||||||
|
let mut term_parts: Vec<TermPart> = Vec::new();
|
||||||
|
let mut i = 0usize;
|
||||||
|
// Only plain unquoted words are wildcard-eligible; everything else
|
||||||
|
// (quoted phrases, demoted AND/OR, reassembled key:value glue) is
|
||||||
|
// searched verbatim.
|
||||||
|
let literal = |text: &str| TermPart {
|
||||||
|
text: text.to_string(),
|
||||||
|
glob: false,
|
||||||
|
};
|
||||||
|
|
||||||
|
while i < tokens.len() {
|
||||||
|
match &tokens[i] {
|
||||||
|
Token::Word(word) => {
|
||||||
|
// Candidate filter: Word(key) Op [Op] (Word|Quoted).
|
||||||
|
// The lexer emits `modified:>=x` as Word Op(:) Op(>=) Word.
|
||||||
|
if let Some(Token::Op(op1)) = tokens.get(i + 1) {
|
||||||
|
let (op, value_idx) = match tokens.get(i + 2) {
|
||||||
|
Some(Token::Op(op2)) => (*op2, i + 3),
|
||||||
|
_ => (*op1, i + 2),
|
||||||
|
};
|
||||||
|
let value = match tokens.get(value_idx) {
|
||||||
|
Some(Token::Word(v)) | Some(Token::Quoted(v)) => Some(v.clone()),
|
||||||
|
_ => None,
|
||||||
|
};
|
||||||
|
if let Some(value) = value {
|
||||||
|
// Quoted values keep `*` literal; only a bare word's
|
||||||
|
// stars act as wildcards (`name:` honors this too).
|
||||||
|
let value_is_word =
|
||||||
|
matches!(tokens.get(value_idx), Some(Token::Word(_)));
|
||||||
|
if word.eq_ignore_ascii_case("regex") {
|
||||||
|
// Not a SQL filter: compiled here, matched in
|
||||||
|
// Rust against name, path and content.
|
||||||
|
if op != Op::Contains {
|
||||||
|
return Err(TranslateError::UnsupportedOp {
|
||||||
|
key: word.clone(),
|
||||||
|
op,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if out.regex.is_some() {
|
||||||
|
return Err(TranslateError::BadRegex(
|
||||||
|
"only one regex: per query".into(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
out.regex = Some(RegexQuery::new(&value)?);
|
||||||
|
i = value_idx + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if is_filter_key(word) {
|
||||||
|
let frag = build_filter(word, op, &value, value_is_word)?;
|
||||||
|
out.filter_sql.push_str(" AND (");
|
||||||
|
out.filter_sql.push_str(&frag.sql);
|
||||||
|
out.filter_sql.push(')');
|
||||||
|
out.filter_params.extend(frag.params);
|
||||||
|
i = value_idx + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Unrecognized key — reassemble verbatim (`12:30`,
|
||||||
|
// `foo:bar`), gluing any further `:value` chains
|
||||||
|
// (`foo:bar:baz`).
|
||||||
|
let mut glued = format!("{}{}{}", word, op_str(op), value);
|
||||||
|
i = value_idx + 1;
|
||||||
|
while let Some(Token::Op(next_op)) = tokens.get(i) {
|
||||||
|
glued.push_str(op_str(*next_op));
|
||||||
|
i += 1;
|
||||||
|
if let Some(Token::Word(v)) | Some(Token::Quoted(v)) = tokens.get(i)
|
||||||
|
{
|
||||||
|
glued.push_str(v);
|
||||||
|
i += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
term_parts.push(literal(&glued));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Key + op with no value yet (mid-typing "type:"):
|
||||||
|
// pass through as literal text.
|
||||||
|
term_parts.push(literal(&format!("{}{}", word, op_str(*op1))));
|
||||||
|
i += 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
term_parts.push(TermPart {
|
||||||
|
text: word.clone(),
|
||||||
|
glob: word.contains('*'),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
Token::Quoted(q) => term_parts.push(literal(q)),
|
||||||
|
// Not operators in the cascade grammar — plain words.
|
||||||
|
Token::And => term_parts.push(literal("AND")),
|
||||||
|
Token::Or => term_parts.push(literal("OR")),
|
||||||
|
// Grouping has no meaning without boolean logic.
|
||||||
|
Token::LParen | Token::RParen => {}
|
||||||
|
// Dangling operator (e.g. "a > b" typed literally).
|
||||||
|
Token::Op(op) => term_parts.push(literal(op_str(*op))),
|
||||||
|
}
|
||||||
|
i += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
out.term = term_parts
|
||||||
|
.iter()
|
||||||
|
.map(|p| p.text.as_str())
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(" ");
|
||||||
|
out.pattern = TermPattern::build(&term_parts)?;
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use rusqlite::types::Value;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn plain_words_join_in_order() {
|
||||||
|
let q = split_for_cascade("hello brave world").unwrap();
|
||||||
|
assert_eq!(q.term, "hello brave world");
|
||||||
|
assert!(q.filter_sql.is_empty());
|
||||||
|
assert!(q.filter_params.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn empty_input() {
|
||||||
|
let q = split_for_cascade("").unwrap();
|
||||||
|
assert_eq!(q.term, "");
|
||||||
|
assert!(q.filter_sql.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn each_recognized_filter_key_extracts() {
|
||||||
|
for input in [
|
||||||
|
"type:Audio",
|
||||||
|
"modified:>=2024-01-01",
|
||||||
|
"mtime:<2023-12-01",
|
||||||
|
"path:/home/me",
|
||||||
|
"folder:/home/me",
|
||||||
|
"includefolder:/home/me",
|
||||||
|
"name:report",
|
||||||
|
"filename:report",
|
||||||
|
"mime:application/pdf",
|
||||||
|
] {
|
||||||
|
let q = split_for_cascade(input).unwrap();
|
||||||
|
assert_eq!(q.term, "", "input {:?} should be pure filter", input);
|
||||||
|
assert!(
|
||||||
|
q.filter_sql.starts_with(" AND ("),
|
||||||
|
"input {:?} → {:?}",
|
||||||
|
input,
|
||||||
|
q.filter_sql
|
||||||
|
);
|
||||||
|
assert!(!q.filter_params.is_empty(), "input {:?}", input);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn filters_and_term_mix() {
|
||||||
|
let q = split_for_cascade("type:Document budget report modified:>=2024-01-01").unwrap();
|
||||||
|
assert_eq!(q.term, "budget report");
|
||||||
|
assert_eq!(q.filter_sql.matches(" AND (").count(), 2);
|
||||||
|
assert_eq!(q.filter_params.len(), 2); // type bits + mtime bound
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unknown_key_stays_literal() {
|
||||||
|
let q = split_for_cascade("meeting 12:30 notes").unwrap();
|
||||||
|
assert_eq!(q.term, "meeting 12:30 notes");
|
||||||
|
assert!(q.filter_sql.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unknown_key_chain_reassembles() {
|
||||||
|
let q = split_for_cascade("foo:bar:baz").unwrap();
|
||||||
|
assert_eq!(q.term, "foo:bar:baz");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn half_typed_quote_is_whole_term() {
|
||||||
|
let q = split_for_cascade("\"unclosed phrase").unwrap();
|
||||||
|
assert_eq!(q.term, "\"unclosed phrase");
|
||||||
|
assert!(q.filter_sql.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn half_typed_filter_key_is_literal() {
|
||||||
|
let q = split_for_cascade("type:").unwrap();
|
||||||
|
assert_eq!(q.term, "type:");
|
||||||
|
assert!(q.filter_sql.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn recognized_key_bad_value_errors() {
|
||||||
|
assert!(matches!(
|
||||||
|
split_for_cascade("modified:>=not-a-date"),
|
||||||
|
Err(TranslateError::BadDate(_))
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
split_for_cascade("type:NotAThing"),
|
||||||
|
Err(TranslateError::UnknownProperty(_))
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn and_or_parens_are_plain_text() {
|
||||||
|
let q = split_for_cascade("(alpha AND beta) OR gamma").unwrap();
|
||||||
|
assert_eq!(q.term, "alpha AND beta OR gamma");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The end-to-end shape of the bug: before the lexer fix this produced
|
||||||
|
/// the filter `parent = "C"` plus a junk term, and returned nothing.
|
||||||
|
#[test]
|
||||||
|
fn a_windows_drive_path_reaches_the_filter_intact() {
|
||||||
|
let q = split_for_cascade(r"path:C:\Users\me\docs").unwrap();
|
||||||
|
assert_eq!(q.term, "", "the whole input is a filter");
|
||||||
|
assert!(matches!(
|
||||||
|
&q.filter_params[0],
|
||||||
|
Value::Text(t) if t == r"C:\Users\me\docs"
|
||||||
|
), "{:?}", q.filter_params);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_value_for_filter() {
|
||||||
|
let q = split_for_cascade("path:\"/home/me/My Documents\"").unwrap();
|
||||||
|
assert_eq!(q.term, "");
|
||||||
|
assert_eq!(
|
||||||
|
q.filter_params[0],
|
||||||
|
Value::Text("/home/me/My Documents".into())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_phrase_joins_term() {
|
||||||
|
let q = split_for_cascade("\"exact phrase\" extra").unwrap();
|
||||||
|
assert_eq!(q.term, "exact phrase extra");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn injection_shapes_stay_bound() {
|
||||||
|
// Everything lands either in the term (never interpolated into
|
||||||
|
// SQL by the cascade — bound as parameters there too) or in
|
||||||
|
// filter_params. filter_sql must never contain user text.
|
||||||
|
let q = split_for_cascade("mime:application/x-foo'; DROP TABLE files; --").unwrap();
|
||||||
|
assert!(!q.filter_sql.contains("DROP"), "{}", q.filter_sql);
|
||||||
|
// The value went into params (term got the trailing junk words).
|
||||||
|
assert!(matches!(&q.filter_params[0], Value::Text(t) if t.contains("x-foo'")));
|
||||||
|
|
||||||
|
let q = split_for_cascade("name:%_\\").unwrap();
|
||||||
|
// LIKE-escaped inside the bound param, not the SQL.
|
||||||
|
assert!(matches!(&q.filter_params[0], Value::Text(t) if t == "%\\%\\_\\\\%"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unquoted_star_builds_a_wildcard_pattern() {
|
||||||
|
let q = split_for_cascade("foo*").unwrap();
|
||||||
|
assert_eq!(q.term, "foo*");
|
||||||
|
assert!(q.pattern.is_wildcard());
|
||||||
|
assert!(q.pattern.whole_match("foobar", false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_star_stays_literal() {
|
||||||
|
let q = split_for_cascade("\"foo*\"").unwrap();
|
||||||
|
assert_eq!(q.pattern.literal(), Some("foo*"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn bare_star_matches_nothing() {
|
||||||
|
for input in ["*", "**", "* *"] {
|
||||||
|
let q = split_for_cascade(input).unwrap();
|
||||||
|
assert!(
|
||||||
|
q.pattern.find_first("anything", true).is_none(),
|
||||||
|
"{:?}",
|
||||||
|
input
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// "* *" has an interior literal space; plain stars are Empty.
|
||||||
|
assert!(split_for_cascade("*").unwrap().pattern.is_empty());
|
||||||
|
assert!(split_for_cascade("*").unwrap().is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn glued_unknown_keys_keep_stars_literal() {
|
||||||
|
let q = split_for_cascade("foo:ba*r").unwrap();
|
||||||
|
assert_eq!(q.pattern.literal(), Some("foo:ba*r"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn lex_error_degrade_keeps_stars_literal() {
|
||||||
|
let q = split_for_cascade("re*port \"unclosed").unwrap();
|
||||||
|
assert_eq!(q.pattern.literal(), Some("re*port \"unclosed"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn regex_keyword_compiles_out_of_band() {
|
||||||
|
let q = split_for_cascade("regex:foo\\d+").unwrap();
|
||||||
|
assert_eq!(q.term, "");
|
||||||
|
assert!(q.pattern.is_empty());
|
||||||
|
assert!(!q.is_empty(), "a regex-only query still searches");
|
||||||
|
let re = q.regex.unwrap();
|
||||||
|
assert!(re.is_match("FOO12"));
|
||||||
|
assert!(q.filter_sql.is_empty(), "regex is not a SQL filter");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn regex_value_may_be_quoted_and_key_is_case_insensitive() {
|
||||||
|
let q = split_for_cascade("REGEX:\"foo (bar|baz)\"").unwrap();
|
||||||
|
assert!(q.regex.unwrap().is_match("foo bar"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn regex_mixes_with_filters_and_term() {
|
||||||
|
let q = split_for_cascade("regex:\\d+ type:Text budget").unwrap();
|
||||||
|
assert_eq!(q.term, "budget");
|
||||||
|
assert!(q.regex.is_some());
|
||||||
|
assert_eq!(q.filter_sql.matches(" AND (").count(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn regex_error_shapes() {
|
||||||
|
assert!(matches!(
|
||||||
|
split_for_cascade("regex:["),
|
||||||
|
Err(TranslateError::BadRegex(_))
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
split_for_cascade("regex=x"),
|
||||||
|
Err(TranslateError::UnsupportedOp { .. })
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
split_for_cascade("regex:a regex:b"),
|
||||||
|
Err(TranslateError::BadRegex(_))
|
||||||
|
));
|
||||||
|
// Empty-matchable patterns are rejected loudly.
|
||||||
|
assert!(matches!(
|
||||||
|
split_for_cascade("regex:.*"),
|
||||||
|
Err(TranslateError::BadRegex(_))
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn dangling_regex_key_is_literal_text() {
|
||||||
|
let q = split_for_cascade("regex:").unwrap();
|
||||||
|
assert_eq!(q.term, "regex:");
|
||||||
|
assert!(q.regex.is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn name_filter_star_becomes_like_wildcard() {
|
||||||
|
let q = split_for_cascade("name:foo*bar").unwrap();
|
||||||
|
assert!(matches!(&q.filter_params[0], Value::Text(t) if t == "%foo%bar%"));
|
||||||
|
|
||||||
|
// Quoted value: star stays a literal character.
|
||||||
|
let q = split_for_cascade("name:\"fo*o\"").unwrap();
|
||||||
|
assert!(matches!(&q.filter_params[0], Value::Text(t) if t == "%fo*o%"));
|
||||||
|
|
||||||
|
// User LIKE metacharacters stay escaped even in glob values.
|
||||||
|
let q = split_for_cascade("name:%*_").unwrap();
|
||||||
|
assert!(matches!(&q.filter_params[0], Value::Text(t) if t == "%\\%%\\_%"));
|
||||||
|
|
||||||
|
// path: values never glob.
|
||||||
|
let q = split_for_cascade("path:/da*ta").unwrap();
|
||||||
|
assert!(matches!(&q.filter_params[0], Value::Text(t) if t == "/da*ta"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn nul_bytes_stripped_and_long_terms_pass_through() {
|
||||||
|
// NULs would break SQLite binding / FTS5 parsing downstream.
|
||||||
|
let q = split_for_cascade("abc\0def").unwrap();
|
||||||
|
assert_eq!(q.term, "abcdef");
|
||||||
|
|
||||||
|
let long = "x".repeat(10_240);
|
||||||
|
let q = split_for_cascade(&long).unwrap();
|
||||||
|
assert_eq!(q.term.len(), 10_240);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -35,6 +35,7 @@ pub enum TranslateError {
|
||||||
Parse(ParseError),
|
Parse(ParseError),
|
||||||
UnknownProperty(String),
|
UnknownProperty(String),
|
||||||
BadDate(String),
|
BadDate(String),
|
||||||
|
BadRegex(String),
|
||||||
UnsupportedOp {
|
UnsupportedOp {
|
||||||
key: String,
|
key: String,
|
||||||
op: Op,
|
op: Op,
|
||||||
|
|
@ -47,6 +48,7 @@ impl std::fmt::Display for TranslateError {
|
||||||
TranslateError::Parse(e) => write!(f, "{}", e),
|
TranslateError::Parse(e) => write!(f, "{}", e),
|
||||||
TranslateError::UnknownProperty(k) => write!(f, "unknown property '{}'", k),
|
TranslateError::UnknownProperty(k) => write!(f, "unknown property '{}'", k),
|
||||||
TranslateError::BadDate(s) => write!(f, "bad date '{}'", s),
|
TranslateError::BadDate(s) => write!(f, "bad date '{}'", s),
|
||||||
|
TranslateError::BadRegex(s) => write!(f, "regex error: {}", s),
|
||||||
TranslateError::UnsupportedOp { key, op } => {
|
TranslateError::UnsupportedOp { key, op } => {
|
||||||
write!(f, "operator {:?} is not supported for property '{}'", op, key)
|
write!(f, "operator {:?} is not supported for property '{}'", op, key)
|
||||||
}
|
}
|
||||||
|
|
@ -172,8 +174,10 @@ fn fts_expr(parts: &[FtsFragment]) -> String {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Escape a phrase for FTS5 MATCH. FTS5 itself uses doubled quotes for
|
/// Escape a phrase for FTS5 MATCH. FTS5 itself uses doubled quotes for
|
||||||
/// literal quotes inside a quoted phrase.
|
/// literal quotes inside a quoted phrase; wrapping in quotes renders all
|
||||||
fn quote_phrase(s: &str) -> String {
|
/// other MATCH metacharacters (`( ) * :` etc.) inert. Injection-safe by
|
||||||
|
/// construction.
|
||||||
|
pub fn quote_phrase(s: &str) -> String {
|
||||||
let mut buf = String::with_capacity(s.len() + 2);
|
let mut buf = String::with_capacity(s.len() + 2);
|
||||||
buf.push('"');
|
buf.push('"');
|
||||||
for c in s.chars() {
|
for c in s.chars() {
|
||||||
|
|
@ -243,30 +247,86 @@ impl Builder {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Delegate to the shared [`build_filter`] fragment builder, then
|
||||||
|
/// convert its anonymous `?` placeholders to this builder's numbered
|
||||||
|
/// scheme (params[0] is reserved for the FTS MATCH when one exists;
|
||||||
|
/// `build` shifts numbers afterwards).
|
||||||
fn translate_property(
|
fn translate_property(
|
||||||
&mut self,
|
&mut self,
|
||||||
key: &str,
|
key: &str,
|
||||||
op: Op,
|
op: Op,
|
||||||
value: &str,
|
value: &str,
|
||||||
) -> Result<String, TranslateError> {
|
) -> Result<String, TranslateError> {
|
||||||
let lower_key = key.to_ascii_lowercase();
|
let frag = build_filter(key, op, value, false)?;
|
||||||
match lower_key.as_str() {
|
let mut params = frag.params.into_iter();
|
||||||
"type" => self.prop_type(op, value, key),
|
let mut out = String::with_capacity(frag.sql.len() + 8);
|
||||||
"modified" | "mtime" => self.prop_mtime(op, value, key),
|
for c in frag.sql.chars() {
|
||||||
"path" | "folder" | "includefolder" => self.prop_path(op, value, key),
|
if c == '?' {
|
||||||
"name" | "filename" => self.prop_name(op, value, key),
|
let v = params
|
||||||
"mime" => self.prop_mime(op, value, key),
|
.next()
|
||||||
_ => Err(TranslateError::UnknownProperty(key.to_string())),
|
.expect("FilterFragment placeholder/param counts match");
|
||||||
|
self.all_params.push(v);
|
||||||
|
out.push('?');
|
||||||
|
out.push_str(&self.all_params.len().to_string());
|
||||||
|
} else {
|
||||||
|
out.push(c);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(out)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn prop_type(&mut self, op: Op, value: &str, key: &str) -> Result<String, TranslateError> {
|
/// A structured-filter fragment over table alias `f`: SQL with anonymous
|
||||||
|
/// `?` placeholders plus the values they bind. Anonymous placeholders
|
||||||
|
/// compose by simple appending — the search cascade tacks fragments onto
|
||||||
|
/// every stage's WHERE clause with `AND (...)`.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct FilterFragment {
|
||||||
|
pub sql: String,
|
||||||
|
pub params: Vec<rusqlite::types::Value>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether `key` is a recognized structured-filter property.
|
||||||
|
pub fn is_filter_key(key: &str) -> bool {
|
||||||
|
matches!(
|
||||||
|
key.to_ascii_lowercase().as_str(),
|
||||||
|
"type" | "modified" | "mtime" | "path" | "folder" | "includefolder" | "name"
|
||||||
|
| "filename" | "mime"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Translate one `key op value` filter into a [`FilterFragment`]. The
|
||||||
|
/// single source of filter semantics, shared by the legacy numbered
|
||||||
|
/// [`build`] path and the cascade's [`super::split`].
|
||||||
|
///
|
||||||
|
/// `glob` marks a value whose unquoted `*` should act as a wildcard — only
|
||||||
|
/// `name:`/`filename:` honor it; every other key treats the star literally.
|
||||||
|
/// The caller decides, because only the tokenizer knows whether the value
|
||||||
|
/// was quoted (quoted stars are always literal).
|
||||||
|
pub fn build_filter(
|
||||||
|
key: &str,
|
||||||
|
op: Op,
|
||||||
|
value: &str,
|
||||||
|
glob: bool,
|
||||||
|
) -> Result<FilterFragment, TranslateError> {
|
||||||
|
use rusqlite::types::Value;
|
||||||
|
let frag = |sql: &str, params: Vec<Value>| FilterFragment {
|
||||||
|
sql: sql.to_string(),
|
||||||
|
params,
|
||||||
|
};
|
||||||
|
let eq_like_only = |op: Op| -> Result<(), TranslateError> {
|
||||||
if op != Op::Contains && op != Op::Eq {
|
if op != Op::Contains && op != Op::Eq {
|
||||||
return Err(TranslateError::UnsupportedOp {
|
return Err(TranslateError::UnsupportedOp {
|
||||||
key: key.into(),
|
key: key.into(),
|
||||||
op,
|
op,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
Ok(())
|
||||||
|
};
|
||||||
|
|
||||||
|
match key.to_ascii_lowercase().as_str() {
|
||||||
|
"type" => {
|
||||||
|
eq_like_only(op)?;
|
||||||
let bits = FileType::from_name(value).bits() as i64;
|
let bits = FileType::from_name(value).bits() as i64;
|
||||||
if bits == 0 {
|
if bits == 0 {
|
||||||
return Err(TranslateError::UnknownProperty(format!(
|
return Err(TranslateError::UnknownProperty(format!(
|
||||||
|
|
@ -274,88 +334,127 @@ impl Builder {
|
||||||
value
|
value
|
||||||
)));
|
)));
|
||||||
}
|
}
|
||||||
self.all_params
|
Ok(frag("(f.type & ?) != 0", vec![Value::Integer(bits)]))
|
||||||
.push(rusqlite::types::Value::Integer(bits));
|
}
|
||||||
Ok(format!("(f.type & ?{}) != 0", self.param_placeholder_idx()))
|
"modified" | "mtime" => {
|
||||||
|
let unix =
|
||||||
|
parse_date_to_unix(value).ok_or_else(|| TranslateError::BadDate(value.into()))?;
|
||||||
|
// `modified:=2024-01-01` matches the whole day, not the second.
|
||||||
|
if op == Op::Eq || op == Op::Contains {
|
||||||
|
return Ok(frag(
|
||||||
|
"(f.mtime >= ? AND f.mtime < ?)",
|
||||||
|
vec![Value::Integer(unix), Value::Integer(unix + 86_400)],
|
||||||
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
fn prop_mtime(&mut self, op: Op, value: &str, key: &str) -> Result<String, TranslateError> {
|
|
||||||
let unix = parse_date_to_unix(value).ok_or_else(|| TranslateError::BadDate(value.into()))?;
|
|
||||||
let col = "f.mtime";
|
|
||||||
let sql_op = match op {
|
let sql_op = match op {
|
||||||
Op::Contains | Op::Eq => "=",
|
|
||||||
Op::Lt => "<",
|
Op::Lt => "<",
|
||||||
Op::Le => "<=",
|
Op::Le => "<=",
|
||||||
Op::Gt => ">",
|
Op::Gt => ">",
|
||||||
Op::Ge => ">=",
|
Op::Ge => ">=",
|
||||||
|
Op::Contains | Op::Eq => unreachable!(),
|
||||||
};
|
};
|
||||||
// `modified:=2024-01-01` should match the whole day, not the second.
|
Ok(frag(
|
||||||
if op == Op::Eq || op == Op::Contains {
|
&format!("f.mtime {} ?", sql_op),
|
||||||
let start = unix;
|
vec![Value::Integer(unix)],
|
||||||
let end = unix + 86_400;
|
))
|
||||||
self.all_params.push(rusqlite::types::Value::Integer(start));
|
|
||||||
let i = self.param_placeholder_idx();
|
|
||||||
self.all_params.push(rusqlite::types::Value::Integer(end));
|
|
||||||
let j = self.param_placeholder_idx();
|
|
||||||
return Ok(format!("({} >= ?{} AND {} < ?{})", col, i, col, j));
|
|
||||||
}
|
}
|
||||||
self.all_params.push(rusqlite::types::Value::Integer(unix));
|
"path" | "folder" | "includefolder" => {
|
||||||
let i = self.param_placeholder_idx();
|
eq_like_only(op)?;
|
||||||
let _ = key;
|
let base = normalize_folder_value(value);
|
||||||
Ok(format!("{} {} ?{}", col, sql_op, i))
|
if base.is_empty() {
|
||||||
|
// "everything". On Unix the old `parent = '/' OR parent LIKE
|
||||||
|
// '/%'` happened to match every absolute path; Windows has no
|
||||||
|
// single root, so say it directly rather than by accident.
|
||||||
|
return Ok(frag("1=1", Vec::new()));
|
||||||
}
|
}
|
||||||
|
// The `=` half needs the collation spelled out: `LIKE` folds ASCII
|
||||||
fn prop_path(&mut self, op: Op, value: &str, key: &str) -> Result<String, TranslateError> {
|
// case on its own, so without this the two halves of the same
|
||||||
if op != Op::Contains && op != Op::Eq {
|
// filter disagree about `C:\Users` versus `c:\users`.
|
||||||
return Err(TranslateError::UnsupportedOp {
|
Ok(frag(
|
||||||
key: key.into(),
|
&format!(
|
||||||
op,
|
"(f.parent = ? COLLATE {} OR f.parent LIKE ? ESCAPE '\\')",
|
||||||
});
|
crate::platform::PATH_COLLATION
|
||||||
|
),
|
||||||
|
vec![
|
||||||
|
Value::Text(base.clone()),
|
||||||
|
Value::Text(like_subtree_pattern(&base)),
|
||||||
|
],
|
||||||
|
))
|
||||||
}
|
}
|
||||||
self.all_params
|
"name" | "filename" => {
|
||||||
.push(rusqlite::types::Value::Text(value.into()));
|
|
||||||
let i = self.param_placeholder_idx();
|
|
||||||
self.all_params
|
|
||||||
.push(rusqlite::types::Value::Text(format!("{}/%", value.trim_end_matches('/'))));
|
|
||||||
let j = self.param_placeholder_idx();
|
|
||||||
Ok(format!("(f.parent = ?{} OR f.parent LIKE ?{})", i, j))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn prop_name(&mut self, op: Op, value: &str, key: &str) -> Result<String, TranslateError> {
|
|
||||||
if op != Op::Contains {
|
if op != Op::Contains {
|
||||||
return Err(TranslateError::UnsupportedOp {
|
return Err(TranslateError::UnsupportedOp {
|
||||||
key: key.into(),
|
key: key.into(),
|
||||||
op,
|
op,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
self.all_params
|
// With `glob`, each `*` becomes an unescaped `%`; the pieces
|
||||||
.push(rusqlite::types::Value::Text(format!("%{}%", value)));
|
// around it still get `%`/`_`/`\` escaped so user metacharacters
|
||||||
let i = self.param_placeholder_idx();
|
// stay literal either way.
|
||||||
Ok(format!("f.name LIKE ?{}", i))
|
let pattern = if glob && value.contains('*') {
|
||||||
|
value
|
||||||
|
.split('*')
|
||||||
|
.map(escape_like)
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join("%")
|
||||||
|
} else {
|
||||||
|
escape_like(value)
|
||||||
|
};
|
||||||
|
Ok(frag(
|
||||||
|
"f.name LIKE ? ESCAPE '\\'",
|
||||||
|
vec![Value::Text(format!("%{}%", pattern))],
|
||||||
|
))
|
||||||
|
}
|
||||||
|
"mime" => {
|
||||||
|
eq_like_only(op)?;
|
||||||
|
Ok(frag("f.mime = ?", vec![Value::Text(value.into())]))
|
||||||
|
}
|
||||||
|
_ => Err(TranslateError::UnknownProperty(key.to_string())),
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn prop_mime(&mut self, op: Op, value: &str, key: &str) -> Result<String, TranslateError> {
|
/// Escape `%`, `_` and `\` for use inside a `LIKE ... ESCAPE '\'` pattern.
|
||||||
if op != Op::Contains && op != Op::Eq {
|
pub fn escape_like(s: &str) -> String {
|
||||||
return Err(TranslateError::UnsupportedOp {
|
let mut out = String::with_capacity(s.len());
|
||||||
key: key.into(),
|
for c in s.chars() {
|
||||||
op,
|
if matches!(c, '%' | '_' | '\\') {
|
||||||
});
|
out.push('\\');
|
||||||
}
|
}
|
||||||
self.all_params
|
out.push(c);
|
||||||
.push(rusqlite::types::Value::Text(value.into()));
|
}
|
||||||
let i = self.param_placeholder_idx();
|
out
|
||||||
Ok(format!("f.mime = ?{}", i))
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn param_placeholder_idx(&self) -> usize {
|
/// Tidy a user-supplied folder value into the spelling `files.parent` stores.
|
||||||
// params[0] is reserved for the FTS MATCH if one is built; structured
|
///
|
||||||
// params start at index 2 in that case (1-based). We track it by
|
/// Trailing separators are how people naturally write directories, and either
|
||||||
// calling this *after* pushing the value; result is `len` so the SQL
|
/// separator may show up on Windows. A bare drive (`C:`) is *not* a path — the
|
||||||
// says `?<len>` which matches the 1-based positional binding rusqlite
|
/// stored parent is `C:\` — so the separator goes back on.
|
||||||
// uses for `?N` placeholders. When an FTS match is prepended at
|
fn normalize_folder_value(value: &str) -> String {
|
||||||
// `build`, each index shifts by 1 implicitly.
|
let base = value.trim().trim_end_matches(['/', '\\']);
|
||||||
self.all_params.len()
|
if base.len() == 2 && base.ends_with(':') && base.starts_with(|c: char| c.is_ascii_alphabetic())
|
||||||
|
{
|
||||||
|
return format!("{}{}", base, std::path::MAIN_SEPARATOR);
|
||||||
}
|
}
|
||||||
|
base.to_string()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A `LIKE ... ESCAPE '\'` pattern matching every path strictly beneath `dir`.
|
||||||
|
///
|
||||||
|
/// The separator is escaped along with the base, because on Windows the
|
||||||
|
/// separator *is* the escape character — a hand-written `format!("{}/%", dir)`
|
||||||
|
/// is wrong twice over there: wrong separator, and the one it emits would be
|
||||||
|
/// swallowed as an escape.
|
||||||
|
///
|
||||||
|
/// SQLite's `patternCompare` takes the character after the escape literally
|
||||||
|
/// whatever it is, so a doubled `\` is well defined here; the folklore that an
|
||||||
|
/// escape must be followed by `%`, `_` or itself does not apply.
|
||||||
|
pub fn like_subtree_pattern(dir: &str) -> String {
|
||||||
|
format!(
|
||||||
|
"{}{}%",
|
||||||
|
escape_like(dir.trim_end_matches(['/', '\\'])),
|
||||||
|
escape_like(std::path::MAIN_SEPARATOR_STR)
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn join_with(sep: &str, pieces: &[String]) -> String {
|
fn join_with(sep: &str, pieces: &[String]) -> String {
|
||||||
|
|
@ -497,6 +596,101 @@ mod tests {
|
||||||
let q = build_q("path:/home/me/docs");
|
let q = build_q("path:/home/me/docs");
|
||||||
assert!(q.sql.contains("f.parent = ?"));
|
assert!(q.sql.contains("f.parent = ?"));
|
||||||
assert!(q.sql.contains("f.parent LIKE ?"));
|
assert!(q.sql.contains("f.parent LIKE ?"));
|
||||||
|
// The LIKE half must be escaped and declare its escape character;
|
||||||
|
// without the clause a Windows separator would be eaten as an escape.
|
||||||
|
assert!(q.sql.contains("ESCAPE '\\'"), "{}", q.sql);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The subtree pattern is the one place the separator and the LIKE escape
|
||||||
|
/// character collide (on Windows they are the same byte), so it is checked
|
||||||
|
/// against real SQLite rather than by string comparison.
|
||||||
|
#[test]
|
||||||
|
fn like_subtree_pattern_matches_only_the_subtree() {
|
||||||
|
use std::path::MAIN_SEPARATOR as SEP;
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open_in_memory().unwrap();
|
||||||
|
conn.execute("CREATE TABLE files (parent TEXT NOT NULL)", [])
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let base = format!("{}a{}b", root_prefix(), SEP);
|
||||||
|
let rows = [
|
||||||
|
format!("{}{}sub", base, SEP), // inside
|
||||||
|
format!("{}{}sub{}deep", base, SEP, SEP), // deeper
|
||||||
|
base.clone(), // the folder itself
|
||||||
|
format!("{}a{}bc", root_prefix(), SEP), // prefix sibling: outside
|
||||||
|
format!("{}a", root_prefix()), // parent: outside
|
||||||
|
];
|
||||||
|
for r in &rows {
|
||||||
|
conn.execute("INSERT INTO files (parent) VALUES (?1)", [r])
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
let matched: i64 = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM files WHERE parent LIKE ?1 ESCAPE '\\'",
|
||||||
|
[like_subtree_pattern(&base)],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(matched, 2, "only the two rows strictly beneath {}", base);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn like_subtree_pattern_escapes_metacharacters() {
|
||||||
|
use std::path::MAIN_SEPARATOR as SEP;
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open_in_memory().unwrap();
|
||||||
|
conn.execute("CREATE TABLE files (parent TEXT NOT NULL)", [])
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let base = format!("{}a_b", root_prefix());
|
||||||
|
for r in [
|
||||||
|
format!("{}{}inside", base, SEP), // real child
|
||||||
|
format!("{}axb{}bait", root_prefix(), SEP), // `_` must not glob to `x`
|
||||||
|
format!("{}100%_done{}x", root_prefix(), SEP),
|
||||||
|
] {
|
||||||
|
conn.execute("INSERT INTO files (parent) VALUES (?1)", [&r])
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
let matched: i64 = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM files WHERE parent LIKE ?1 ESCAPE '\\'",
|
||||||
|
[like_subtree_pattern(&base)],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(matched, 1, "`_` and `%` are literals, not wildcards");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn folder_value_normalization() {
|
||||||
|
use std::path::MAIN_SEPARATOR as SEP;
|
||||||
|
// Trailing separators of either flavour are stripped.
|
||||||
|
assert_eq!(normalize_folder_value("/home/me/"), "/home/me");
|
||||||
|
assert_eq!(normalize_folder_value(r"C:\Users\me\"), r"C:\Users\me");
|
||||||
|
// A bare drive is not a path; the stored parent is `C:\`.
|
||||||
|
assert_eq!(normalize_folder_value("C:"), format!("C:{}", SEP));
|
||||||
|
assert_eq!(normalize_folder_value(r"C:\"), format!("C:{}", SEP));
|
||||||
|
// Empty means "everywhere".
|
||||||
|
assert_eq!(normalize_folder_value("/"), "");
|
||||||
|
assert_eq!(normalize_folder_value(" "), "");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn empty_folder_value_matches_everything() {
|
||||||
|
let frag = build_filter("path", Op::Contains, "/", false).unwrap();
|
||||||
|
assert_eq!(frag.sql, "1=1");
|
||||||
|
assert!(frag.params.is_empty(), "no placeholders to renumber");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An absolute-path prefix for the running platform.
|
||||||
|
fn root_prefix() -> String {
|
||||||
|
if cfg!(windows) {
|
||||||
|
r"C:\".to_string()
|
||||||
|
} else {
|
||||||
|
"/".to_string()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
937
crates/quicksearch-core/src/search/cascade.rs
Normal file
|
|
@ -0,0 +1,937 @@
|
||||||
|
//! The ranked search cascade.
|
||||||
|
//!
|
||||||
|
//! One term, four table scans, eleven ranks. Rank base = stage number, so
|
||||||
|
//! later stages only ever append to a rank-sorted result list:
|
||||||
|
//!
|
||||||
|
//! | rank | meaning | scan |
|
||||||
|
//! |-----:|----------------------------------|------|
|
||||||
|
//! | 1.x | exact filename, exact case | A |
|
||||||
|
//! | 2.x | exact filename, any case | A |
|
||||||
|
//! | 3.x | filename substring, exact case | A |
|
||||||
|
//! | 4.x | filename substring, any case | A |
|
||||||
|
//! | 5.x | full text occurrence, exact case | B |
|
||||||
|
//! | 6.x | full text occurrence, any case | B |
|
||||||
|
//! | 7.x | fuzzy filename | C |
|
||||||
|
//! | 8.x | fuzzy full text | D |
|
||||||
|
//! | 9.x | full path substring, exact case | A |
|
||||||
|
//! | 10.x | full path substring, any case | A |
|
||||||
|
//! | 11.x | fuzzy full path | C |
|
||||||
|
//!
|
||||||
|
//! Pass A is a single `files` scan (`LIKE`, the ASCII-nocase superset of
|
||||||
|
//! its ranks) classified per-row in Rust — no index needed, the substring
|
||||||
|
//! stage visits every row anyway. Because a path always ends in its own
|
||||||
|
//! name, `path LIKE` is a superset of `name LIKE`, so that one scan covers
|
||||||
|
//! the filename *and* the path tiers. Pass B is one FTS phrase MATCH
|
||||||
|
//! verified against the decompressed text. Passes C/D (opt-in) iterate the
|
||||||
|
//! whole table with a bitap matcher, C covering both the name and the path.
|
||||||
|
//!
|
||||||
|
//! Wildcard terms (`rep*rt`) rank through the same tiers, with 1/2 meaning
|
||||||
|
//! the whole name matches the pattern; they skip the fuzzy passes (bitap is
|
||||||
|
//! a literal matcher). A regex-only query (`regex:…` with no term) runs two
|
||||||
|
//! dedicated scans that reuse tiers 4 (name), 6 (content) and 10 (path), so
|
||||||
|
//! downstream stage handling is unchanged. When `regex:` accompanies a
|
||||||
|
//! term, it is an accept-predicate on every pass, not a rank source.
|
||||||
|
//!
|
||||||
|
//! The path tiers rank below everything else, so pass A and pass C buffer
|
||||||
|
//! them instead of emitting them — stages E and F flush those buffers at
|
||||||
|
//! the end, dropping files an earlier stage already emitted. Path matching
|
||||||
|
//! needs a term of at least three characters, the same floor pass B has.
|
||||||
|
//!
|
||||||
|
//! Full-text ranks order equal-based hits by occurrence count via a
|
||||||
|
//! decimal fraction: `base + (1000 - min(count, 1000)) / 1000` — more
|
||||||
|
//! occurrences sorts earlier, 1000+ occurrences adds zero. Fuzzy ranks add
|
||||||
|
//! `0.1 × edit_distance` instead.
|
||||||
|
//!
|
||||||
|
//! Every scan appends the caller's structured-filter SQL (anonymous
|
||||||
|
//! placeholders over alias `f`) and checks the generation counter as it
|
||||||
|
//! streams; a bumped generation aborts mid-statement.
|
||||||
|
|
||||||
|
use std::collections::HashSet;
|
||||||
|
use std::sync::atomic::{AtomicU64, Ordering};
|
||||||
|
|
||||||
|
use rusqlite::Connection;
|
||||||
|
|
||||||
|
use rusqlite::OptionalExtension;
|
||||||
|
|
||||||
|
use crate::config::IgnoreSet;
|
||||||
|
use crate::query::pattern::clamp_match_range;
|
||||||
|
use crate::query::split::CascadeQuery;
|
||||||
|
use crate::query::translator::{escape_like, quote_phrase};
|
||||||
|
use crate::snippet;
|
||||||
|
|
||||||
|
use super::fuzzy::{edit_budget, Bitap};
|
||||||
|
use super::{SearchHit, SearchOptions};
|
||||||
|
|
||||||
|
/// Cancellation is checked every this many scanned rows in row-cheap
|
||||||
|
/// passes; decompression-heavy passes check every row.
|
||||||
|
const CANCEL_CHECK_ROWS: usize = 256;
|
||||||
|
|
||||||
|
/// Snippet window budget. Generous on purpose: the GUI trims the cell
|
||||||
|
/// text down to its column width around the match, and the mouseover
|
||||||
|
/// shows the rest of this window as extended context.
|
||||||
|
const SNIPPET_WINDOW_CHARS: usize = 600;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub struct Outcome {
|
||||||
|
pub total: usize,
|
||||||
|
pub limited: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Run the cascade, streaming rank-ordered batches into `sink`.
|
||||||
|
/// `Ok(None)` means the search was cancelled (generation moved on) — the
|
||||||
|
/// caller sends no completion. SQL errors are returned as strings *unless*
|
||||||
|
/// the search was already cancelled (an interrupted statement is normal
|
||||||
|
/// cancellation, not an error).
|
||||||
|
pub fn run(
|
||||||
|
conn: &Connection,
|
||||||
|
query: &CascadeQuery,
|
||||||
|
options: &SearchOptions,
|
||||||
|
generation: u64,
|
||||||
|
latest_gen: &AtomicU64,
|
||||||
|
sink: &mut dyn FnMut(Vec<SearchHit>),
|
||||||
|
) -> Result<Option<Outcome>, String> {
|
||||||
|
if query.is_empty() {
|
||||||
|
return Ok(Some(Outcome {
|
||||||
|
total: 0,
|
||||||
|
limited: false,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
let ignore = IgnoreSet::compile(&options.session_ignores)
|
||||||
|
.map_err(|e| format!("session ignore filter: {}", e))?;
|
||||||
|
|
||||||
|
let mut cx = Cx {
|
||||||
|
conn,
|
||||||
|
query,
|
||||||
|
options,
|
||||||
|
generation,
|
||||||
|
latest_gen,
|
||||||
|
ignore,
|
||||||
|
emitted: HashSet::new(),
|
||||||
|
deferred_path: Deferred::default(),
|
||||||
|
deferred_fuzzy_path: Deferred::default(),
|
||||||
|
total: 0,
|
||||||
|
limited: false,
|
||||||
|
sink,
|
||||||
|
};
|
||||||
|
|
||||||
|
// With no term at all the regex drives its own scans; `Path` still
|
||||||
|
// flushes the deferred rank-10 buffer the name pass sets aside.
|
||||||
|
let passes: &[Pass] = if query.pattern.is_empty() {
|
||||||
|
&[Pass::RegexName, Pass::RegexContent, Pass::Path]
|
||||||
|
} else {
|
||||||
|
&[
|
||||||
|
Pass::Filename,
|
||||||
|
Pass::FullText,
|
||||||
|
Pass::FuzzyFilename,
|
||||||
|
Pass::FuzzyFullText,
|
||||||
|
Pass::Path,
|
||||||
|
Pass::FuzzyPath,
|
||||||
|
]
|
||||||
|
};
|
||||||
|
for pass in passes {
|
||||||
|
if cx.cancelled() {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
if cx.remaining() == 0 {
|
||||||
|
cx.limited = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
let run_pass = match pass {
|
||||||
|
Pass::Filename => cx.pass_filename(),
|
||||||
|
Pass::FullText => cx.pass_fulltext(),
|
||||||
|
Pass::FuzzyFilename => cx.pass_fuzzy_filename(),
|
||||||
|
Pass::FuzzyFullText => cx.pass_fuzzy_fulltext(),
|
||||||
|
Pass::RegexName => cx.pass_regex_name(),
|
||||||
|
Pass::RegexContent => cx.pass_regex_content(),
|
||||||
|
Pass::Path => {
|
||||||
|
let d = std::mem::take(&mut cx.deferred_path);
|
||||||
|
cx.flush_deferred(d)
|
||||||
|
}
|
||||||
|
Pass::FuzzyPath => {
|
||||||
|
let d = std::mem::take(&mut cx.deferred_fuzzy_path);
|
||||||
|
cx.flush_deferred(d)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
match run_pass {
|
||||||
|
Ok(true) => {}
|
||||||
|
Ok(false) => return Ok(None), // cancelled mid-pass
|
||||||
|
Err(e) => {
|
||||||
|
if cx.cancelled() {
|
||||||
|
return Ok(None); // interrupt() killed the statement
|
||||||
|
}
|
||||||
|
return Err(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(Some(Outcome {
|
||||||
|
total: cx.total,
|
||||||
|
limited: cx.limited,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
enum Pass {
|
||||||
|
Filename,
|
||||||
|
FullText,
|
||||||
|
FuzzyFilename,
|
||||||
|
FuzzyFullText,
|
||||||
|
/// Regex-only: name hits at rank 4 now, path hits deferred to rank 10.
|
||||||
|
RegexName,
|
||||||
|
/// Regex-only: content hits at rank 6.
|
||||||
|
RegexContent,
|
||||||
|
/// Flush of the rank 9–10 hits pass A set aside.
|
||||||
|
Path,
|
||||||
|
/// Flush of the rank 11 hits pass C set aside.
|
||||||
|
FuzzyPath,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Occurrence-count fraction: more occurrences → smaller fraction → sorts
|
||||||
|
/// earlier within a rank base; 1000+ adds zero.
|
||||||
|
fn count_frac(count: usize) -> f64 {
|
||||||
|
(1000usize.saturating_sub(count.min(1000))) as f64 / 1000.0
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The path tiers only make sense with enough term to be specific — same
|
||||||
|
/// floor the trigram full-text pass uses. Wildcards count only their
|
||||||
|
/// literal content (`a*b` is two characters of specificity, not three).
|
||||||
|
fn path_tiers_enabled(pattern: &crate::query::pattern::TermPattern) -> bool {
|
||||||
|
pattern.literal_char_count() >= 3
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Hits collected by one scan but ranked below later scans, so held back
|
||||||
|
/// until every better stage has emitted.
|
||||||
|
#[derive(Default)]
|
||||||
|
struct Deferred {
|
||||||
|
hits: Vec<SearchHit>,
|
||||||
|
overflowed: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Cx<'a> {
|
||||||
|
conn: &'a Connection,
|
||||||
|
query: &'a CascadeQuery,
|
||||||
|
options: &'a SearchOptions,
|
||||||
|
generation: u64,
|
||||||
|
latest_gen: &'a AtomicU64,
|
||||||
|
ignore: IgnoreSet,
|
||||||
|
emitted: HashSet<i64>,
|
||||||
|
/// Ranks 9–10, filled by pass A.
|
||||||
|
deferred_path: Deferred,
|
||||||
|
/// Rank 11, filled by pass C.
|
||||||
|
deferred_fuzzy_path: Deferred,
|
||||||
|
total: usize,
|
||||||
|
limited: bool,
|
||||||
|
sink: &'a mut dyn FnMut(Vec<SearchHit>),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<'a> Cx<'a> {
|
||||||
|
fn cancelled(&self) -> bool {
|
||||||
|
self.generation != self.latest_gen.load(Ordering::Relaxed)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn remaining(&self) -> usize {
|
||||||
|
self.options.limit.saturating_sub(self.total)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Buffer cap for scan passes: enough headroom that sorting keeps the
|
||||||
|
/// best candidates, without unbounded growth on huge hit sets.
|
||||||
|
fn buffer_cap(&self) -> usize {
|
||||||
|
4096.max(2 * self.remaining())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn params_with_filters(
|
||||||
|
&self,
|
||||||
|
leading: Vec<rusqlite::types::Value>,
|
||||||
|
) -> Vec<rusqlite::types::Value> {
|
||||||
|
let mut p = leading;
|
||||||
|
p.extend(self.query.filter_params.iter().cloned());
|
||||||
|
p
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Skip rows already emitted at a better rank or hidden by session
|
||||||
|
/// ignore chips.
|
||||||
|
fn skip(&self, file_id: i64, path: &str) -> bool {
|
||||||
|
self.emitted.contains(&file_id) || self.ignore.matches_path(std::path::Path::new(path))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The `regex:` accept-predicate applied to every candidate row when a
|
||||||
|
/// regex accompanies a term. The path contains the name, so one path
|
||||||
|
/// check covers both; content is fetched (and decompressed) only for
|
||||||
|
/// rows whose path missed — bounded by the pass's hit count, not its
|
||||||
|
/// scan count. Pass `text` when the pass already has the content.
|
||||||
|
fn regex_accepts(
|
||||||
|
&self,
|
||||||
|
file_id: i64,
|
||||||
|
path: &str,
|
||||||
|
text: Option<&str>,
|
||||||
|
) -> Result<bool, String> {
|
||||||
|
let Some(re) = &self.query.regex else {
|
||||||
|
return Ok(true);
|
||||||
|
};
|
||||||
|
if re.is_match(path) {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
if let Some(text) = text {
|
||||||
|
return Ok(re.is_match(text));
|
||||||
|
}
|
||||||
|
let blob: Option<Vec<u8>> = self
|
||||||
|
.conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT text_zstd FROM documents_text WHERE file_id = ?1",
|
||||||
|
[file_id],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.optional()
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
let Some(raw) = blob.and_then(|b| zstd::decode_all(b.as_slice()).ok()) else {
|
||||||
|
return Ok(false);
|
||||||
|
};
|
||||||
|
Ok(re.is_match(&String::from_utf8_lossy(&raw)))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Sort a finished pass buffer, truncate to what's left of the display
|
||||||
|
/// limit, and stream it out in `options.batch`-sized events.
|
||||||
|
fn flush_pass(&mut self, mut buf: Vec<SearchHit>, overflowed: bool) {
|
||||||
|
buf.sort_by(|a, b| {
|
||||||
|
a.rank
|
||||||
|
.partial_cmp(&b.rank)
|
||||||
|
.unwrap_or(std::cmp::Ordering::Equal)
|
||||||
|
.then_with(|| a.name.cmp(&b.name))
|
||||||
|
.then_with(|| a.path.cmp(&b.path))
|
||||||
|
});
|
||||||
|
let room = self.remaining();
|
||||||
|
if buf.len() > room {
|
||||||
|
buf.truncate(room);
|
||||||
|
self.limited = true;
|
||||||
|
}
|
||||||
|
if overflowed {
|
||||||
|
self.limited = true;
|
||||||
|
}
|
||||||
|
self.total += buf.len();
|
||||||
|
for hit in &buf {
|
||||||
|
self.emitted.insert(hit.file_id);
|
||||||
|
}
|
||||||
|
let batch = self.options.batch.max(1);
|
||||||
|
let mut buf = buf.into_iter().peekable();
|
||||||
|
while buf.peek().is_some() {
|
||||||
|
// A cancelled search stops emitting immediately — the newer
|
||||||
|
// generation owns the UI.
|
||||||
|
if self.cancelled() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let chunk: Vec<SearchHit> = buf.by_ref().take(batch).collect();
|
||||||
|
(self.sink)(chunk);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Emit a buffer held back from an earlier scan. Anything a better
|
||||||
|
/// stage already emitted drops out here — `emitted` was still empty (or
|
||||||
|
/// smaller) when these hits were collected.
|
||||||
|
fn flush_deferred(&mut self, mut deferred: Deferred) -> Result<bool, String> {
|
||||||
|
deferred.hits.retain(|h| !self.emitted.contains(&h.file_id));
|
||||||
|
self.flush_pass(deferred.hits, deferred.overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Keep a scan buffer bounded: sort + cut back to the display-limit
|
||||||
|
/// room once it doubles past it. Returns whether anything was dropped.
|
||||||
|
fn enforce_cap(&self, buf: &mut Vec<SearchHit>) -> bool {
|
||||||
|
if buf.len() <= self.buffer_cap() {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
buf.sort_by(|a, b| {
|
||||||
|
a.rank
|
||||||
|
.partial_cmp(&b.rank)
|
||||||
|
.unwrap_or(std::cmp::Ordering::Equal)
|
||||||
|
.then_with(|| a.name.cmp(&b.name))
|
||||||
|
.then_with(|| a.path.cmp(&b.path))
|
||||||
|
});
|
||||||
|
buf.truncate(self.remaining());
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pass A — ranks 1–4 now, ranks 9–10 deferred, from one `files` scan.
|
||||||
|
/// Returns Ok(false) on cancellation.
|
||||||
|
fn pass_filename(&mut self) -> Result<bool, String> {
|
||||||
|
let query = self.query;
|
||||||
|
let pattern = &query.pattern;
|
||||||
|
let with_paths = path_tiers_enabled(pattern);
|
||||||
|
// A path always ends in its own name, so `path LIKE` is the
|
||||||
|
// superset that feeds both the name and the path tiers.
|
||||||
|
let sql = format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime FROM files f \
|
||||||
|
WHERE {} LIKE ? ESCAPE '\\'{}",
|
||||||
|
if with_paths { "f.path" } else { "f.name" },
|
||||||
|
query.filter_sql
|
||||||
|
);
|
||||||
|
// Wildcard patterns turn each star into an unescaped `%`; the
|
||||||
|
// substring wrap absorbs leading/trailing stars. User `%`/`_`
|
||||||
|
// remain escaped literals either way.
|
||||||
|
let like = pattern
|
||||||
|
.segments()
|
||||||
|
.iter()
|
||||||
|
.map(|s| escape_like(s))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join("%");
|
||||||
|
let params = self.params_with_filters(vec![rusqlite::types::Value::Text(format!(
|
||||||
|
"%{}%",
|
||||||
|
like
|
||||||
|
))]);
|
||||||
|
|
||||||
|
let mut stmt = self.conn.prepare(&sql).map_err(|e| e.to_string())?;
|
||||||
|
let mut rows = stmt
|
||||||
|
.query(rusqlite::params_from_iter(params))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
|
||||||
|
let mut buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut path_buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut overflowed = false;
|
||||||
|
let mut path_overflowed = false;
|
||||||
|
let mut scanned = 0usize;
|
||||||
|
while let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||||
|
scanned += 1;
|
||||||
|
if scanned % CANCEL_CHECK_ROWS == 0 && self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
let file_id: i64 = row.get(0).map_err(|e| e.to_string())?;
|
||||||
|
let name: String = row.get(1).map_err(|e| e.to_string())?;
|
||||||
|
let path: String = row.get(2).map_err(|e| e.to_string())?;
|
||||||
|
if self.skip(file_id, &path) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// For a literal pattern these are exactly the old `==` /
|
||||||
|
// `eq_ignore_ascii_case` / `find` / folded-`find` operations
|
||||||
|
// (folding is byte-length preserving, so folded offsets are
|
||||||
|
// valid in the original). Wildcards run the same ladder through
|
||||||
|
// their compiled matcher — tiers 1/2 mean "whole name matches
|
||||||
|
// the pattern", which is what `*.txt` should do.
|
||||||
|
let (rank, match_range) = if pattern.whole_match(&name, false) {
|
||||||
|
(1.0, (0, name.len()))
|
||||||
|
} else if pattern.whole_match(&name, true) {
|
||||||
|
(2.0, (0, name.len()))
|
||||||
|
} else if let Some(r) = pattern.find_first(&name, false) {
|
||||||
|
(3.0, (r.start, r.end))
|
||||||
|
} else if let Some(r) = pattern.find_first(&name, true) {
|
||||||
|
(4.0, (r.start, r.end))
|
||||||
|
} else if !with_paths {
|
||||||
|
continue;
|
||||||
|
} else if let Some(r) = pattern.find_first(&path, false) {
|
||||||
|
(9.0, (r.start, r.end))
|
||||||
|
} else if let Some(r) = pattern.find_first(&path, true) {
|
||||||
|
(10.0, (r.start, r.end))
|
||||||
|
} else {
|
||||||
|
// LIKE folds ASCII case only; a row that matched it but
|
||||||
|
// neither field is a non-ASCII near-miss. Drop it.
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
if !self.regex_accepts(file_id, &path, None)? {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let is_path_tier = rank >= 9.0;
|
||||||
|
// The "snippet" of a name or path hit is that field itself with
|
||||||
|
// the matched span marked — the GUI renders it as [the field].
|
||||||
|
let snip = snippet::Snippet {
|
||||||
|
ranges: vec![match_range],
|
||||||
|
window: if is_path_tier { path.clone() } else { name.clone() },
|
||||||
|
truncated_start: false,
|
||||||
|
truncated_end: false,
|
||||||
|
};
|
||||||
|
let hit = SearchHit {
|
||||||
|
file_id,
|
||||||
|
name,
|
||||||
|
path,
|
||||||
|
size: row.get::<_, i64>(3).map_err(|e| e.to_string())?.max(0) as u64,
|
||||||
|
mtime: row.get(4).map_err(|e| e.to_string())?,
|
||||||
|
rank,
|
||||||
|
stage: rank as u8,
|
||||||
|
snippet: Some(snip),
|
||||||
|
};
|
||||||
|
if is_path_tier {
|
||||||
|
path_buf.push(hit);
|
||||||
|
path_overflowed |= self.enforce_cap(&mut path_buf);
|
||||||
|
} else {
|
||||||
|
buf.push(hit);
|
||||||
|
overflowed |= self.enforce_cap(&mut buf);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
drop(rows);
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
self.deferred_path = Deferred {
|
||||||
|
hits: path_buf,
|
||||||
|
overflowed: path_overflowed,
|
||||||
|
};
|
||||||
|
self.flush_pass(buf, overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pass B — ranks 5–6 from one FTS MATCH, verified/counted in the
|
||||||
|
/// decompressed text.
|
||||||
|
fn pass_fulltext(&mut self) -> Result<bool, String> {
|
||||||
|
let query = self.query;
|
||||||
|
let pattern = &query.pattern;
|
||||||
|
if pattern.literal_char_count() < 3 {
|
||||||
|
// Below the trigram floor the MATCH can't return anything.
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
// Column filter goes inside the MATCH expression (`text: "..."`)
|
||||||
|
// so only document bodies match — filenames get ranks 1-4 from the
|
||||||
|
// filename pass instead. A literal term is one quoted phrase; a
|
||||||
|
// wildcard term narrows with an AND of its trigram-sized segments.
|
||||||
|
// With no segment of 3+ chars (`ab*cd`) FTS can't narrow at all, so
|
||||||
|
// fall back to scanning every stored document — every row is
|
||||||
|
// pattern-verified either way.
|
||||||
|
let match_expr: Option<String> = match pattern.literal() {
|
||||||
|
Some(term) => Some(format!("text: {}", quote_phrase(term))),
|
||||||
|
None => {
|
||||||
|
let usable: Vec<String> = pattern
|
||||||
|
.segments()
|
||||||
|
.iter()
|
||||||
|
.filter(|s| s.chars().count() >= 3)
|
||||||
|
.map(|s| format!("text: {}", quote_phrase(s)))
|
||||||
|
.collect();
|
||||||
|
if usable.is_empty() {
|
||||||
|
None
|
||||||
|
} else {
|
||||||
|
Some(usable.join(" AND "))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let narrowed = match_expr.is_some();
|
||||||
|
let (sql, params) = match match_expr {
|
||||||
|
Some(expr) => (
|
||||||
|
format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime, dt.text_zstd \
|
||||||
|
FROM searchabletext \
|
||||||
|
JOIN files f ON f.id = searchabletext.rowid \
|
||||||
|
LEFT JOIN documents_text dt ON dt.file_id = f.id \
|
||||||
|
WHERE searchabletext MATCH ?{}",
|
||||||
|
query.filter_sql
|
||||||
|
),
|
||||||
|
self.params_with_filters(vec![rusqlite::types::Value::Text(expr)]),
|
||||||
|
),
|
||||||
|
None => (
|
||||||
|
format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime, dt.text_zstd \
|
||||||
|
FROM documents_text dt \
|
||||||
|
JOIN files f ON f.id = dt.file_id WHERE 1=1{}",
|
||||||
|
query.filter_sql
|
||||||
|
),
|
||||||
|
self.params_with_filters(Vec::new()),
|
||||||
|
),
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut stmt = self.conn.prepare(&sql).map_err(|e| e.to_string())?;
|
||||||
|
let mut rows = stmt
|
||||||
|
.query(rusqlite::params_from_iter(params))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
|
||||||
|
let snippet_opts = snippet::Options { approx_chars: SNIPPET_WINDOW_CHARS };
|
||||||
|
let mut buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut overflowed = false;
|
||||||
|
while let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||||
|
// Decompression dominates: check every row.
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
let file_id: i64 = row.get(0).map_err(|e| e.to_string())?;
|
||||||
|
let path: String = row.get(2).map_err(|e| e.to_string())?;
|
||||||
|
if self.skip(file_id, &path) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let blob: Option<Vec<u8>> = row.get(5).map_err(|e| e.to_string())?;
|
||||||
|
let text = blob.and_then(|b| zstd::decode_all(b.as_slice()).ok()).map(
|
||||||
|
|raw| String::from_utf8_lossy(&raw).into_owned(),
|
||||||
|
);
|
||||||
|
|
||||||
|
let (rank, stage, snip) = match &text {
|
||||||
|
Some(text) => {
|
||||||
|
let (count, stage, ci) = {
|
||||||
|
let count_cs = pattern.count(text, false);
|
||||||
|
if count_cs > 0 {
|
||||||
|
(count_cs, 5, false)
|
||||||
|
} else {
|
||||||
|
let count_ci = pattern.count(text, true);
|
||||||
|
if count_ci > 0 {
|
||||||
|
(count_ci, 6, true)
|
||||||
|
} else {
|
||||||
|
// Folded/unordered FTS candidate: the
|
||||||
|
// pattern never occurs — drop it.
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// Literal terms keep the richer multi-occurrence
|
||||||
|
// extract; a wildcard match marks its own first range.
|
||||||
|
let snip = match pattern.literal() {
|
||||||
|
Some(term) => Some(snippet::extract(text, &[term], &snippet_opts)),
|
||||||
|
None => pattern.find_first(text, ci).map(|r| {
|
||||||
|
let r = clamp_match_range(text, r, SNIPPET_WINDOW_CHARS);
|
||||||
|
snippet::window_around(text, (r.start, r.end), &snippet_opts)
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
(stage as f64 + count_frac(count), stage as u8, snip)
|
||||||
|
}
|
||||||
|
// No stored text (store_text_for_snippets = false or empty
|
||||||
|
// body): can't case-verify or count. On the FTS-narrowed
|
||||||
|
// path accept at the bottom of rank 6 as count-unknown (for
|
||||||
|
// wildcards the AND-of-segments guarantee is weaker —
|
||||||
|
// unordered co-occurrence — accepted for recall). On the
|
||||||
|
// full-scan fallback there is no FTS evidence at all, so an
|
||||||
|
// unverifiable row is just skipped.
|
||||||
|
None => {
|
||||||
|
if !narrowed {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
(6.0 + count_frac(1), 6, None)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if !self.regex_accepts(file_id, &path, text.as_deref())? {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
buf.push(SearchHit {
|
||||||
|
file_id,
|
||||||
|
name: row.get(1).map_err(|e| e.to_string())?,
|
||||||
|
path,
|
||||||
|
size: row.get::<_, i64>(3).map_err(|e| e.to_string())?.max(0) as u64,
|
||||||
|
mtime: row.get(4).map_err(|e| e.to_string())?,
|
||||||
|
rank,
|
||||||
|
stage,
|
||||||
|
snippet: snip,
|
||||||
|
});
|
||||||
|
overflowed |= self.enforce_cap(&mut buf);
|
||||||
|
}
|
||||||
|
drop(rows);
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
self.flush_pass(buf, overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pass C — rank 7 now, rank 11 deferred: one bitap sweep over every
|
||||||
|
/// filename, falling back to the full path where the name misses.
|
||||||
|
fn pass_fuzzy_filename(&mut self) -> Result<bool, String> {
|
||||||
|
if !self.options.fuzzy {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
// Bitap is a literal matcher; wildcard terms don't fuzz.
|
||||||
|
if self.query.pattern.is_wildcard() {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
let folded_term = self.query.term.to_ascii_lowercase();
|
||||||
|
let Some(k) = edit_budget(folded_term.len(), self.options.fuzzy_max_edits) else {
|
||||||
|
return Ok(true);
|
||||||
|
};
|
||||||
|
let Some(bitap) = Bitap::new(folded_term.as_bytes(), k) else {
|
||||||
|
return Ok(true);
|
||||||
|
};
|
||||||
|
let with_paths = path_tiers_enabled(&self.query.pattern);
|
||||||
|
|
||||||
|
let sql = format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime FROM files f WHERE 1=1{}",
|
||||||
|
self.query.filter_sql
|
||||||
|
);
|
||||||
|
let params = self.params_with_filters(Vec::new());
|
||||||
|
let mut stmt = self.conn.prepare(&sql).map_err(|e| e.to_string())?;
|
||||||
|
let mut rows = stmt
|
||||||
|
.query(rusqlite::params_from_iter(params))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
|
||||||
|
let mut buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut path_buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut overflowed = false;
|
||||||
|
let mut path_overflowed = false;
|
||||||
|
let mut scanned = 0usize;
|
||||||
|
while let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||||
|
scanned += 1;
|
||||||
|
if scanned % 1024 == 0 && self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
let file_id: i64 = row.get(0).map_err(|e| e.to_string())?;
|
||||||
|
let name: String = row.get(1).map_err(|e| e.to_string())?;
|
||||||
|
let path: String = row.get(2).map_err(|e| e.to_string())?;
|
||||||
|
if self.skip(file_id, &path) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// The name is the better match when both fire, so it wins and
|
||||||
|
// only a name miss falls through to the path tier.
|
||||||
|
let folded_name = name.to_ascii_lowercase();
|
||||||
|
let (rank, field, folded_field) = match bitap.best_distance(folded_name.as_bytes()) {
|
||||||
|
Some(distance) => (7.0 + 0.1 * distance as f64, &name, folded_name),
|
||||||
|
None if with_paths => {
|
||||||
|
let folded_path = path.to_ascii_lowercase();
|
||||||
|
match bitap.best_distance(folded_path.as_bytes()) {
|
||||||
|
Some(distance) => (11.0 + 0.1 * distance as f64, &path, folded_path),
|
||||||
|
None => continue,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None => continue,
|
||||||
|
};
|
||||||
|
if !self.regex_accepts(file_id, &path, None)? {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Mark the approximate matched span in the matched field for
|
||||||
|
// the GUI's [matched field] rendering. window_around clamps
|
||||||
|
// and aligns.
|
||||||
|
let snip = bitap.count_and_first(folded_field.as_bytes()).1.map(|range| {
|
||||||
|
snippet::window_around(
|
||||||
|
field,
|
||||||
|
range,
|
||||||
|
&snippet::Options {
|
||||||
|
approx_chars: field.len().saturating_mul(2).max(8),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
});
|
||||||
|
let is_path_tier = rank >= 11.0;
|
||||||
|
let hit = SearchHit {
|
||||||
|
file_id,
|
||||||
|
name,
|
||||||
|
path,
|
||||||
|
size: row.get::<_, i64>(3).map_err(|e| e.to_string())?.max(0) as u64,
|
||||||
|
mtime: row.get(4).map_err(|e| e.to_string())?,
|
||||||
|
rank,
|
||||||
|
stage: rank as u8,
|
||||||
|
snippet: snip,
|
||||||
|
};
|
||||||
|
if is_path_tier {
|
||||||
|
path_buf.push(hit);
|
||||||
|
path_overflowed |= self.enforce_cap(&mut path_buf);
|
||||||
|
} else {
|
||||||
|
buf.push(hit);
|
||||||
|
overflowed |= self.enforce_cap(&mut buf);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
drop(rows);
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
self.deferred_fuzzy_path = Deferred {
|
||||||
|
hits: path_buf,
|
||||||
|
overflowed: path_overflowed,
|
||||||
|
};
|
||||||
|
self.flush_pass(buf, overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pass D — rank 8, bitap over every stored document text.
|
||||||
|
fn pass_fuzzy_fulltext(&mut self) -> Result<bool, String> {
|
||||||
|
if !self.options.fuzzy {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
// Bitap is a literal matcher; wildcard terms don't fuzz.
|
||||||
|
if self.query.pattern.is_wildcard() {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
let folded_term = self.query.term.to_ascii_lowercase();
|
||||||
|
let Some(k) = edit_budget(folded_term.len(), self.options.fuzzy_max_edits) else {
|
||||||
|
return Ok(true);
|
||||||
|
};
|
||||||
|
let Some(bitap) = Bitap::new(folded_term.as_bytes(), k) else {
|
||||||
|
return Ok(true);
|
||||||
|
};
|
||||||
|
|
||||||
|
let sql = format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime, dt.text_zstd \
|
||||||
|
FROM documents_text dt JOIN files f ON f.id = dt.file_id WHERE 1=1{}",
|
||||||
|
self.query.filter_sql
|
||||||
|
);
|
||||||
|
let params = self.params_with_filters(Vec::new());
|
||||||
|
let mut stmt = self.conn.prepare(&sql).map_err(|e| e.to_string())?;
|
||||||
|
let mut rows = stmt
|
||||||
|
.query(rusqlite::params_from_iter(params))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
|
||||||
|
let snippet_opts = snippet::Options { approx_chars: SNIPPET_WINDOW_CHARS };
|
||||||
|
let mut buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut overflowed = false;
|
||||||
|
while let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
let file_id: i64 = row.get(0).map_err(|e| e.to_string())?;
|
||||||
|
let path: String = row.get(2).map_err(|e| e.to_string())?;
|
||||||
|
if self.skip(file_id, &path) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let blob: Option<Vec<u8>> = row.get(5).map_err(|e| e.to_string())?;
|
||||||
|
let Some(blob) = blob else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let Ok(raw) = zstd::decode_all(blob.as_slice()) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let text = String::from_utf8_lossy(&raw).into_owned();
|
||||||
|
// ASCII folding is byte-length preserving, so ranges found in
|
||||||
|
// the folded buffer are valid in the original.
|
||||||
|
let folded = text.to_ascii_lowercase();
|
||||||
|
let (count, first) = bitap.count_and_first(folded.as_bytes());
|
||||||
|
if count == 0 {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if !self.regex_accepts(file_id, &path, Some(&text))? {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let snip = first.map(|range| snippet::window_around(&text, range, &snippet_opts));
|
||||||
|
buf.push(SearchHit {
|
||||||
|
file_id,
|
||||||
|
name: row.get(1).map_err(|e| e.to_string())?,
|
||||||
|
path,
|
||||||
|
size: row.get::<_, i64>(3).map_err(|e| e.to_string())?.max(0) as u64,
|
||||||
|
mtime: row.get(4).map_err(|e| e.to_string())?,
|
||||||
|
rank: 8.0 + count_frac(count),
|
||||||
|
stage: 8,
|
||||||
|
snippet: snip,
|
||||||
|
});
|
||||||
|
overflowed |= self.enforce_cap(&mut buf);
|
||||||
|
}
|
||||||
|
drop(rows);
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
self.flush_pass(buf, overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Regex-only pass over `files`: the regex bypasses the FTS trigram
|
||||||
|
/// entirely and runs on every name, falling back to the full path.
|
||||||
|
/// Name hits reuse rank 4, path hits defer to rank 10, so the GUI's
|
||||||
|
/// stage-based rendering needs no new cases.
|
||||||
|
fn pass_regex_name(&mut self) -> Result<bool, String> {
|
||||||
|
let query = self.query;
|
||||||
|
let re = query.regex.as_ref().expect("regex-only pass list");
|
||||||
|
let sql = format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime FROM files f WHERE 1=1{}",
|
||||||
|
query.filter_sql
|
||||||
|
);
|
||||||
|
let params = self.params_with_filters(Vec::new());
|
||||||
|
let mut stmt = self.conn.prepare(&sql).map_err(|e| e.to_string())?;
|
||||||
|
let mut rows = stmt
|
||||||
|
.query(rusqlite::params_from_iter(params))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
|
||||||
|
let mut buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut path_buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut overflowed = false;
|
||||||
|
let mut path_overflowed = false;
|
||||||
|
let mut scanned = 0usize;
|
||||||
|
while let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||||
|
scanned += 1;
|
||||||
|
if scanned % 1024 == 0 && self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
let file_id: i64 = row.get(0).map_err(|e| e.to_string())?;
|
||||||
|
let name: String = row.get(1).map_err(|e| e.to_string())?;
|
||||||
|
let path: String = row.get(2).map_err(|e| e.to_string())?;
|
||||||
|
if self.skip(file_id, &path) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// The name is the better hit; only a name miss falls through
|
||||||
|
// to the path tier — mirroring pass A.
|
||||||
|
let (rank, match_range, is_path_tier) = match re.find_first(&name) {
|
||||||
|
Some(r) => (4.0, (r.start, r.end), false),
|
||||||
|
None => match re.find_first(&path) {
|
||||||
|
Some(r) => (10.0, (r.start, r.end), true),
|
||||||
|
None => continue,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let snip = snippet::Snippet {
|
||||||
|
ranges: vec![match_range],
|
||||||
|
window: if is_path_tier { path.clone() } else { name.clone() },
|
||||||
|
truncated_start: false,
|
||||||
|
truncated_end: false,
|
||||||
|
};
|
||||||
|
let hit = SearchHit {
|
||||||
|
file_id,
|
||||||
|
name,
|
||||||
|
path,
|
||||||
|
size: row.get::<_, i64>(3).map_err(|e| e.to_string())?.max(0) as u64,
|
||||||
|
mtime: row.get(4).map_err(|e| e.to_string())?,
|
||||||
|
rank,
|
||||||
|
stage: rank as u8,
|
||||||
|
snippet: Some(snip),
|
||||||
|
};
|
||||||
|
if is_path_tier {
|
||||||
|
path_buf.push(hit);
|
||||||
|
path_overflowed |= self.enforce_cap(&mut path_buf);
|
||||||
|
} else {
|
||||||
|
buf.push(hit);
|
||||||
|
overflowed |= self.enforce_cap(&mut buf);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
drop(rows);
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
self.deferred_path = Deferred {
|
||||||
|
hits: path_buf,
|
||||||
|
overflowed: path_overflowed,
|
||||||
|
};
|
||||||
|
self.flush_pass(buf, overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Regex-only pass over every stored document text, reusing rank 6.
|
||||||
|
fn pass_regex_content(&mut self) -> Result<bool, String> {
|
||||||
|
let query = self.query;
|
||||||
|
let re = query.regex.as_ref().expect("regex-only pass list");
|
||||||
|
let sql = format!(
|
||||||
|
"SELECT f.id, f.name, f.path, f.size, f.mtime, dt.text_zstd \
|
||||||
|
FROM documents_text dt JOIN files f ON f.id = dt.file_id WHERE 1=1{}",
|
||||||
|
query.filter_sql
|
||||||
|
);
|
||||||
|
let params = self.params_with_filters(Vec::new());
|
||||||
|
let mut stmt = self.conn.prepare(&sql).map_err(|e| e.to_string())?;
|
||||||
|
let mut rows = stmt
|
||||||
|
.query(rusqlite::params_from_iter(params))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
|
||||||
|
let snippet_opts = snippet::Options { approx_chars: SNIPPET_WINDOW_CHARS };
|
||||||
|
let mut buf: Vec<SearchHit> = Vec::new();
|
||||||
|
let mut overflowed = false;
|
||||||
|
while let Some(row) = rows.next().map_err(|e| e.to_string())? {
|
||||||
|
// Decompression dominates: check every row.
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
let file_id: i64 = row.get(0).map_err(|e| e.to_string())?;
|
||||||
|
let path: String = row.get(2).map_err(|e| e.to_string())?;
|
||||||
|
if self.skip(file_id, &path) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let blob: Option<Vec<u8>> = row.get(5).map_err(|e| e.to_string())?;
|
||||||
|
let Some(raw) = blob.and_then(|b| zstd::decode_all(b.as_slice()).ok()) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let text = String::from_utf8_lossy(&raw).into_owned();
|
||||||
|
let count = re.count(&text);
|
||||||
|
if count == 0 {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// A greedy user regex can match megabytes; clamp the range
|
||||||
|
// before the snippet window is cut.
|
||||||
|
let snip = re.find_first(&text).map(|r| {
|
||||||
|
let r = clamp_match_range(&text, r, SNIPPET_WINDOW_CHARS);
|
||||||
|
snippet::window_around(&text, (r.start, r.end), &snippet_opts)
|
||||||
|
});
|
||||||
|
buf.push(SearchHit {
|
||||||
|
file_id,
|
||||||
|
name: row.get(1).map_err(|e| e.to_string())?,
|
||||||
|
path,
|
||||||
|
size: row.get::<_, i64>(3).map_err(|e| e.to_string())?.max(0) as u64,
|
||||||
|
mtime: row.get(4).map_err(|e| e.to_string())?,
|
||||||
|
rank: 6.0 + count_frac(count),
|
||||||
|
stage: 6,
|
||||||
|
snippet: snip,
|
||||||
|
});
|
||||||
|
overflowed |= self.enforce_cap(&mut buf);
|
||||||
|
}
|
||||||
|
drop(rows);
|
||||||
|
if self.cancelled() {
|
||||||
|
return Ok(false);
|
||||||
|
}
|
||||||
|
self.flush_pass(buf, overflowed);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
}
|
||||||
175
crates/quicksearch-core/src/search/duplicates.rs
Normal file
|
|
@ -0,0 +1,175 @@
|
||||||
|
//! Duplicate-file listing, grouped by content hash. Backs the GUI's
|
||||||
|
//! Duplicates tab; standalone and synchronous — callers run it on their
|
||||||
|
//! own worker thread.
|
||||||
|
|
||||||
|
use rusqlite::params;
|
||||||
|
|
||||||
|
use crate::db;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq)]
|
||||||
|
pub struct DuplicateGroup {
|
||||||
|
pub hash: Vec<u8>,
|
||||||
|
pub count: i64,
|
||||||
|
pub total_size: i64,
|
||||||
|
/// Bytes reclaimable by deduplicating: `size × (count - 1)` — the
|
||||||
|
/// group's sort key.
|
||||||
|
pub redundant_size: i64,
|
||||||
|
/// `(file_id, name, path, size, mtime)` per member, path-ordered.
|
||||||
|
pub members: Vec<(i64, String, String, u64, i64)>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Page through hash groups having more than one member, ordered by
|
||||||
|
/// reclaimable bytes (largest first). Rows with a NULL hash (never
|
||||||
|
/// hashed) and zero-size files (all trivially identical) are excluded.
|
||||||
|
pub fn find_duplicate_groups(
|
||||||
|
db_path: &str,
|
||||||
|
limit: u32,
|
||||||
|
offset: u32,
|
||||||
|
) -> Result<Vec<DuplicateGroup>, String> {
|
||||||
|
let conn = db::open_existing(db_path, false)?;
|
||||||
|
let mut groups: Vec<DuplicateGroup> = Vec::new();
|
||||||
|
{
|
||||||
|
// SUM(size) - MAX(size) == size × (count - 1); members of a group
|
||||||
|
// share a size because the hash covers it.
|
||||||
|
let mut stmt = conn
|
||||||
|
.prepare(
|
||||||
|
"SELECT hash, COUNT(*) AS cnt, SUM(size), SUM(size) - MAX(size) AS redundant \
|
||||||
|
FROM files \
|
||||||
|
WHERE hash IS NOT NULL AND size > 0 \
|
||||||
|
GROUP BY hash HAVING cnt > 1 \
|
||||||
|
ORDER BY redundant DESC, hash \
|
||||||
|
LIMIT ?1 OFFSET ?2",
|
||||||
|
)
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
let rows = stmt
|
||||||
|
.query_map(params![limit, offset], |r| {
|
||||||
|
Ok(DuplicateGroup {
|
||||||
|
hash: r.get(0)?,
|
||||||
|
count: r.get(1)?,
|
||||||
|
total_size: r.get(2)?,
|
||||||
|
redundant_size: r.get(3)?,
|
||||||
|
members: Vec::new(),
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
for g in rows {
|
||||||
|
groups.push(g.map_err(|e| e.to_string())?);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut member_stmt = conn
|
||||||
|
.prepare(
|
||||||
|
"SELECT id, name, path, size, mtime FROM files WHERE hash = ?1 ORDER BY path",
|
||||||
|
)
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
for group in &mut groups {
|
||||||
|
let rows = member_stmt
|
||||||
|
.query_map(params![group.hash], |r| {
|
||||||
|
Ok((
|
||||||
|
r.get::<_, i64>(0)?,
|
||||||
|
r.get::<_, String>(1)?,
|
||||||
|
r.get::<_, String>(2)?,
|
||||||
|
r.get::<_, i64>(3)?.max(0) as u64,
|
||||||
|
r.get::<_, i64>(4)?,
|
||||||
|
))
|
||||||
|
})
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
for m in rows {
|
||||||
|
group.members.push(m.map_err(|e| e.to_string())?);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(groups)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::db::open_or_recreate;
|
||||||
|
use crate::db::repo::{insert_file, NewFile};
|
||||||
|
use crate::mime::FileType;
|
||||||
|
|
||||||
|
fn seed_db() -> std::path::PathBuf {
|
||||||
|
let mut p = std::env::temp_dir();
|
||||||
|
p.push(format!(
|
||||||
|
"qs-dups-{}-{}.sqlite",
|
||||||
|
std::process::id(),
|
||||||
|
std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.unwrap()
|
||||||
|
.as_nanos()
|
||||||
|
));
|
||||||
|
let mut conn = open_or_recreate(p.to_str().unwrap(), "trigram").unwrap();
|
||||||
|
let tx = conn.transaction().unwrap();
|
||||||
|
let add = |name: &str, path: &str, size: u64, hash: Option<&[u8]>| {
|
||||||
|
insert_file(
|
||||||
|
&tx,
|
||||||
|
&NewFile {
|
||||||
|
name,
|
||||||
|
path,
|
||||||
|
parent: "/d",
|
||||||
|
size,
|
||||||
|
mtime: 1_700_000_000,
|
||||||
|
inode: None,
|
||||||
|
device_id: None,
|
||||||
|
mime: None,
|
||||||
|
ftype: FileType::TEXT,
|
||||||
|
hash,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
.expect("unique path");
|
||||||
|
};
|
||||||
|
// Triple group of small files: redundant = 10 × 2 = 20.
|
||||||
|
add("a1.txt", "/d/a1.txt", 10, Some(b"AAA"));
|
||||||
|
add("a2.txt", "/d/a2.txt", 10, Some(b"AAA"));
|
||||||
|
add("a3.txt", "/d/a3.txt", 10, Some(b"AAA"));
|
||||||
|
// Pair of large files: redundant = 100 × 1 = 100 — sorts first
|
||||||
|
// despite the smaller member count.
|
||||||
|
add("b1.txt", "/d/b1.txt", 100, Some(b"BBB"));
|
||||||
|
add("b2.txt", "/d/b2.txt", 100, Some(b"BBB"));
|
||||||
|
// Singletons and NULL hashes never appear.
|
||||||
|
add("c.txt", "/d/c.txt", 30, Some(b"CCC"));
|
||||||
|
add("n1.txt", "/d/n1.txt", 40, None);
|
||||||
|
add("n2.txt", "/d/n2.txt", 40, None);
|
||||||
|
// Zero-size files are trivially identical — excluded outright.
|
||||||
|
add("z1.txt", "/d/z1.txt", 0, Some(b"ZZZ"));
|
||||||
|
add("z2.txt", "/d/z2.txt", 0, Some(b"ZZZ"));
|
||||||
|
tx.commit().unwrap();
|
||||||
|
drop(conn);
|
||||||
|
p
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn groups_ordered_by_redundant_size_zero_size_excluded() {
|
||||||
|
let p = seed_db();
|
||||||
|
let groups = find_duplicate_groups(p.to_str().unwrap(), 10, 0).unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
groups.len(),
|
||||||
|
2,
|
||||||
|
"singletons, NULL hashes, and zero-size groups excluded"
|
||||||
|
);
|
||||||
|
// Reclaimable bytes beat member count for ordering.
|
||||||
|
assert_eq!(groups[0].count, 2);
|
||||||
|
assert_eq!(groups[0].total_size, 200);
|
||||||
|
assert_eq!(groups[0].redundant_size, 100);
|
||||||
|
assert_eq!(groups[1].count, 3);
|
||||||
|
assert_eq!(groups[1].redundant_size, 20);
|
||||||
|
assert_eq!(groups[1].members.len(), 3);
|
||||||
|
assert_eq!(groups[1].members[0].1, "a1.txt", "members path-ordered");
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pagination() {
|
||||||
|
let p = seed_db();
|
||||||
|
let page1 = find_duplicate_groups(p.to_str().unwrap(), 1, 0).unwrap();
|
||||||
|
let page2 = find_duplicate_groups(p.to_str().unwrap(), 1, 1).unwrap();
|
||||||
|
assert_eq!(page1.len(), 1);
|
||||||
|
assert_eq!(page2.len(), 1);
|
||||||
|
assert_ne!(page1[0].hash, page2[0].hash);
|
||||||
|
let page3 = find_duplicate_groups(p.to_str().unwrap(), 1, 2).unwrap();
|
||||||
|
assert!(page3.is_empty());
|
||||||
|
std::fs::remove_file(&p).ok();
|
||||||
|
}
|
||||||
|
}
|
||||||
330
crates/quicksearch-core/src/search/fuzzy.rs
Normal file
|
|
@ -0,0 +1,330 @@
|
||||||
|
//! Approximate substring matching for the fuzzy cascade stages.
|
||||||
|
//!
|
||||||
|
//! Bitap (shift-and with errors, Wu–Manber): finds occurrences of a
|
||||||
|
//! pattern *within* a haystack with at most `k` Levenshtein edits
|
||||||
|
//! (insertion / deletion / substitution). Substring semantics are the
|
||||||
|
//! right fit for search-as-you-type — users type fragments, not whole
|
||||||
|
//! filenames — and the u64 bit-parallel update costs O(k) word ops per
|
||||||
|
//! haystack byte with zero allocations.
|
||||||
|
//!
|
||||||
|
//! Callers fold both sides to ASCII lowercase first (the pipeline-wide
|
||||||
|
//! convention). Patterns are limited to 64 bytes by the machine word; the
|
||||||
|
//! cascade skips fuzzy stages for longer terms.
|
||||||
|
|
||||||
|
pub struct Bitap {
|
||||||
|
/// `masks[c]` has bit `i` set iff `pattern[i] == c`.
|
||||||
|
masks: [u64; 256],
|
||||||
|
/// Pattern length in bytes (1..=64).
|
||||||
|
len: usize,
|
||||||
|
/// Maximum edit distance.
|
||||||
|
k: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Bitap {
|
||||||
|
/// `None` when the pattern is empty, longer than 64 bytes, or the edit
|
||||||
|
/// budget reaches the word size (`initial_registers` shifts by `k`).
|
||||||
|
pub fn new(pattern: &[u8], k: usize) -> Option<Bitap> {
|
||||||
|
if pattern.is_empty() || pattern.len() > 64 || k >= 64 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let mut masks = [0u64; 256];
|
||||||
|
for (i, &b) in pattern.iter().enumerate() {
|
||||||
|
masks[b as usize] |= 1u64 << i;
|
||||||
|
}
|
||||||
|
Some(Bitap {
|
||||||
|
masks,
|
||||||
|
len: pattern.len(),
|
||||||
|
k,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fresh per-distance state registers. Bit `i` of `r[d]` set means "a
|
||||||
|
/// match of pattern[..=i] with ≤ d errors ends at the current text
|
||||||
|
/// position". With d errors the first d pattern bytes can be deleted
|
||||||
|
/// before any text is read, hence the pre-set low bits.
|
||||||
|
fn initial_registers(&self) -> Vec<u64> {
|
||||||
|
(0..=self.k)
|
||||||
|
.map(|d| if d == 0 { 0 } else { (1u64 << d) - 1 })
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Advance all registers by one haystack byte. Returns the smallest
|
||||||
|
/// error count d for which the full pattern just matched, if any.
|
||||||
|
#[inline]
|
||||||
|
fn step(&self, r: &mut [u64], byte: u8) -> Option<usize> {
|
||||||
|
let mask = self.masks[byte as usize];
|
||||||
|
let done = 1u64 << (self.len - 1);
|
||||||
|
let mut hit = None;
|
||||||
|
let mut prev_old = r[0]; // R_old[d-1] for the d-th iteration
|
||||||
|
// d = 0: exact prefix extension only.
|
||||||
|
r[0] = ((r[0] << 1) | 1) & mask;
|
||||||
|
if r[0] & done != 0 {
|
||||||
|
hit = Some(0);
|
||||||
|
}
|
||||||
|
for d in 1..=self.k {
|
||||||
|
let old = r[d];
|
||||||
|
r[d] = (((old << 1) | 1) & mask) // extend a ≤d-error state
|
||||||
|
| prev_old // insertion in text
|
||||||
|
| (prev_old << 1) // substitution
|
||||||
|
| ((r[d - 1] << 1) | 1); // deletion (pattern byte skipped)
|
||||||
|
prev_old = old;
|
||||||
|
if hit.is_none() && r[d] & done != 0 {
|
||||||
|
hit = Some(d);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
hit
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Minimum edit distance (≤ k) of any occurrence of the pattern in
|
||||||
|
/// `hay`, or `None` if nothing matches within k edits.
|
||||||
|
pub fn best_distance(&self, hay: &[u8]) -> Option<usize> {
|
||||||
|
let mut r = self.initial_registers();
|
||||||
|
let mut best: Option<usize> = None;
|
||||||
|
for &b in hay {
|
||||||
|
if let Some(d) = self.step(&mut r, b) {
|
||||||
|
if d == 0 {
|
||||||
|
return Some(0);
|
||||||
|
}
|
||||||
|
if best.map_or(true, |cur| d < cur) {
|
||||||
|
best = Some(d);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
best
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Count non-overlapping occurrences (at ≤ k edits) and report the
|
||||||
|
/// first match's approximate byte range in `hay`. After each hit the
|
||||||
|
/// automaton resets, so an exact match followed by trailing bytes
|
||||||
|
/// counts once, and overlapping suffix matches don't inflate counts.
|
||||||
|
/// The reported range assumes pattern-length matches — edits can shift
|
||||||
|
/// the true start by up to k bytes, which is fine for snippet windows.
|
||||||
|
pub fn count_and_first(&self, hay: &[u8]) -> (usize, Option<(usize, usize)>) {
|
||||||
|
let mut r = self.initial_registers();
|
||||||
|
let mut count = 0usize;
|
||||||
|
let mut first: Option<(usize, usize)> = None;
|
||||||
|
for (i, &b) in hay.iter().enumerate() {
|
||||||
|
if self.step(&mut r, b).is_some() {
|
||||||
|
count += 1;
|
||||||
|
if first.is_none() {
|
||||||
|
let end = i + 1;
|
||||||
|
first = Some((end.saturating_sub(self.len), end));
|
||||||
|
}
|
||||||
|
for (d, reg) in r.iter_mut().enumerate() {
|
||||||
|
*reg = if d == 0 { 0 } else { (1u64 << d) - 1 };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(count, first)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The cascade's edit-distance budget for a folded term: one edit per
|
||||||
|
/// three characters, capped by `[search].fuzzy_max_edits`. Terms outside
|
||||||
|
/// 3..=64 bytes skip the fuzzy stages entirely (< 3 is noise, > 64 exceeds
|
||||||
|
/// the word size), and a cap of 0 disables them everywhere.
|
||||||
|
///
|
||||||
|
/// At the default cap of 2 this is the historic ladder: 3–5 bytes get one
|
||||||
|
/// edit, 6–64 get two.
|
||||||
|
pub fn edit_budget(term_len: usize, max_edits: usize) -> Option<usize> {
|
||||||
|
if !(3..=64).contains(&term_len) || max_edits == 0 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some((term_len / 3).min(max_edits))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn best(pattern: &str, hay: &str, k: usize) -> Option<usize> {
|
||||||
|
Bitap::new(pattern.as_bytes(), k)
|
||||||
|
.unwrap()
|
||||||
|
.best_distance(hay.as_bytes())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn exact_substring_is_distance_zero() {
|
||||||
|
assert_eq!(best("hello", "say hello world", 2), Some(0));
|
||||||
|
assert_eq!(best("hello", "hello", 0), Some(0));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn single_edits_are_distance_one() {
|
||||||
|
assert_eq!(best("hello", "xx hxllo xx", 2), Some(1), "substitution");
|
||||||
|
assert_eq!(best("hello", "xx helo xx", 2), Some(1), "deletion");
|
||||||
|
assert_eq!(best("hello", "xx heXllo xx", 2), Some(1), "insertion");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn two_edits() {
|
||||||
|
assert_eq!(best("hello", "xx hxlo xx", 2), Some(2));
|
||||||
|
assert_eq!(best("hello", "xx ho xx", 2), None, "3 edits > k");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn no_match_within_budget() {
|
||||||
|
assert_eq!(best("hello", "completely different", 1), None);
|
||||||
|
assert_eq!(best("abc", "", 1), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn k_zero_is_exact_search() {
|
||||||
|
assert_eq!(best("abc", "xxabcxx", 0), Some(0));
|
||||||
|
assert_eq!(best("abc", "xxabxcx", 0), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pattern_length_limits() {
|
||||||
|
assert!(Bitap::new(b"", 1).is_none());
|
||||||
|
assert!(Bitap::new(&[b'a'; 65], 1).is_none());
|
||||||
|
assert!(Bitap::new(&[b'a'; 64], 1).is_some());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn oversized_k_is_rejected_not_shifted() {
|
||||||
|
// `initial_registers` shifts by k; k >= 64 would overflow u64.
|
||||||
|
assert!(Bitap::new(b"abc", 64).is_none());
|
||||||
|
assert!(Bitap::new(b"abc", usize::MAX).is_none());
|
||||||
|
assert!(Bitap::new(b"abc", 63).is_some());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn count_non_overlapping() {
|
||||||
|
let b = Bitap::new(b"ab", 0).unwrap();
|
||||||
|
let (count, first) = b.count_and_first(b"ab ab ab");
|
||||||
|
assert_eq!(count, 3);
|
||||||
|
assert_eq!(first, Some((0, 2)));
|
||||||
|
|
||||||
|
// "aaaa" contains "aaa" once non-overlapping.
|
||||||
|
let b = Bitap::new(b"aaa", 0).unwrap();
|
||||||
|
let (count, _) = b.count_and_first(b"aaaa");
|
||||||
|
assert_eq!(count, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn count_fuzzy_and_range_sane() {
|
||||||
|
let b = Bitap::new(b"hello", 1).unwrap();
|
||||||
|
let hay = b"say helo and hxllo again";
|
||||||
|
let (count, first) = b.count_and_first(hay);
|
||||||
|
assert_eq!(count, 2);
|
||||||
|
let (s, e) = first.unwrap();
|
||||||
|
assert!(s < e && e <= hay.len());
|
||||||
|
let window = &hay[s..e];
|
||||||
|
assert!(
|
||||||
|
std::str::from_utf8(window).unwrap().contains("hel"),
|
||||||
|
"first range should cover the first hit, got {:?}",
|
||||||
|
std::str::from_utf8(window)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn edit_budget_default_cap_is_the_historic_ladder() {
|
||||||
|
assert_eq!(edit_budget(0, 2), None);
|
||||||
|
assert_eq!(edit_budget(2, 2), None);
|
||||||
|
assert_eq!(edit_budget(3, 2), Some(1));
|
||||||
|
assert_eq!(edit_budget(5, 2), Some(1));
|
||||||
|
assert_eq!(edit_budget(6, 2), Some(2));
|
||||||
|
assert_eq!(edit_budget(64, 2), Some(2));
|
||||||
|
assert_eq!(edit_budget(65, 2), None);
|
||||||
|
assert_eq!(edit_budget(usize::MAX, 2), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn edit_budget_scales_with_length_up_to_the_cap() {
|
||||||
|
assert_eq!(edit_budget(3, 4), Some(1));
|
||||||
|
assert_eq!(edit_budget(6, 4), Some(2));
|
||||||
|
assert_eq!(edit_budget(9, 4), Some(3));
|
||||||
|
assert_eq!(edit_budget(12, 4), Some(4));
|
||||||
|
assert_eq!(edit_budget(64, 4), Some(4), "cap wins over length");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn edit_budget_cap_of_one_stays_strict() {
|
||||||
|
for len in 3..=64 {
|
||||||
|
assert_eq!(edit_budget(len, 1), Some(1));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn edit_budget_zero_disables_fuzzy() {
|
||||||
|
for len in 0..=70 {
|
||||||
|
assert_eq!(edit_budget(len, 0), None);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Even a hostile config value can't produce a k the bitap rejects:
|
||||||
|
/// the length ladder caps it at 21 for the longest legal term.
|
||||||
|
#[test]
|
||||||
|
fn edit_budget_stays_within_the_bitap_word_size() {
|
||||||
|
for len in 3..=64 {
|
||||||
|
let k = edit_budget(len, usize::MAX).unwrap();
|
||||||
|
assert!(k <= 21, "len={} gave k={}", len, k);
|
||||||
|
assert!(Bitap::new(&vec![b'a'; len], k).is_some());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Brute-force oracle: minimum Levenshtein distance between `pattern`
|
||||||
|
/// and any substring of `hay`, capped at k.
|
||||||
|
fn oracle(pattern: &[u8], hay: &[u8], k: usize) -> Option<usize> {
|
||||||
|
// An occurrence must end at some text position; empty text has
|
||||||
|
// none. Without this, k >= pattern-length "matches" empty text by
|
||||||
|
// deleting every pattern byte — a degenerate non-occurrence the
|
||||||
|
// automaton rightly never reports. (Production keeps k < len via
|
||||||
|
// the len/3 budget, so only the oracle ever saw this edge.)
|
||||||
|
if hay.is_empty() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
// Standard DP where row 0 is all zeros (match can start anywhere).
|
||||||
|
let m = pattern.len();
|
||||||
|
let mut prev: Vec<usize> = vec![0; hay.len() + 1];
|
||||||
|
let mut cur = vec![0; hay.len() + 1];
|
||||||
|
let mut best = usize::MAX;
|
||||||
|
// dp[i][j] = min edits to match pattern[..i] ending at hay[..j]
|
||||||
|
for i in 1..=m {
|
||||||
|
cur[0] = i;
|
||||||
|
for j in 1..=hay.len() {
|
||||||
|
let cost = if pattern[i - 1] == hay[j - 1] { 0 } else { 1 };
|
||||||
|
cur[j] = (prev[j - 1] + cost).min(prev[j] + 1).min(cur[j - 1] + 1);
|
||||||
|
}
|
||||||
|
std::mem::swap(&mut prev, &mut cur);
|
||||||
|
}
|
||||||
|
for j in 0..=hay.len() {
|
||||||
|
best = best.min(prev[j]);
|
||||||
|
}
|
||||||
|
if best <= k {
|
||||||
|
Some(best)
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn matches_brute_force_oracle() {
|
||||||
|
// Deterministic LCG so the test is reproducible.
|
||||||
|
let mut seed: u64 = 0x2545F4914F6CDD1D;
|
||||||
|
let mut rng = move || {
|
||||||
|
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
|
||||||
|
(seed >> 33) as usize
|
||||||
|
};
|
||||||
|
let alphabet = b"abcx";
|
||||||
|
for _ in 0..500 {
|
||||||
|
let plen = 3 + rng() % 6;
|
||||||
|
let hlen = rng() % 20;
|
||||||
|
let pattern: Vec<u8> = (0..plen).map(|_| alphabet[rng() % 4]).collect();
|
||||||
|
let hay: Vec<u8> = (0..hlen).map(|_| alphabet[rng() % 4]).collect();
|
||||||
|
for k in 0..=4 {
|
||||||
|
let got = Bitap::new(&pattern, k).unwrap().best_distance(&hay);
|
||||||
|
let want = oracle(&pattern, &hay, k);
|
||||||
|
assert_eq!(
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
"pattern={:?} hay={:?} k={}",
|
||||||
|
std::str::from_utf8(&pattern),
|
||||||
|
std::str::from_utf8(&hay),
|
||||||
|
k
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
307
crates/quicksearch-core/src/search/mod.rs
Normal file
|
|
@ -0,0 +1,307 @@
|
||||||
|
//! Interruptible, streaming search service.
|
||||||
|
//!
|
||||||
|
//! One dedicated worker thread owns the cascade. The GUI (or any caller)
|
||||||
|
//! sends queries via [`SearchService::search`]; results stream back over
|
||||||
|
//! an mpsc receiver as [`SearchUpdate`] events tagged with a generation
|
||||||
|
//! number. Starting a new search bumps the generation and interrupts the
|
||||||
|
//! in-flight SQLite statement, so a keystroke never waits on the previous
|
||||||
|
//! query.
|
||||||
|
//!
|
||||||
|
//! Cancellation is two-layer:
|
||||||
|
//! - **cooperative** — the cascade compares its generation against the
|
||||||
|
//! latest every few hundred rows and stops silently when stale;
|
||||||
|
//! - **interrupt** — [`rusqlite::InterruptHandle::interrupt`] kills the
|
||||||
|
//! statement currently executing (covering the "no rows produced yet"
|
||||||
|
//! phases like FTS candidate gathering). An interrupted stale search is
|
||||||
|
//! normal cancellation, not an error.
|
||||||
|
//!
|
||||||
|
//! Consumers that want a plain blocking search (the CLI mode) skip the
|
||||||
|
//! service entirely and call [`cascade::run`] with a collecting sink.
|
||||||
|
|
||||||
|
pub mod cascade;
|
||||||
|
pub mod duplicates;
|
||||||
|
pub mod fuzzy;
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::sync::atomic::{AtomicU64, Ordering};
|
||||||
|
use std::sync::{mpsc, Arc, Mutex};
|
||||||
|
use std::thread::JoinHandle;
|
||||||
|
|
||||||
|
use crate::db;
|
||||||
|
use crate::query::split::split_for_cascade;
|
||||||
|
use crate::snippet::Snippet;
|
||||||
|
|
||||||
|
pub use cascade::Outcome;
|
||||||
|
pub use duplicates::{find_duplicate_groups, DuplicateGroup};
|
||||||
|
|
||||||
|
/// One search result. `rank` is the sort key (lower = better): integer
|
||||||
|
/// part = cascade stage (1–11), fraction = occurrence-count or
|
||||||
|
/// edit-distance tiebreak. Batches arrive already rank-ordered and later
|
||||||
|
/// batches only append, so a rank-sorted view never reshuffles.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SearchHit {
|
||||||
|
pub file_id: i64,
|
||||||
|
pub name: String,
|
||||||
|
pub path: String,
|
||||||
|
pub size: u64,
|
||||||
|
pub mtime: i64,
|
||||||
|
pub rank: f64,
|
||||||
|
pub stage: u8,
|
||||||
|
/// The matched span in context: the filename for name stages, the full
|
||||||
|
/// path for path stages, a window of the body for full-text stages
|
||||||
|
/// (absent there when document text isn't stored).
|
||||||
|
pub snippet: Option<Snippet>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub enum SearchUpdate {
|
||||||
|
Started { generation: u64 },
|
||||||
|
Hits { generation: u64, hits: Vec<SearchHit> },
|
||||||
|
Completed { generation: u64, total: usize, limited: bool },
|
||||||
|
Error { generation: u64, message: String },
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SearchUpdate {
|
||||||
|
pub fn generation(&self) -> u64 {
|
||||||
|
match self {
|
||||||
|
SearchUpdate::Started { generation }
|
||||||
|
| SearchUpdate::Hits { generation, .. }
|
||||||
|
| SearchUpdate::Completed { generation, .. }
|
||||||
|
| SearchUpdate::Error { generation, .. } => *generation,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct SearchOptions {
|
||||||
|
/// Enable the fuzzy stages (ranks 7, 8 and 11).
|
||||||
|
pub fuzzy: bool,
|
||||||
|
/// Ceiling on the fuzzy edit budget (`[search].fuzzy_max_edits`); see
|
||||||
|
/// [`fuzzy::edit_budget`]. 0 disables the fuzzy stages.
|
||||||
|
pub fuzzy_max_edits: usize,
|
||||||
|
/// Hard cap on total hits per search (`[search].display_limit`).
|
||||||
|
pub limit: usize,
|
||||||
|
/// Streaming batch size (`[search].results_per_page`).
|
||||||
|
pub batch: usize,
|
||||||
|
/// Session-scoped ignore patterns (GUI chips), same glob semantics as
|
||||||
|
/// the config's `ignore_patterns`. Applied before the display cap.
|
||||||
|
pub session_ignores: Vec<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for SearchOptions {
|
||||||
|
fn default() -> Self {
|
||||||
|
SearchOptions {
|
||||||
|
fuzzy: false,
|
||||||
|
fuzzy_max_edits: 2,
|
||||||
|
limit: 1000,
|
||||||
|
batch: 100,
|
||||||
|
session_ignores: Vec::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct SearchRequest {
|
||||||
|
generation: u64,
|
||||||
|
input: String,
|
||||||
|
options: SearchOptions,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct SearchService {
|
||||||
|
req_tx: mpsc::Sender<SearchRequest>,
|
||||||
|
latest_gen: Arc<AtomicU64>,
|
||||||
|
interrupt: Arc<Mutex<Option<rusqlite::InterruptHandle>>>,
|
||||||
|
db_path: Arc<Mutex<PathBuf>>,
|
||||||
|
handle: Option<JoinHandle<()>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SearchService {
|
||||||
|
/// Spawn the worker. `notify` is invoked after every update event so
|
||||||
|
/// an egui frontend can `request_repaint` (pass a no-op for headless
|
||||||
|
/// use). Returns the service handle plus the update receiver, which
|
||||||
|
/// the caller drains non-blockingly.
|
||||||
|
pub fn new(
|
||||||
|
db_path: PathBuf,
|
||||||
|
notify: Arc<dyn Fn() + Send + Sync>,
|
||||||
|
) -> (SearchService, mpsc::Receiver<SearchUpdate>) {
|
||||||
|
let (req_tx, req_rx) = mpsc::channel::<SearchRequest>();
|
||||||
|
let (update_tx, update_rx) = mpsc::channel::<SearchUpdate>();
|
||||||
|
let latest_gen = Arc::new(AtomicU64::new(0));
|
||||||
|
let interrupt = Arc::new(Mutex::new(None));
|
||||||
|
let db_path = Arc::new(Mutex::new(db_path));
|
||||||
|
|
||||||
|
let worker = Worker {
|
||||||
|
req_rx,
|
||||||
|
update_tx,
|
||||||
|
notify,
|
||||||
|
latest_gen: latest_gen.clone(),
|
||||||
|
interrupt: interrupt.clone(),
|
||||||
|
db_path: db_path.clone(),
|
||||||
|
};
|
||||||
|
let handle = std::thread::Builder::new()
|
||||||
|
.name("qs-search".into())
|
||||||
|
.spawn(move || worker.run())
|
||||||
|
.expect("spawn search worker");
|
||||||
|
|
||||||
|
(
|
||||||
|
SearchService {
|
||||||
|
req_tx,
|
||||||
|
latest_gen,
|
||||||
|
interrupt,
|
||||||
|
db_path,
|
||||||
|
handle: Some(handle),
|
||||||
|
},
|
||||||
|
update_rx,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Start a new search, cancelling any in-flight one. Returns the
|
||||||
|
/// generation whose events to keep.
|
||||||
|
pub fn search(&self, input: &str, options: SearchOptions) -> u64 {
|
||||||
|
let generation = self.latest_gen.fetch_add(1, Ordering::SeqCst) + 1;
|
||||||
|
let _ = self.req_tx.send(SearchRequest {
|
||||||
|
generation,
|
||||||
|
input: input.to_string(),
|
||||||
|
options,
|
||||||
|
});
|
||||||
|
// The new request can't be running yet (the worker hasn't dequeued
|
||||||
|
// it), so this only ever kills a stale generation's statement.
|
||||||
|
self.interrupt_current();
|
||||||
|
generation
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Cancel without starting anything new.
|
||||||
|
pub fn cancel(&self) {
|
||||||
|
self.latest_gen.fetch_add(1, Ordering::SeqCst);
|
||||||
|
self.interrupt_current();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Point subsequent searches at a different index file.
|
||||||
|
pub fn set_db_path(&self, path: PathBuf) {
|
||||||
|
*self.db_path.lock().unwrap() = path;
|
||||||
|
self.cancel();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn interrupt_current(&self) {
|
||||||
|
if let Ok(guard) = self.interrupt.lock() {
|
||||||
|
if let Some(handle) = guard.as_ref() {
|
||||||
|
handle.interrupt();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Cancel, close the request channel, and join the worker.
|
||||||
|
pub fn shutdown(self) {
|
||||||
|
self.cancel();
|
||||||
|
let SearchService { req_tx, handle, .. } = self;
|
||||||
|
drop(req_tx);
|
||||||
|
if let Some(handle) = handle {
|
||||||
|
let _ = handle.join();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Map SQLite-level errors to the tagged strings frontends key off.
|
||||||
|
/// `DATABASE_CORRUPTED:` drives the GUI's recovery dialog.
|
||||||
|
pub fn classify_sql_err(error_msg: &str) -> String {
|
||||||
|
if error_msg.contains("malformed")
|
||||||
|
|| error_msg.contains("corrupt")
|
||||||
|
|| error_msg.contains("database disk image is malformed")
|
||||||
|
{
|
||||||
|
format!("DATABASE_CORRUPTED: {}", error_msg)
|
||||||
|
} else if error_msg.contains("fts5: syntax error") {
|
||||||
|
"Search syntax error: the search term contains characters that cannot be processed."
|
||||||
|
.to_string()
|
||||||
|
} else {
|
||||||
|
format!("Search failed: {}", error_msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Worker {
|
||||||
|
req_rx: mpsc::Receiver<SearchRequest>,
|
||||||
|
update_tx: mpsc::Sender<SearchUpdate>,
|
||||||
|
notify: Arc<dyn Fn() + Send + Sync>,
|
||||||
|
latest_gen: Arc<AtomicU64>,
|
||||||
|
interrupt: Arc<Mutex<Option<rusqlite::InterruptHandle>>>,
|
||||||
|
db_path: Arc<Mutex<PathBuf>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Worker {
|
||||||
|
fn run(self) {
|
||||||
|
while let Ok(first) = self.req_rx.recv() {
|
||||||
|
// A fast typist queues several requests; only the newest one
|
||||||
|
// matters.
|
||||||
|
let mut req = first;
|
||||||
|
while let Ok(newer) = self.req_rx.try_recv() {
|
||||||
|
req = newer;
|
||||||
|
}
|
||||||
|
if req.generation != self.latest_gen.load(Ordering::SeqCst) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
self.handle(req);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn send(&self, update: SearchUpdate) {
|
||||||
|
let _ = self.update_tx.send(update);
|
||||||
|
(self.notify)();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn handle(&self, req: SearchRequest) {
|
||||||
|
let generation = req.generation;
|
||||||
|
self.send(SearchUpdate::Started { generation });
|
||||||
|
|
||||||
|
let split = match split_for_cascade(&req.input) {
|
||||||
|
Ok(s) => s,
|
||||||
|
Err(e) => {
|
||||||
|
self.send(SearchUpdate::Error {
|
||||||
|
generation,
|
||||||
|
message: e.to_string(),
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let db_path = self.db_path.lock().unwrap().clone();
|
||||||
|
// Per-request open: microseconds, and always sees a freshly
|
||||||
|
// rebuilt index file rather than pinning a deleted inode.
|
||||||
|
let conn = match db::open_existing(&db_path.to_string_lossy(), false) {
|
||||||
|
Ok(c) => c,
|
||||||
|
Err(e) => {
|
||||||
|
self.send(SearchUpdate::Error {
|
||||||
|
generation,
|
||||||
|
message: classify_sql_err(&e),
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
*self.interrupt.lock().unwrap() = Some(conn.get_interrupt_handle());
|
||||||
|
|
||||||
|
let mut sink = |hits: Vec<SearchHit>| {
|
||||||
|
self.send(SearchUpdate::Hits { generation, hits });
|
||||||
|
};
|
||||||
|
let outcome = cascade::run(
|
||||||
|
&conn,
|
||||||
|
&split,
|
||||||
|
&req.options,
|
||||||
|
generation,
|
||||||
|
&self.latest_gen,
|
||||||
|
&mut sink,
|
||||||
|
);
|
||||||
|
|
||||||
|
*self.interrupt.lock().unwrap() = None;
|
||||||
|
|
||||||
|
match outcome {
|
||||||
|
Ok(Some(Outcome { total, limited })) => self.send(SearchUpdate::Completed {
|
||||||
|
generation,
|
||||||
|
total,
|
||||||
|
limited,
|
||||||
|
}),
|
||||||
|
// Cancelled — the newer generation owns the UI now.
|
||||||
|
Ok(None) => {}
|
||||||
|
Err(e) => self.send(SearchUpdate::Error {
|
||||||
|
generation,
|
||||||
|
message: classify_sql_err(&e),
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,396 +0,0 @@
|
||||||
//! SQL builders for the GUI's three legacy search modes (full-text,
|
|
||||||
//! filename, duplicate). Lives in core so it has unit-test coverage; the
|
|
||||||
//! GUI just composes these into per-page queries.
|
|
||||||
//!
|
|
||||||
//! For the structured Baloo-subset query language (`type:`, `modified:`,
|
|
||||||
//! …), see [`crate::query`].
|
|
||||||
|
|
||||||
/// All inputs needed to run (and re-run) one of the three search modes.
|
|
||||||
/// Cached after a fresh search so paging buttons don't have to rebuild
|
|
||||||
/// from form state.
|
|
||||||
#[derive(Debug, Clone, PartialEq)]
|
|
||||||
pub struct SearchArgs {
|
|
||||||
pub search_type: String,
|
|
||||||
pub term: String,
|
|
||||||
pub fulltext_exact: bool,
|
|
||||||
pub fulltext_case_sensitive: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// SQL that counts every row matching `args`, ignoring pagination. Used to
|
|
||||||
/// drive the "page X of Y" UI. For very large FTS hit sets this can take a
|
|
||||||
/// noticeable fraction of the per-page query time, but it only runs on a
|
|
||||||
/// fresh search — page navigation reuses the cached total.
|
|
||||||
pub fn build_count(args: &SearchArgs) -> Result<String, String> {
|
|
||||||
match args.search_type.as_str() {
|
|
||||||
"fulltext" => {
|
|
||||||
let where_clause = build_fulltext_where(args)?;
|
|
||||||
Ok(format!(
|
|
||||||
"SELECT COUNT(*) FROM searchabletext AS st WHERE {}",
|
|
||||||
where_clause
|
|
||||||
))
|
|
||||||
}
|
|
||||||
"filename" => {
|
|
||||||
if args.term.trim().is_empty() {
|
|
||||||
return Err("Please enter a filename pattern".into());
|
|
||||||
}
|
|
||||||
Ok(format!(
|
|
||||||
"SELECT COUNT(*) FROM files WHERE name LIKE '%{}%'",
|
|
||||||
args.term.replace('\'', "''")
|
|
||||||
))
|
|
||||||
}
|
|
||||||
"duplicates" => Ok(
|
|
||||||
"SELECT COUNT(*) FROM (SELECT 1 FROM files WHERE hash IS NOT NULL \
|
|
||||||
GROUP BY hash HAVING count(*) > 1)"
|
|
||||||
.into(),
|
|
||||||
),
|
|
||||||
_ => Err("Unknown search type".into()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// SQL that returns one page of results. Columns emitted by the `fulltext`
|
|
||||||
/// branch are `(name, path, file_id, text_zstd)` — the snippet is rendered
|
|
||||||
/// in Rust from the zstd-compressed `documents_text` row (FTS5 is
|
|
||||||
/// contentless, so SQLite's `snippet()` doesn't work on it). The GUI
|
|
||||||
/// should use [`crate::indexing::IndexingService::execute_fulltext_search`]
|
|
||||||
/// which stitches the decompress + snippet step on top of this SQL.
|
|
||||||
pub fn build_select(args: &SearchArgs, limit: u32, offset: u32) -> Result<String, String> {
|
|
||||||
match args.search_type.as_str() {
|
|
||||||
"fulltext" => {
|
|
||||||
let where_clause = build_fulltext_where(args)?;
|
|
||||||
Ok(format!(
|
|
||||||
"SELECT f.name, f.path, f.id, dt.text_zstd \
|
|
||||||
FROM searchabletext AS st \
|
|
||||||
JOIN files f ON f.id = st.rowid \
|
|
||||||
LEFT JOIN documents_text dt ON dt.file_id = f.id \
|
|
||||||
WHERE {} ORDER BY rank LIMIT {} OFFSET {}",
|
|
||||||
where_clause, limit, offset
|
|
||||||
))
|
|
||||||
}
|
|
||||||
"filename" => {
|
|
||||||
if args.term.trim().is_empty() {
|
|
||||||
return Err("Please enter a filename pattern".into());
|
|
||||||
}
|
|
||||||
Ok(format!(
|
|
||||||
"SELECT name, path FROM files WHERE name LIKE '%{}%' ORDER BY name LIMIT {} OFFSET {}",
|
|
||||||
args.term.replace('\'', "''"),
|
|
||||||
limit,
|
|
||||||
offset
|
|
||||||
))
|
|
||||||
}
|
|
||||||
"duplicates" => Ok(format!(
|
|
||||||
"SELECT name, count(*) as cnt, path FROM files WHERE hash IS NOT NULL \
|
|
||||||
GROUP BY hash HAVING cnt > 1 ORDER BY cnt DESC LIMIT {} OFFSET {}",
|
|
||||||
limit, offset
|
|
||||||
)),
|
|
||||||
_ => Err("Unknown search type".into()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Translate the user-typed term into the FTS5 `MATCH` expression and any
|
|
||||||
/// supplemental case-sensitivity filters. Shared by count and select so
|
|
||||||
/// pagination doesn't accidentally diverge from the totals.
|
|
||||||
fn build_fulltext_where(args: &SearchArgs) -> Result<String, String> {
|
|
||||||
let trimmed = args.term.trim();
|
|
||||||
if trimmed.is_empty() {
|
|
||||||
return Err("Please enter a search term".into());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Strip FTS5 control characters that confuse the parser. Replace with
|
|
||||||
// spaces so word boundaries survive.
|
|
||||||
let sanitized: String = trimmed
|
|
||||||
.chars()
|
|
||||||
.map(|c| {
|
|
||||||
if matches!(
|
|
||||||
c,
|
|
||||||
':' | ';' | '(' | ')' | '[' | ']' | '{' | '}' | '^' | '~' | '"'
|
|
||||||
) {
|
|
||||||
' '
|
|
||||||
} else {
|
|
||||||
c
|
|
||||||
}
|
|
||||||
})
|
|
||||||
.collect();
|
|
||||||
|
|
||||||
let tokens: Vec<&str> = sanitized.split_whitespace().collect();
|
|
||||||
if tokens.is_empty() {
|
|
||||||
return Err("Please enter a valid search term".into());
|
|
||||||
}
|
|
||||||
|
|
||||||
// Trigram tokenizer needs each word to be at least 3 characters. Exact
|
|
||||||
// phrase mode skips this filter — a quoted phrase of short words still
|
|
||||||
// matches because the trigrams overlap the spaces between words.
|
|
||||||
let words: Vec<&str> = if args.fulltext_exact {
|
|
||||||
tokens
|
|
||||||
} else {
|
|
||||||
let filtered: Vec<&str> = tokens
|
|
||||||
.into_iter()
|
|
||||||
.filter(|w| w.chars().count() >= 3)
|
|
||||||
.collect();
|
|
||||||
if filtered.is_empty() {
|
|
||||||
return Err(
|
|
||||||
"Trigram index needs each word to be at least 3 characters unless you use exact phrase search."
|
|
||||||
.into(),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
filtered
|
|
||||||
};
|
|
||||||
|
|
||||||
let sql_quote = |s: &str| s.replace('\'', "''");
|
|
||||||
|
|
||||||
let fts_match = if args.fulltext_exact {
|
|
||||||
let phrase = words.join(" ");
|
|
||||||
format!("\"{}\"", phrase.replace('"', "\"\""))
|
|
||||||
} else {
|
|
||||||
words.join(" AND ")
|
|
||||||
};
|
|
||||||
|
|
||||||
// Contentless FTS5 doesn't store column text, so case-sensitive
|
|
||||||
// filtering can't live in SQL anymore. It's re-applied in
|
|
||||||
// `IndexingService::execute_fulltext_search` by checking the
|
|
||||||
// decompressed body text for literal-case matches before returning
|
|
||||||
// the row. The MATCH itself stays case-insensitive (tokenizer folds),
|
|
||||||
// which is the correct candidate-set for a post-filter.
|
|
||||||
let where_clause = format!("st.text MATCH '{}'", sql_quote(&fts_match));
|
|
||||||
Ok(where_clause)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Pull out the raw words the user typed so the snippet renderer and the
|
|
||||||
/// case-sensitive post-filter can see the same tokens `build_fulltext_where`
|
|
||||||
/// fed into FTS5. Returns empty when the user's term is empty or contains
|
|
||||||
/// only too-short words under the non-exact path.
|
|
||||||
pub fn fulltext_terms(args: &SearchArgs) -> Vec<String> {
|
|
||||||
let trimmed = args.term.trim();
|
|
||||||
if trimmed.is_empty() {
|
|
||||||
return Vec::new();
|
|
||||||
}
|
|
||||||
let sanitized: String = trimmed
|
|
||||||
.chars()
|
|
||||||
.map(|c| {
|
|
||||||
if matches!(
|
|
||||||
c,
|
|
||||||
':' | ';' | '(' | ')' | '[' | ']' | '{' | '}' | '^' | '~' | '"'
|
|
||||||
) {
|
|
||||||
' '
|
|
||||||
} else {
|
|
||||||
c
|
|
||||||
}
|
|
||||||
})
|
|
||||||
.collect();
|
|
||||||
let tokens: Vec<String> = sanitized
|
|
||||||
.split_whitespace()
|
|
||||||
.map(|s| s.to_string())
|
|
||||||
.collect();
|
|
||||||
if tokens.is_empty() {
|
|
||||||
return Vec::new();
|
|
||||||
}
|
|
||||||
if args.fulltext_exact {
|
|
||||||
// One composite phrase. Snippet rendering wants to highlight the
|
|
||||||
// whole phrase contiguously; the renderer supports multiple terms
|
|
||||||
// already so we collapse to the joined form.
|
|
||||||
vec![tokens.join(" ")]
|
|
||||||
} else {
|
|
||||||
tokens
|
|
||||||
.into_iter()
|
|
||||||
.filter(|w| w.chars().count() >= 3)
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use crate::db::{open_or_recreate, repo::{insert_file, set_content_done, NewFile}};
|
|
||||||
use crate::mime::FileType;
|
|
||||||
|
|
||||||
fn args(search_type: &str, term: &str) -> SearchArgs {
|
|
||||||
SearchArgs {
|
|
||||||
search_type: search_type.into(),
|
|
||||||
term: term.into(),
|
|
||||||
fulltext_exact: false,
|
|
||||||
fulltext_case_sensitive: false,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_select_has_limit_and_offset() {
|
|
||||||
let sql = build_select(&args("fulltext", "hello world"), 50, 100).unwrap();
|
|
||||||
assert!(sql.contains("LIMIT 50"));
|
|
||||||
assert!(sql.contains("OFFSET 100"));
|
|
||||||
assert!(sql.contains("ORDER BY rank"));
|
|
||||||
// Snippet rendering moved to Rust; SQL returns the compressed blob
|
|
||||||
// so the post-processor can decompress + highlight.
|
|
||||||
assert!(sql.contains("dt.text_zstd"), "got {sql}");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_case_sensitive_no_longer_in_sql() {
|
|
||||||
let mut a = args("fulltext", "Hello");
|
|
||||||
a.fulltext_case_sensitive = true;
|
|
||||||
let sql = build_select(&a, 50, 0).unwrap();
|
|
||||||
// Post-filter lives in Rust now — no instr() or st.text reference.
|
|
||||||
assert!(!sql.contains("instr("), "got {sql}");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_terms_extract_non_exact() {
|
|
||||||
let a = args("fulltext", "the quick brown");
|
|
||||||
let t = fulltext_terms(&a);
|
|
||||||
// Short words like "the" are dropped (trigram min length 3 applies
|
|
||||||
// in non-exact mode — matches the SQL build rules).
|
|
||||||
assert!(t.iter().any(|s| s == "quick"));
|
|
||||||
assert!(t.iter().any(|s| s == "brown"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_terms_exact_mode_returns_joined_phrase() {
|
|
||||||
let mut a = args("fulltext", "hello world");
|
|
||||||
a.fulltext_exact = true;
|
|
||||||
let t = fulltext_terms(&a);
|
|
||||||
assert_eq!(t, vec!["hello world".to_string()]);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_count_lacks_pagination_and_join() {
|
|
||||||
let sql = build_count(&args("fulltext", "hello world")).unwrap();
|
|
||||||
assert!(sql.starts_with("SELECT COUNT(*)"));
|
|
||||||
assert!(!sql.contains("LIMIT"));
|
|
||||||
assert!(!sql.contains("OFFSET"));
|
|
||||||
assert!(!sql.contains("JOIN files"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_short_words_filtered_unless_exact() {
|
|
||||||
let err = build_select(&args("fulltext", "a b"), 50, 0).unwrap_err();
|
|
||||||
assert!(err.contains("3 characters"));
|
|
||||||
let mut a = args("fulltext", "a b");
|
|
||||||
a.fulltext_exact = true;
|
|
||||||
assert!(build_select(&a, 50, 0).is_ok());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn fulltext_quotes_are_escaped() {
|
|
||||||
let sql = build_select(&args("fulltext", "it's working"), 50, 0).unwrap();
|
|
||||||
// SQL literals double single quotes.
|
|
||||||
assert!(sql.contains("it''s"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn filename_select_has_limit_offset_and_order() {
|
|
||||||
let sql = build_select(&args("filename", "report"), 50, 0).unwrap();
|
|
||||||
assert!(sql.contains("LIMIT 50"));
|
|
||||||
assert!(sql.contains("OFFSET 0"));
|
|
||||||
assert!(sql.contains("ORDER BY name"));
|
|
||||||
assert!(sql.contains("name LIKE '%report%'"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn filename_empty_term_errors() {
|
|
||||||
assert!(build_select(&args("filename", " "), 50, 0).is_err());
|
|
||||||
assert!(build_count(&args("filename", "")).is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn duplicates_select_has_limit_offset() {
|
|
||||||
let sql = build_select(&args("duplicates", ""), 50, 100).unwrap();
|
|
||||||
assert!(sql.contains("LIMIT 50"));
|
|
||||||
assert!(sql.contains("OFFSET 100"));
|
|
||||||
assert!(sql.contains("GROUP BY hash"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn unknown_search_type_errors() {
|
|
||||||
assert!(build_select(&args("nope", ""), 50, 0).is_err());
|
|
||||||
assert!(build_count(&args("nope", "")).is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
fn tmp_path() -> std::path::PathBuf {
|
|
||||||
let mut p = std::env::temp_dir();
|
|
||||||
p.push(format!(
|
|
||||||
"qs-search-sql-{}-{}.sqlite",
|
|
||||||
std::process::id(),
|
|
||||||
std::time::SystemTime::now()
|
|
||||||
.duration_since(std::time::UNIX_EPOCH)
|
|
||||||
.unwrap()
|
|
||||||
.as_nanos()
|
|
||||||
));
|
|
||||||
p
|
|
||||||
}
|
|
||||||
|
|
||||||
/// End-to-end: seed three rows, run count + paged select, verify
|
|
||||||
/// pagination boundaries actually behave on a real DB.
|
|
||||||
#[test]
|
|
||||||
fn end_to_end_pagination_smoke() {
|
|
||||||
let p = tmp_path();
|
|
||||||
let mut conn = open_or_recreate(p.to_str().unwrap(), "trigram").unwrap();
|
|
||||||
{
|
|
||||||
let tx = conn.transaction().unwrap();
|
|
||||||
for i in 0..7 {
|
|
||||||
let path = format!("/tmp/file_{}.txt", i);
|
|
||||||
let id = insert_file(
|
|
||||||
&tx,
|
|
||||||
&NewFile {
|
|
||||||
name: &format!("file_{}.txt", i),
|
|
||||||
path: &path,
|
|
||||||
parent: "/tmp",
|
|
||||||
size: 1,
|
|
||||||
mtime: 1,
|
|
||||||
inode: None,
|
|
||||||
device_id: None,
|
|
||||||
mime: Some("text/plain"),
|
|
||||||
ftype: FileType::TEXT,
|
|
||||||
hash: None,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
.unwrap()
|
|
||||||
.expect("unique path");
|
|
||||||
set_content_done(&tx, id, &format!("file_{}.txt", i), "shared body content", &[], true).unwrap();
|
|
||||||
}
|
|
||||||
tx.commit().unwrap();
|
|
||||||
}
|
|
||||||
|
|
||||||
// Count: all 7 rows match "shared".
|
|
||||||
let count_sql = build_count(&args("fulltext", "shared body content")).unwrap();
|
|
||||||
let n: i64 = conn.query_row(&count_sql, [], |r| r.get(0)).unwrap();
|
|
||||||
assert_eq!(n, 7);
|
|
||||||
|
|
||||||
// Page 1, page_size 3 → 3 rows.
|
|
||||||
let sel1 = build_select(&args("fulltext", "shared body content"), 3, 0).unwrap();
|
|
||||||
let rows1: Vec<String> = conn
|
|
||||||
.prepare(&sel1)
|
|
||||||
.unwrap()
|
|
||||||
.query_map([], |r| r.get::<_, String>(1))
|
|
||||||
.unwrap()
|
|
||||||
.map(|r| r.unwrap())
|
|
||||||
.collect();
|
|
||||||
assert_eq!(rows1.len(), 3);
|
|
||||||
|
|
||||||
// Page 3, page_size 3 → 1 row (offset 6, 7 total).
|
|
||||||
let sel3 = build_select(&args("fulltext", "shared body content"), 3, 6).unwrap();
|
|
||||||
let rows3: Vec<String> = conn
|
|
||||||
.prepare(&sel3)
|
|
||||||
.unwrap()
|
|
||||||
.query_map([], |r| r.get::<_, String>(1))
|
|
||||||
.unwrap()
|
|
||||||
.map(|r| r.unwrap())
|
|
||||||
.collect();
|
|
||||||
assert_eq!(rows3.len(), 1);
|
|
||||||
|
|
||||||
// Filename pagination on the same fixture.
|
|
||||||
let fn_count = build_count(&args("filename", "file_")).unwrap();
|
|
||||||
let n2: i64 = conn.query_row(&fn_count, [], |r| r.get(0)).unwrap();
|
|
||||||
assert_eq!(n2, 7);
|
|
||||||
|
|
||||||
let fn_sel = build_select(&args("filename", "file_"), 5, 0).unwrap();
|
|
||||||
let rows: Vec<String> = conn
|
|
||||||
.prepare(&fn_sel)
|
|
||||||
.unwrap()
|
|
||||||
.query_map([], |r| r.get::<_, String>(1))
|
|
||||||
.unwrap()
|
|
||||||
.map(|r| r.unwrap())
|
|
||||||
.collect();
|
|
||||||
assert_eq!(rows.len(), 5);
|
|
||||||
|
|
||||||
drop(conn);
|
|
||||||
std::fs::remove_file(&p).ok();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,26 +1,23 @@
|
||||||
//! Process-level shutdown helpers.
|
//! Process-level shutdown helpers.
|
||||||
//!
|
//!
|
||||||
//! Wires Ctrl-C (and on Unix, SIGTERM) to a graceful shutdown that flushes
|
//! Wires Ctrl-C (and on Unix, SIGTERM) to a graceful shutdown that stops
|
||||||
//! the indexing DB and exits. Replaces the ad-hoc `ctrlc::set_handler` +
|
//! the watcher, aborts any running index pass, flushes the WAL, and
|
||||||
//! `OnceLock` dance the GUI used to carry. Call
|
//! exits. Call [`install_signal_handler`] once from a binary's `main`
|
||||||
//! [`install_signal_handler`] once from a binary's `main` with a cloned
|
//! with a cloned [`IndexCoordinator`] handle.
|
||||||
//! [`IndexingService`] handle.
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use crate::indexing::IndexingService;
|
use crate::coordinator::IndexCoordinator;
|
||||||
|
|
||||||
/// Install a Ctrl-C (and, where supported, SIGTERM) handler that calls
|
/// Install a Ctrl-C (and, where supported, SIGTERM) handler that calls
|
||||||
/// [`IndexingService::graceful_shutdown`] and then exits with status 0.
|
/// [`IndexCoordinator::shutdown`] and then exits with status 0.
|
||||||
///
|
///
|
||||||
/// Returns an error only if a handler was already installed elsewhere in
|
/// Returns an error only if a handler was already installed elsewhere in
|
||||||
/// this process (ctrlc::set_handler is one-shot).
|
/// this process (ctrlc::set_handler is one-shot).
|
||||||
pub fn install_signal_handler(service: Arc<IndexingService>) -> Result<(), String> {
|
pub fn install_signal_handler(coordinator: Arc<IndexCoordinator>) -> Result<(), String> {
|
||||||
ctrlc::set_handler(move || {
|
ctrlc::set_handler(move || {
|
||||||
eprintln!("Received Ctrl-C, shutting down gracefully...");
|
crate::log_info!("Received Ctrl-C, shutting down gracefully...");
|
||||||
if let Err(e) = service.graceful_shutdown() {
|
coordinator.shutdown();
|
||||||
eprintln!("Error during graceful shutdown: {}", e);
|
|
||||||
}
|
|
||||||
std::process::exit(0);
|
std::process::exit(0);
|
||||||
})
|
})
|
||||||
.map_err(|e| format!("install signal handler: {}", e))
|
.map_err(|e| format!("install signal handler: {}", e))
|
||||||
|
|
|
||||||
|
|
@ -1,60 +1,70 @@
|
||||||
//! Snippet / highlight rendering for search results.
|
//! Snippet extraction for search results.
|
||||||
//!
|
//!
|
||||||
//! We store extracted text in the `documents_text` sidecar (zstd-compressed)
|
//! We store extracted text in the `documents_text` sidecar (zstd-compressed)
|
||||||
//! rather than in FTS5, so SQLite's built-in `snippet()` / `highlight()`
|
//! rather than in FTS5, so SQLite's built-in `snippet()` / `highlight()`
|
||||||
//! auxiliary functions aren't available (contentless FTS5 doesn't support
|
//! auxiliary functions aren't available (contentless FTS5 doesn't support
|
||||||
//! them). This module reproduces the parts we actually need in Rust: find
|
//! them). This module reproduces the parts we actually need in Rust: find a
|
||||||
//! a window of context around the first match, bold every query-term
|
//! window of context around the first match and report every query-term
|
||||||
//! occurrence inside that window, trim with ellipsis markers.
|
//! occurrence inside that window.
|
||||||
//!
|
//!
|
||||||
//! Matching is ASCII-case-insensitive on the *rendering* side. That aligns
|
//! Output is *structural* — the window text plus byte ranges of the matches
|
||||||
//! with the search path which is already case-insensitive via the trigram
|
//! within it — so any frontend can render highlights natively (egui builds
|
||||||
//! tokenizer; exact-case-only snippets aren't a feature users expect here.
|
//! a `LayoutJob`, the CLI emits ANSI bold). Nothing here produces markup.
|
||||||
//! Unicode accent folding isn't applied at the rendering layer — a query
|
|
||||||
//! for `cafe` will still *find* a file containing `café` (because the FTS
|
|
||||||
//! tokenizer strips diacritics) but the snippet won't highlight the
|
|
||||||
//! accented occurrence. The surrounding text is still returned verbatim.
|
|
||||||
//!
|
//!
|
||||||
//! The API is intentionally small: one `render` function plus an `Options`
|
//! Matching is ASCII-case-insensitive. That aligns with the search path,
|
||||||
//! struct. Callers that want different pre/post tags, ellipsis, or window
|
//! which is already case-insensitive via the trigram tokenizer; Unicode
|
||||||
//! size pass them in; there are sensible defaults for the GUI case.
|
//! accent folding isn't applied at this layer — a query for `cafe` will
|
||||||
|
//! still *find* a file containing `café` (the FTS tokenizer strips
|
||||||
|
//! diacritics) but the snippet won't mark the accented occurrence. The
|
||||||
|
//! window text is returned verbatim either way.
|
||||||
|
|
||||||
/// Options controlling snippet rendering. The defaults mirror the old
|
/// Options controlling snippet extraction.
|
||||||
/// `snippet(searchabletext, 1, '<b>', '</b>', '<b>...</b>', 64)` call that
|
|
||||||
/// the GUI used to run directly as SQL.
|
|
||||||
#[derive(Debug, Clone)]
|
#[derive(Debug, Clone)]
|
||||||
pub struct Options<'a> {
|
pub struct Options {
|
||||||
pub pre: &'a str,
|
/// Approximate byte budget for the returned window. Matches expand the
|
||||||
pub post: &'a str,
|
/// window if needed so a hit is never cut off; the budget is a soft
|
||||||
pub ellipsis: &'a str,
|
/// target, not a hard cap.
|
||||||
/// Approximate character budget for the returned snippet. Matches
|
|
||||||
/// expand the window if needed to keep their tags on; the budget is a
|
|
||||||
/// soft target, not a hard cap.
|
|
||||||
pub approx_chars: usize,
|
pub approx_chars: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl<'a> Default for Options<'a> {
|
impl Default for Options {
|
||||||
fn default() -> Self {
|
fn default() -> Self {
|
||||||
Options {
|
Options { approx_chars: 200 }
|
||||||
pre: "<b>",
|
}
|
||||||
post: "</b>",
|
}
|
||||||
ellipsis: "<b>...</b>",
|
|
||||||
approx_chars: 200,
|
/// A context window from a document plus the match positions inside it.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct Snippet {
|
||||||
|
/// The excerpt, sliced verbatim from the source text on UTF-8 char
|
||||||
|
/// boundaries.
|
||||||
|
pub window: String,
|
||||||
|
/// Byte ranges *into `window`*, sorted, non-overlapping (overlapping
|
||||||
|
/// term hits are coalesced), always on char boundaries.
|
||||||
|
pub ranges: Vec<(usize, usize)>,
|
||||||
|
/// Content exists before/after the window — frontends render their own
|
||||||
|
/// ellipsis.
|
||||||
|
pub truncated_start: bool,
|
||||||
|
pub truncated_end: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Snippet {
|
||||||
|
fn empty() -> Snippet {
|
||||||
|
Snippet {
|
||||||
|
window: String::new(),
|
||||||
|
ranges: Vec::new(),
|
||||||
|
truncated_start: false,
|
||||||
|
truncated_end: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Render a snippet from `text` highlighting every occurrence of any term
|
/// Extract a snippet from `text` marking every occurrence of any term in
|
||||||
/// in `terms`. Returns a string with `pre`/`post` wrapping each match, and
|
/// `terms` (ASCII-case-insensitive). With no terms or no matches, returns
|
||||||
/// `ellipsis` prepended/appended when the returned window doesn't reach
|
/// the head of the text as the window with no ranges.
|
||||||
/// the text's edges.
|
pub fn extract(text: &str, terms: &[&str], opts: &Options) -> Snippet {
|
||||||
///
|
|
||||||
/// If no term matches, returns the first `approx_chars` of `text` (char-
|
|
||||||
/// aligned), suffixed with `ellipsis` when truncated.
|
|
||||||
pub fn render(text: &str, terms: &[&str], opts: &Options<'_>) -> String {
|
|
||||||
// Short-circuit trivial inputs.
|
|
||||||
if text.is_empty() {
|
if text.is_empty() {
|
||||||
return String::new();
|
return Snippet::empty();
|
||||||
}
|
}
|
||||||
let effective_terms: Vec<&str> = terms
|
let effective_terms: Vec<&str> = terms
|
||||||
.iter()
|
.iter()
|
||||||
|
|
@ -62,32 +72,24 @@ pub fn render(text: &str, terms: &[&str], opts: &Options<'_>) -> String {
|
||||||
.filter(|t| !t.is_empty())
|
.filter(|t| !t.is_empty())
|
||||||
.collect();
|
.collect();
|
||||||
if effective_terms.is_empty() {
|
if effective_terms.is_empty() {
|
||||||
return truncate_head(text, opts.approx_chars, opts.ellipsis);
|
return head_window(text, opts.approx_chars);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Case-fold once; we do all positioning on the folded buffer and emit
|
// Case-fold once; we do all positioning on the folded buffer and slice
|
||||||
// slices from the original. Both buffers have identical byte layout
|
// from the original. Both buffers have identical byte layout because
|
||||||
// because `to_ascii_lowercase` is a byte-for-byte map that preserves
|
// `to_ascii_lowercase` only touches ASCII letters.
|
||||||
// multi-byte UTF-8 sequences unchanged (it only touches ASCII letters).
|
|
||||||
let folded = text.to_ascii_lowercase();
|
let folded = text.to_ascii_lowercase();
|
||||||
let folded_bytes = folded.as_bytes();
|
let folded_bytes = folded.as_bytes();
|
||||||
|
|
||||||
let mut matches: Vec<(usize, usize)> = Vec::new();
|
let mut matches: Vec<(usize, usize)> = Vec::new();
|
||||||
for term in &effective_terms {
|
for term in &effective_terms {
|
||||||
let pattern: String = term.to_ascii_lowercase();
|
let pattern = term.to_ascii_lowercase();
|
||||||
let pbytes = pattern.as_bytes();
|
let pbytes = pattern.as_bytes();
|
||||||
if pbytes.is_empty() {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
let mut start = 0;
|
let mut start = 0;
|
||||||
while start + pbytes.len() <= folded_bytes.len() {
|
while start + pbytes.len() <= folded_bytes.len() {
|
||||||
if let Some(rel) = memfind(&folded_bytes[start..], pbytes) {
|
if let Some(rel) = memfind(&folded_bytes[start..], pbytes) {
|
||||||
let at = start + rel;
|
let at = start + rel;
|
||||||
matches.push((at, at + pbytes.len()));
|
matches.push((at, at + pbytes.len()));
|
||||||
// Advance past this match to avoid zero-width loops on
|
|
||||||
// empty patterns (already guarded above) and to allow
|
|
||||||
// overlapping matches of *different* terms in the next
|
|
||||||
// outer-loop iteration.
|
|
||||||
start = at + pbytes.len();
|
start = at + pbytes.len();
|
||||||
} else {
|
} else {
|
||||||
break;
|
break;
|
||||||
|
|
@ -96,24 +98,18 @@ pub fn render(text: &str, terms: &[&str], opts: &Options<'_>) -> String {
|
||||||
}
|
}
|
||||||
|
|
||||||
if matches.is_empty() {
|
if matches.is_empty() {
|
||||||
return truncate_head(text, opts.approx_chars, opts.ellipsis);
|
return head_window(text, opts.approx_chars);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Dedupe + sort so overlapping matches from different terms (e.g.
|
|
||||||
// "rust" and "rustc") don't produce nested tags.
|
|
||||||
matches.sort_by_key(|(a, _)| *a);
|
matches.sort_by_key(|(a, _)| *a);
|
||||||
matches = coalesce_overlapping(matches);
|
let matches = coalesce_overlapping(matches);
|
||||||
|
|
||||||
// Pick the window. Start a third of the budget before the first match
|
// Pick the window. Start a third of the budget before the first match
|
||||||
// so the hit isn't pinned to the left edge. Round both ends to char
|
// so the hit isn't pinned to the left edge; round both ends to char
|
||||||
// boundaries so we never slice a multi-byte UTF-8 sequence.
|
// boundaries so we never slice a multi-byte UTF-8 sequence.
|
||||||
let pre_pad = opts.approx_chars / 3;
|
let pre_pad = opts.approx_chars / 3;
|
||||||
let first_match_start = matches[0].0;
|
let mut win_start = matches[0].0.saturating_sub(pre_pad);
|
||||||
let mut win_start = first_match_start.saturating_sub(pre_pad);
|
let mut win_end = (win_start + opts.approx_chars).min(text.len());
|
||||||
let mut win_end = win_start + opts.approx_chars;
|
|
||||||
if win_end > text.len() {
|
|
||||||
win_end = text.len();
|
|
||||||
}
|
|
||||||
while win_start > 0 && !text.is_char_boundary(win_start) {
|
while win_start > 0 && !text.is_char_boundary(win_start) {
|
||||||
win_start -= 1;
|
win_start -= 1;
|
||||||
}
|
}
|
||||||
|
|
@ -121,11 +117,9 @@ pub fn render(text: &str, terms: &[&str], opts: &Options<'_>) -> String {
|
||||||
win_end += 1;
|
win_end += 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Expand the window to include the full end of any match that would
|
// Expand the window so a match straddling the right edge is fully
|
||||||
// otherwise be cut off mid-tag. Keeps rendering sane when a long term
|
// included rather than cut mid-hit.
|
||||||
// sits at the right edge of the budget.
|
if let Some((_, end)) = matches.iter().rfind(|(s, _)| *s < win_end) {
|
||||||
let last_match_in_window = matches.iter().rfind(|(s, _)| *s < win_end);
|
|
||||||
if let Some((_, end)) = last_match_in_window {
|
|
||||||
if *end > win_end {
|
if *end > win_end {
|
||||||
win_end = *end;
|
win_end = *end;
|
||||||
while win_end < text.len() && !text.is_char_boundary(win_end) {
|
while win_end < text.len() && !text.is_char_boundary(win_end) {
|
||||||
|
|
@ -134,55 +128,114 @@ pub fn render(text: &str, terms: &[&str], opts: &Options<'_>) -> String {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Render: walk matches that fall inside the window, splicing pre/post
|
let ranges = matches
|
||||||
// around each. Prepend/append ellipsis when we've chopped off content.
|
.iter()
|
||||||
let mut out = String::with_capacity(win_end - win_start + 32);
|
.filter(|(s, e)| *e > win_start && *s < win_end)
|
||||||
if win_start > 0 {
|
.map(|(s, e)| ((*s).max(win_start) - win_start, (*e).min(win_end) - win_start))
|
||||||
out.push_str(opts.ellipsis);
|
.collect();
|
||||||
|
|
||||||
|
Snippet {
|
||||||
|
window: text[win_start..win_end].to_string(),
|
||||||
|
ranges,
|
||||||
|
truncated_start: win_start > 0,
|
||||||
|
truncated_end: win_end < text.len(),
|
||||||
}
|
}
|
||||||
let mut cursor = win_start;
|
|
||||||
for (ms, me) in matches.iter() {
|
|
||||||
if *me <= win_start || *ms >= win_end {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// Clamp to the window.
|
|
||||||
let ms = (*ms).max(win_start);
|
|
||||||
let me = (*me).min(win_end);
|
|
||||||
if ms > cursor {
|
|
||||||
out.push_str(&text[cursor..ms]);
|
|
||||||
}
|
|
||||||
out.push_str(opts.pre);
|
|
||||||
out.push_str(&text[ms..me]);
|
|
||||||
out.push_str(opts.post);
|
|
||||||
cursor = me;
|
|
||||||
}
|
|
||||||
if cursor < win_end {
|
|
||||||
out.push_str(&text[cursor..win_end]);
|
|
||||||
}
|
|
||||||
if win_end < text.len() {
|
|
||||||
out.push_str(opts.ellipsis);
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Return the first `n` characters of `text`, suffixed with `ellipsis` if
|
/// Build a snippet window around one known match range in `text` (byte
|
||||||
/// truncation actually happened. Respects UTF-8 char boundaries.
|
/// offsets into `text`). Used by fuzzy full-text search, where the match
|
||||||
fn truncate_head(text: &str, n: usize, ellipsis: &str) -> String {
|
/// was located by the fuzzy matcher rather than exact term search. The
|
||||||
|
/// range is clamped and char-boundary-aligned defensively.
|
||||||
|
pub fn window_around(text: &str, range: (usize, usize), opts: &Options) -> Snippet {
|
||||||
|
if text.is_empty() {
|
||||||
|
return Snippet::empty();
|
||||||
|
}
|
||||||
|
let (mut ms, mut me) = range;
|
||||||
|
ms = ms.min(text.len());
|
||||||
|
me = me.clamp(ms, text.len());
|
||||||
|
while ms > 0 && !text.is_char_boundary(ms) {
|
||||||
|
ms -= 1;
|
||||||
|
}
|
||||||
|
while me < text.len() && !text.is_char_boundary(me) {
|
||||||
|
me += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
let pre_pad = opts.approx_chars / 3;
|
||||||
|
let mut win_start = ms.saturating_sub(pre_pad);
|
||||||
|
let mut win_end = (win_start + opts.approx_chars).max(me).min(text.len());
|
||||||
|
while win_start > 0 && !text.is_char_boundary(win_start) {
|
||||||
|
win_start -= 1;
|
||||||
|
}
|
||||||
|
while win_end < text.len() && !text.is_char_boundary(win_end) {
|
||||||
|
win_end += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
let ranges = if me > ms {
|
||||||
|
vec![(ms - win_start, me - win_start)]
|
||||||
|
} else {
|
||||||
|
Vec::new()
|
||||||
|
};
|
||||||
|
Snippet {
|
||||||
|
window: text[win_start..win_end].to_string(),
|
||||||
|
ranges,
|
||||||
|
truncated_start: win_start > 0,
|
||||||
|
truncated_end: win_end < text.len(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Count non-overlapping occurrences of `term` in `text`. Empty terms count
|
||||||
|
/// zero. Case-insensitive counting folds ASCII only, matching the rest of
|
||||||
|
/// the search pipeline.
|
||||||
|
pub fn count_occurrences(text: &str, term: &str, case_sensitive: bool) -> usize {
|
||||||
|
if term.is_empty() || text.len() < term.len() {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
let (hay, needle);
|
||||||
|
let (hay_ref, needle_ref): (&[u8], &[u8]) = if case_sensitive {
|
||||||
|
(text.as_bytes(), term.as_bytes())
|
||||||
|
} else {
|
||||||
|
hay = text.to_ascii_lowercase();
|
||||||
|
needle = term.to_ascii_lowercase();
|
||||||
|
(hay.as_bytes(), needle.as_bytes())
|
||||||
|
};
|
||||||
|
let mut count = 0;
|
||||||
|
let mut start = 0;
|
||||||
|
while start + needle_ref.len() <= hay_ref.len() {
|
||||||
|
match memfind(&hay_ref[start..], needle_ref) {
|
||||||
|
Some(rel) => {
|
||||||
|
count += 1;
|
||||||
|
start += rel + needle_ref.len();
|
||||||
|
}
|
||||||
|
None => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
count
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The first `n` bytes of `text` (char-aligned) as a match-less window.
|
||||||
|
fn head_window(text: &str, n: usize) -> Snippet {
|
||||||
if text.len() <= n {
|
if text.len() <= n {
|
||||||
return text.to_string();
|
return Snippet {
|
||||||
|
window: text.to_string(),
|
||||||
|
ranges: Vec::new(),
|
||||||
|
truncated_start: false,
|
||||||
|
truncated_end: false,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
let mut cut = n;
|
let mut cut = n;
|
||||||
while cut > 0 && !text.is_char_boundary(cut) {
|
while cut > 0 && !text.is_char_boundary(cut) {
|
||||||
cut -= 1;
|
cut -= 1;
|
||||||
}
|
}
|
||||||
let mut out = String::with_capacity(cut + ellipsis.len());
|
Snippet {
|
||||||
out.push_str(&text[..cut]);
|
window: text[..cut].to_string(),
|
||||||
out.push_str(ellipsis);
|
ranges: Vec::new(),
|
||||||
out
|
truncated_start: false,
|
||||||
|
truncated_end: true,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Merge adjacent / overlapping (start, end) ranges in place. Input must be
|
/// Merge adjacent / overlapping (start, end) ranges. Input must be sorted
|
||||||
/// sorted by start.
|
/// by start.
|
||||||
fn coalesce_overlapping(mut v: Vec<(usize, usize)>) -> Vec<(usize, usize)> {
|
fn coalesce_overlapping(mut v: Vec<(usize, usize)>) -> Vec<(usize, usize)> {
|
||||||
if v.len() < 2 {
|
if v.len() < 2 {
|
||||||
return v;
|
return v;
|
||||||
|
|
@ -202,7 +255,7 @@ fn coalesce_overlapping(mut v: Vec<(usize, usize)>) -> Vec<(usize, usize)> {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Locate the first occurrence of `needle` in `hay`. A byte-level search;
|
/// Locate the first occurrence of `needle` in `hay`. A byte-level search;
|
||||||
/// callers have already lowercased both sides so case is normalized.
|
/// callers have already normalized case where needed.
|
||||||
fn memfind(hay: &[u8], needle: &[u8]) -> Option<usize> {
|
fn memfind(hay: &[u8], needle: &[u8]) -> Option<usize> {
|
||||||
if needle.is_empty() || needle.len() > hay.len() {
|
if needle.is_empty() || needle.len() > hay.len() {
|
||||||
return None;
|
return None;
|
||||||
|
|
@ -222,115 +275,170 @@ fn memfind(hay: &[u8], needle: &[u8]) -> Option<usize> {
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
fn opts_small() -> Options<'static> {
|
fn opts_small() -> Options {
|
||||||
Options {
|
Options { approx_chars: 40 }
|
||||||
pre: "<b>",
|
|
||||||
post: "</b>",
|
|
||||||
ellipsis: "…",
|
|
||||||
approx_chars: 40,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Every range must be in-bounds, ordered, non-overlapping, and sit on
|
||||||
|
/// char boundaries — the contract egui's LayoutJob sections rely on.
|
||||||
|
fn assert_ranges_valid(s: &Snippet) {
|
||||||
|
let mut prev_end = 0;
|
||||||
|
for &(a, b) in &s.ranges {
|
||||||
|
assert!(a < b, "empty/inverted range {:?}", (a, b));
|
||||||
|
assert!(b <= s.window.len(), "range {:?} beyond window", (a, b));
|
||||||
|
assert!(a >= prev_end, "overlapping ranges");
|
||||||
|
assert!(s.window.is_char_boundary(a) && s.window.is_char_boundary(b));
|
||||||
|
prev_end = b;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn marked(s: &Snippet) -> Vec<&str> {
|
||||||
|
s.ranges.iter().map(|&(a, b)| &s.window[a..b]).collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn empty_text_returns_empty() {
|
fn empty_text_returns_empty() {
|
||||||
let s = render("", &["foo"], &Options::default());
|
let s = extract("", &["foo"], &Options::default());
|
||||||
assert_eq!(s, "");
|
assert_eq!(s, Snippet::empty());
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn no_terms_returns_head_with_ellipsis_when_truncated() {
|
fn no_terms_returns_head_marked_truncated() {
|
||||||
let long = "abcdefghijklmnop".repeat(10);
|
let long = "abcdefghijklmnop".repeat(10);
|
||||||
let s = render(&long, &[], &opts_small());
|
let s = extract(&long, &[], &opts_small());
|
||||||
assert!(s.ends_with("…"));
|
assert!(s.truncated_end);
|
||||||
assert!(s.len() < long.len() + 4);
|
assert!(!s.truncated_start);
|
||||||
|
assert!(s.ranges.is_empty());
|
||||||
|
assert!(s.window.len() <= 40);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn no_terms_untruncated_has_no_ellipsis() {
|
fn no_terms_untruncated() {
|
||||||
let s = render("short text", &[], &opts_small());
|
let s = extract("short text", &[], &opts_small());
|
||||||
assert_eq!(s, "short text");
|
assert_eq!(s.window, "short text");
|
||||||
|
assert!(!s.truncated_end && !s.truncated_start);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn simple_highlight_wraps_matches() {
|
fn simple_match_range() {
|
||||||
let s = render("the quick brown fox", &["quick"], &opts_small());
|
let s = extract("the quick brown fox", &["quick"], &opts_small());
|
||||||
assert!(s.contains("<b>quick</b>"));
|
assert_eq!(marked(&s), vec!["quick"]);
|
||||||
|
assert_ranges_valid(&s);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn case_insensitive_match() {
|
fn case_insensitive_match_reports_original_case() {
|
||||||
let s = render("The QUICK brown fox", &["quick"], &opts_small());
|
let s = extract("The QUICK brown fox", &["quick"], &opts_small());
|
||||||
assert!(s.contains("<b>QUICK</b>"), "got {s}");
|
assert_eq!(marked(&s), vec!["QUICK"]);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn multiple_terms_both_highlighted() {
|
fn multiple_terms_both_marked() {
|
||||||
let s = render(
|
let s = extract(
|
||||||
"the quick brown fox jumps over the lazy dog",
|
"the quick brown fox jumps over the lazy dog",
|
||||||
&["quick", "lazy"],
|
&["quick", "lazy"],
|
||||||
&Options {
|
&Options::default(),
|
||||||
approx_chars: 200,
|
|
||||||
..Options::default()
|
|
||||||
},
|
|
||||||
);
|
);
|
||||||
assert!(s.contains("<b>quick</b>"), "got {s}");
|
assert_eq!(marked(&s), vec!["quick", "lazy"]);
|
||||||
assert!(s.contains("<b>lazy</b>"), "got {s}");
|
assert_ranges_valid(&s);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn window_trims_with_ellipsis_on_both_sides() {
|
fn window_truncation_flags_on_both_sides() {
|
||||||
let text =
|
let text = "prefix ".repeat(20) + "MATCH in middle " + &"suffix ".repeat(20);
|
||||||
"prefix ".repeat(20) + "MATCH in middle " + &"suffix ".repeat(20);
|
let s = extract(&text, &["MATCH"], &opts_small());
|
||||||
let s = render(&text, &["MATCH"], &opts_small());
|
assert!(s.truncated_start);
|
||||||
assert!(s.starts_with("…"), "got {s}");
|
assert!(s.truncated_end);
|
||||||
assert!(s.ends_with("…"), "got {s}");
|
assert_eq!(marked(&s), vec!["MATCH"]);
|
||||||
assert!(s.contains("<b>MATCH</b>"), "got {s}");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn match_at_start_has_no_leading_ellipsis() {
|
fn match_at_start_not_truncated_left() {
|
||||||
let s = render("MATCH right at the start of this paragraph", &["match"], &opts_small());
|
let s = extract(
|
||||||
assert!(!s.starts_with("…"), "got {s}");
|
"MATCH right at the start of this paragraph",
|
||||||
|
&["match"],
|
||||||
|
&opts_small(),
|
||||||
|
);
|
||||||
|
assert!(!s.truncated_start);
|
||||||
|
assert_eq!(s.ranges[0].0, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn no_match_on_tail_returns_head() {
|
fn no_match_returns_head_without_ranges() {
|
||||||
let text = "alpha beta gamma delta epsilon zeta eta theta iota kappa";
|
let text = "alpha beta gamma delta epsilon zeta eta theta iota kappa";
|
||||||
let s = render(text, &["nomatch"], &opts_small());
|
let s = extract(text, &["nomatch"], &opts_small());
|
||||||
assert!(!s.contains("<b>"));
|
assert!(s.ranges.is_empty());
|
||||||
assert!(s.starts_with("alpha"));
|
assert!(s.window.starts_with("alpha"));
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn overlapping_terms_do_not_nest_tags() {
|
fn overlapping_terms_coalesce() {
|
||||||
// Two terms matching the same span must coalesce.
|
let s = extract("the RUSTC compiler", &["rust", "rustc"], &opts_small());
|
||||||
let s = render("the RUSTC compiler", &["rust", "rustc"], &opts_small());
|
assert_eq!(marked(&s), vec!["RUSTC"]);
|
||||||
assert!(s.contains("<b>RUSTC</b>"), "got {s}");
|
assert_ranges_valid(&s);
|
||||||
// No nested <b> tags.
|
|
||||||
assert!(!s.contains("<b><b>"), "got {s}");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn utf8_boundary_safe_truncation() {
|
fn utf8_boundaries_hold_with_multibyte_text() {
|
||||||
// Insert multi-byte chars near the window boundary.
|
|
||||||
let text = "café café café café café café café café café café";
|
let text = "café café café café café café café café café café";
|
||||||
let s = render(text, &["nope"], &opts_small());
|
let s = extract(text, &["café"], &opts_small());
|
||||||
// Returned string must be valid UTF-8 (push_str guarantees this only
|
assert!(!s.ranges.is_empty());
|
||||||
// if we sliced on char boundaries). Assert by round-trip.
|
assert_ranges_valid(&s);
|
||||||
assert_eq!(s.as_str(), &s.clone());
|
for m in marked(&s) {
|
||||||
|
assert_eq!(m, "café");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn match_near_right_edge_is_fully_shown() {
|
fn match_near_right_edge_is_fully_included() {
|
||||||
let prefix = "x".repeat(30);
|
let prefix = "x".repeat(30);
|
||||||
let text = format!("{}{}", prefix, "LONGMATCHTERMTEXT");
|
let text = format!("{}{}", prefix, "LONGMATCHTERMTEXT");
|
||||||
let s = render(&text, &["LONGMATCHTERMTEXT"], &opts_small());
|
let s = extract(&text, &["LONGMATCHTERMTEXT"], &opts_small());
|
||||||
assert!(s.contains("<b>LONGMATCHTERMTEXT</b>"), "got {s}");
|
assert_eq!(marked(&s), vec!["LONGMATCHTERMTEXT"]);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn empty_query_term_ignored() {
|
fn empty_query_term_ignored() {
|
||||||
let s = render("hello world", &["", "world"], &opts_small());
|
let s = extract("hello world", &["", "world"], &opts_small());
|
||||||
assert!(s.contains("<b>world</b>"), "got {s}");
|
assert_eq!(marked(&s), vec!["world"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn window_around_basic() {
|
||||||
|
let text = "prefix ".repeat(20) + "NEEDLE" + &" suffix".repeat(20);
|
||||||
|
let at = text.find("NEEDLE").unwrap();
|
||||||
|
let s = window_around(&text, (at, at + 6), &opts_small());
|
||||||
|
assert_eq!(marked(&s), vec!["NEEDLE"]);
|
||||||
|
assert!(s.truncated_start && s.truncated_end);
|
||||||
|
assert_ranges_valid(&s);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn window_around_clamps_out_of_bounds() {
|
||||||
|
let s = window_around("tiny", (2, 999), &opts_small());
|
||||||
|
assert_eq!(s.window, "tiny");
|
||||||
|
assert_eq!(s.ranges, vec![(2, 4)]);
|
||||||
|
// Fully out-of-range → no ranges, but never a panic.
|
||||||
|
let s = window_around("tiny", (999, 1000), &opts_small());
|
||||||
|
assert!(s.ranges.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn window_around_aligns_multibyte_boundaries() {
|
||||||
|
let text = "ééééééééé needle ééééééééé";
|
||||||
|
// Deliberately mis-aligned offsets inside multi-byte sequences.
|
||||||
|
let s = window_around(text, (1, 3), &opts_small());
|
||||||
|
assert_ranges_valid(&s);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn count_occurrences_cases() {
|
||||||
|
assert_eq!(count_occurrences("aaaa", "aaa", true), 1, "non-overlapping");
|
||||||
|
assert_eq!(count_occurrences("abcABC", "abc", true), 1);
|
||||||
|
assert_eq!(count_occurrences("abcABC", "abc", false), 2);
|
||||||
|
assert_eq!(count_occurrences("", "x", true), 0);
|
||||||
|
assert_eq!(count_occurrences("xyz", "", true), 0);
|
||||||
|
assert_eq!(count_occurrences("no hits here", "zzz", false), 0);
|
||||||
|
assert_eq!(count_occurrences("ab ab ab", "ab", true), 3);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
1026
crates/quicksearch-core/src/walk.rs
Normal file
1015
crates/quicksearch-core/tests/cascade.rs
Normal file
636
crates/quicksearch-core/tests/full_index.rs
Normal file
|
|
@ -0,0 +1,636 @@
|
||||||
|
//! End-to-end phase-1 tests over a real tree and a real database.
|
||||||
|
//!
|
||||||
|
//! These cover the failure mode that unit tests structurally cannot: a full
|
||||||
|
//! run deletes index rows for every path it did not see, so any walk that
|
||||||
|
//! quietly reports less than it should destroys data. That damage is
|
||||||
|
//! invisible on a first index — `existing_files` is empty, so nothing is
|
||||||
|
//! stale — and only appears on the second run.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
use quicksearch_core::indexing::{IndexingService, IndexingStatus};
|
||||||
|
|
||||||
|
fn tmp_dir(tag: &str) -> PathBuf {
|
||||||
|
let mut p = std::env::temp_dir();
|
||||||
|
p.push(format!(
|
||||||
|
"quicksearch-e2e-{}-{}-{}",
|
||||||
|
tag,
|
||||||
|
std::process::id(),
|
||||||
|
SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos()
|
||||||
|
));
|
||||||
|
std::fs::create_dir_all(&p).unwrap();
|
||||||
|
p
|
||||||
|
}
|
||||||
|
|
||||||
|
fn touch(p: &Path, body: &[u8]) {
|
||||||
|
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
|
||||||
|
std::fs::write(p, body).unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Run one full index and wait for it to finish.
|
||||||
|
///
|
||||||
|
/// Completion is detected via the `last_full_index` marker, which
|
||||||
|
/// `run_indexing` writes only on a successful finish. Polling the status
|
||||||
|
/// enum instead would race: a small tree finishes between two polls, so
|
||||||
|
/// `Idle` is ambiguous between "not started yet" and "already done".
|
||||||
|
fn index_once(root: &Path, db: &Path, config: &Config) {
|
||||||
|
if db.exists() {
|
||||||
|
let conn = rusqlite::Connection::open(db).unwrap();
|
||||||
|
conn.execute("DELETE FROM schema_info WHERE key = 'last_full_index'", [])
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
let service = IndexingService::new();
|
||||||
|
service
|
||||||
|
.start_indexing(
|
||||||
|
vec![root.to_string_lossy().into_owned()],
|
||||||
|
db.to_string_lossy().into_owned(),
|
||||||
|
config.clone(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let deadline = Instant::now() + Duration::from_secs(120);
|
||||||
|
let mut done = false;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
if let IndexingStatus::Error(e) = service.get_status() {
|
||||||
|
panic!("indexing failed: {}", e);
|
||||||
|
}
|
||||||
|
if db.exists() {
|
||||||
|
if let Ok(conn) = rusqlite::Connection::open(db) {
|
||||||
|
if quicksearch_core::db::repo::get_last_full_index(&conn).is_some() {
|
||||||
|
done = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(10));
|
||||||
|
}
|
||||||
|
assert!(done, "indexing did not finish within the timeout");
|
||||||
|
service.stop_indexing().unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// (path, mtime, content_state) for every indexed row, ordered by path.
|
||||||
|
fn rows(db: &Path) -> Vec<(String, i64, i64)> {
|
||||||
|
let conn = rusqlite::Connection::open(db).unwrap();
|
||||||
|
let mut stmt = conn
|
||||||
|
.prepare("SELECT path, mtime, content_state FROM files ORDER BY path")
|
||||||
|
.unwrap();
|
||||||
|
let out = stmt
|
||||||
|
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))
|
||||||
|
.unwrap()
|
||||||
|
.map(|r| r.unwrap())
|
||||||
|
.collect();
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
fn test_config() -> Config {
|
||||||
|
let config = Config::default();
|
||||||
|
// Keep the run to phase 1 semantics we're asserting on; extraction is
|
||||||
|
// covered elsewhere.
|
||||||
|
config
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn reindexing_an_unchanged_tree_changes_nothing() {
|
||||||
|
let root = tmp_dir("stable");
|
||||||
|
let db_dir = tmp_dir("stable-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
touch(&root.join("a.txt"), b"alpha");
|
||||||
|
touch(&root.join("sub/b.txt"), b"bravo");
|
||||||
|
touch(&root.join("sub/deep/c.txt"), b"charlie");
|
||||||
|
touch(&root.join("other/d.md"), b"delta");
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
let first = rows(&db);
|
||||||
|
assert_eq!(first.len(), 4, "all four files indexed");
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
let second = rows(&db);
|
||||||
|
|
||||||
|
// The whole point: a second run over an unchanged tree must not delete
|
||||||
|
// and re-insert anything. A wiped-and-rebuilt row would come back with
|
||||||
|
// content_state reset, throwing away extracted text for no reason.
|
||||||
|
assert_eq!(first, second, "an unchanged tree must re-index to an identical set");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn deleted_files_are_removed_and_new_ones_added() {
|
||||||
|
let root = tmp_dir("churn");
|
||||||
|
let db_dir = tmp_dir("churn-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
touch(&root.join("keep.txt"), b"keep");
|
||||||
|
touch(&root.join("remove.txt"), b"remove");
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
assert_eq!(rows(&db).len(), 2);
|
||||||
|
|
||||||
|
std::fs::remove_file(root.join("remove.txt")).unwrap();
|
||||||
|
touch(&root.join("added.txt"), b"added");
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
|
||||||
|
let names: Vec<String> = rows(&db)
|
||||||
|
.into_iter()
|
||||||
|
.map(|(p, _, _)| Path::new(&p).file_name().unwrap().to_string_lossy().into_owned())
|
||||||
|
.collect();
|
||||||
|
assert_eq!(names, vec!["added.txt", "keep.txt"], "stale cleanup still works");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_modified_file_is_updated_in_place() {
|
||||||
|
let root = tmp_dir("modify");
|
||||||
|
let db_dir = tmp_dir("modify-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
let target = root.join("doc.txt");
|
||||||
|
touch(&target, b"first");
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
let before = rows(&db);
|
||||||
|
assert_eq!(before.len(), 1);
|
||||||
|
|
||||||
|
// Filesystem mtime has one-second granularity in the stored value, so
|
||||||
|
// move it decisively rather than racing it.
|
||||||
|
touch(&target, b"second body, clearly different");
|
||||||
|
let later = SystemTime::now() + Duration::from_secs(5);
|
||||||
|
filetime_set(&target, later);
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
let after = rows(&db);
|
||||||
|
assert_eq!(after.len(), 1, "still exactly one row");
|
||||||
|
assert_ne!(before[0].1, after[0].1, "mtime was refreshed");
|
||||||
|
assert_eq!(before[0].0, after[0].0, "same path");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Set a file's mtime without pulling in a dependency for it.
|
||||||
|
fn filetime_set(path: &Path, when: SystemTime) {
|
||||||
|
let f = std::fs::OpenOptions::new().write(true).open(path).unwrap();
|
||||||
|
f.set_modified(when).unwrap();
|
||||||
|
f.sync_all().unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[cfg(unix)]
|
||||||
|
fn an_unreadable_directory_does_not_delete_its_rows() {
|
||||||
|
// The scenario this guards: a network share or removable drive that is
|
||||||
|
// briefly unavailable. The walk sees nothing beneath it, which must not
|
||||||
|
// be read as "every file under here was deleted".
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
|
||||||
|
let root = tmp_dir("blip");
|
||||||
|
let db_dir = tmp_dir("blip-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
touch(&root.join("visible.txt"), b"visible");
|
||||||
|
let vault = root.join("vault");
|
||||||
|
touch(&vault.join("secret.txt"), b"secret");
|
||||||
|
touch(&vault.join("nested/deeper.txt"), b"deeper");
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
assert_eq!(rows(&db).len(), 3, "all three indexed while readable");
|
||||||
|
|
||||||
|
std::fs::set_permissions(&vault, std::fs::Permissions::from_mode(0o000)).unwrap();
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
let during = rows(&db);
|
||||||
|
std::fs::set_permissions(&vault, std::fs::Permissions::from_mode(0o755)).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
during.len(),
|
||||||
|
3,
|
||||||
|
"rows under an unreadable directory must survive, not be deleted"
|
||||||
|
);
|
||||||
|
|
||||||
|
// And once it is readable again, everything still lines up.
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
assert_eq!(rows(&db).len(), 3);
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn stopping_mid_run_deletes_nothing() {
|
||||||
|
// Pins the end-to-end property: an interrupted run must never delete the
|
||||||
|
// rows it did not reach.
|
||||||
|
//
|
||||||
|
// Two independent guards currently provide it — `run_indexing` skips
|
||||||
|
// cleanup when the walk did not complete, and `cleanup_stale_index_entries`
|
||||||
|
// re-checks the stop flag before its first delete. This test passes with
|
||||||
|
// either one alone, so it does not prove the former is present; it is here
|
||||||
|
// to catch the day someone removes the last of them.
|
||||||
|
let root = tmp_dir("stop");
|
||||||
|
let db_dir = tmp_dir("stop-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
for i in 0..1500 {
|
||||||
|
touch(&root.join(format!("d{}/f{:04}.txt", i % 25, i)), b"body");
|
||||||
|
}
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
let full = rows(&db);
|
||||||
|
assert_eq!(full.len(), 1500);
|
||||||
|
|
||||||
|
// Start again and stop almost immediately, so the walk is cut short.
|
||||||
|
let service = IndexingService::new();
|
||||||
|
service
|
||||||
|
.start_indexing(
|
||||||
|
vec![root.to_string_lossy().into_owned()],
|
||||||
|
db.to_string_lossy().into_owned(),
|
||||||
|
config.clone(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
std::thread::sleep(Duration::from_millis(15));
|
||||||
|
service.stop_indexing().unwrap();
|
||||||
|
drop(service);
|
||||||
|
std::thread::sleep(Duration::from_millis(250));
|
||||||
|
|
||||||
|
let after = rows(&db);
|
||||||
|
assert_eq!(
|
||||||
|
after.len(),
|
||||||
|
1500,
|
||||||
|
"an interrupted run must not delete the rows it never got to"
|
||||||
|
);
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_wide_tree_indexes_every_file_exactly_once() {
|
||||||
|
// Exercises the parallel walk's chunking and termination against a real
|
||||||
|
// database, where a duplicate path would be a UNIQUE violation and a
|
||||||
|
// dropped path would be a missing row.
|
||||||
|
let root = tmp_dir("wide");
|
||||||
|
let db_dir = tmp_dir("wide-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
let count = 900;
|
||||||
|
for i in 0..count {
|
||||||
|
touch(&root.join(format!("d{}/f{:04}.txt", i % 13, i)), b"body");
|
||||||
|
}
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
assert_eq!(rows(&db).len(), count, "every file indexed exactly once");
|
||||||
|
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
assert_eq!(rows(&db).len(), count, "and the second run is stable");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Like `index_once`, but over several roots at once — the per-root
|
||||||
|
/// pipeline path.
|
||||||
|
fn index_roots_once(roots: &[&Path], db: &Path, config: &Config) {
|
||||||
|
if db.exists() {
|
||||||
|
let conn = rusqlite::Connection::open(db).unwrap();
|
||||||
|
conn.execute("DELETE FROM schema_info WHERE key = 'last_full_index'", [])
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let service = IndexingService::new();
|
||||||
|
service
|
||||||
|
.start_indexing(
|
||||||
|
roots.iter().map(|r| r.to_string_lossy().into_owned()).collect(),
|
||||||
|
db.to_string_lossy().into_owned(),
|
||||||
|
config.clone(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let deadline = Instant::now() + Duration::from_secs(120);
|
||||||
|
let mut done = false;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
if let IndexingStatus::Error(e) = service.get_status() {
|
||||||
|
panic!("indexing failed: {}", e);
|
||||||
|
}
|
||||||
|
if db.exists() {
|
||||||
|
if let Ok(conn) = rusqlite::Connection::open(db) {
|
||||||
|
if quicksearch_core::db::repo::get_last_full_index(&conn).is_some() {
|
||||||
|
done = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(10));
|
||||||
|
}
|
||||||
|
assert!(done, "indexing did not finish within the timeout");
|
||||||
|
service.stop_indexing().unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn two_roots_walk_extract_and_clean_independently() {
|
||||||
|
let root_a = tmp_dir("multi-a");
|
||||||
|
let root_b = tmp_dir("multi-b");
|
||||||
|
let db_dir = tmp_dir("multi-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
let config = test_config();
|
||||||
|
|
||||||
|
// Imbalanced roots so the round-robin writer sees a firehose and a
|
||||||
|
// trickle in the same run.
|
||||||
|
for i in 0..60 {
|
||||||
|
touch(&root_a.join(format!("a{:03}.txt", i)), b"alpha corpus xylophone");
|
||||||
|
}
|
||||||
|
for i in 0..5 {
|
||||||
|
touch(&root_b.join(format!("b{:03}.txt", i)), b"bravo corpus quagmire");
|
||||||
|
}
|
||||||
|
|
||||||
|
index_roots_once(&[&root_a, &root_b], &db, &config);
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let total: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(total, 65, "both roots fully walked");
|
||||||
|
let pending: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files WHERE content_state = 0", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(pending, 0, "per-root extraction drained both roots");
|
||||||
|
// Content from EACH root is searchable.
|
||||||
|
for term in ["\"xylophone\"", "\"quagmire\""] {
|
||||||
|
let hits: i64 = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM searchabletext WHERE searchabletext MATCH ?1",
|
||||||
|
[term],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert!(hits > 0, "content from both roots must be indexed ({})", term);
|
||||||
|
}
|
||||||
|
drop(conn);
|
||||||
|
|
||||||
|
// Stale cleanup is global: deleting a file from the trickle root must
|
||||||
|
// remove exactly that row on the next multi-root run.
|
||||||
|
std::fs::remove_file(root_b.join("b000.txt")).unwrap();
|
||||||
|
index_roots_once(&[&root_a, &root_b], &db, &config);
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let total: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(total, 64, "stale row swept across roots");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root_a).ok();
|
||||||
|
std::fs::remove_dir_all(&root_b).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Inline extraction: the walk finishes files whose head is the whole file.
|
||||||
|
//
|
||||||
|
// `hash_length` is what decides how much of a file the walk reads, so setting
|
||||||
|
// it to 0 leaves an empty head, nothing can be extracted inline, and the run
|
||||||
|
// degrades to the pure two-pass behaviour. That makes it the control against
|
||||||
|
// which the optimised path must produce an identical index.
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/// Everything about a file's indexed content that a user can observe: its
|
||||||
|
/// state, the stored snippet body, and its property rows.
|
||||||
|
fn content_rows(db: &Path) -> Vec<(String, i64, Option<String>, Option<i64>, String)> {
|
||||||
|
let conn = rusqlite::Connection::open(db).unwrap();
|
||||||
|
let mut stmt = conn
|
||||||
|
.prepare(
|
||||||
|
"SELECT f.path, f.content_state, f.failure_msg, d.text_len,
|
||||||
|
COALESCE(GROUP_CONCAT(p.key || '=' || p.value, ','), '')
|
||||||
|
FROM files f
|
||||||
|
LEFT JOIN documents_text d ON d.file_id = f.id
|
||||||
|
LEFT JOIN properties p ON p.file_id = f.id
|
||||||
|
GROUP BY f.id
|
||||||
|
ORDER BY f.path",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let out = stmt
|
||||||
|
.query_map([], |r| {
|
||||||
|
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?))
|
||||||
|
})
|
||||||
|
.unwrap()
|
||||||
|
.map(|r| r.unwrap())
|
||||||
|
.collect();
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The decompressed body stored for a file, if any.
|
||||||
|
fn stored_text(db: &Path, suffix: &str) -> Option<String> {
|
||||||
|
let conn = rusqlite::Connection::open(db).unwrap();
|
||||||
|
let blob: Option<Vec<u8>> = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT d.text_zstd FROM documents_text d
|
||||||
|
JOIN files f ON f.id = d.file_id
|
||||||
|
WHERE f.path LIKE '%' || ?1",
|
||||||
|
[suffix],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.ok();
|
||||||
|
blob.map(|b| String::from_utf8(zstd::decode_all(&b[..]).unwrap()).unwrap())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A tree that exercises every branch of the inline decision at once.
|
||||||
|
fn seed_mixed_tree(root: &Path) {
|
||||||
|
let big = "lorem ipsum dolor sit amet ".repeat(600); // ~16 KiB, past any head
|
||||||
|
touch(&root.join("small.txt"), b"a small plaintext body with xylophone in it");
|
||||||
|
touch(&root.join("large.txt"), big.as_bytes());
|
||||||
|
touch(&root.join("empty.txt"), b"");
|
||||||
|
// Invalid UTF-8 with a .txt extension: claimed by the plaintext extractor,
|
||||||
|
// but not decodable, so it must be reported as a failure either way.
|
||||||
|
touch(&root.join("bad.txt"), &[0x68, 0x69, 0xff, 0xfe, 0x00, 0x41]);
|
||||||
|
// No extension `infer` or `mime_guess` recognises: no extractor claims it.
|
||||||
|
touch(&root.join("blob.bin"), &[0x00, 0x01, 0x02, 0xfd, 0xfe, 0xff]);
|
||||||
|
touch(&root.join("nested/deep/note.md"), b"# heading\n\nquagmire body text\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn inline_extraction_produces_an_identical_index_to_the_two_pass_path() {
|
||||||
|
let root = tmp_dir("inline-equiv");
|
||||||
|
let db_dir = tmp_dir("inline-equiv-db");
|
||||||
|
seed_mixed_tree(&root);
|
||||||
|
|
||||||
|
// Control: hash_length 0 => empty head => nothing can be inlined.
|
||||||
|
let mut control = Config::default();
|
||||||
|
control.processing.hash_length = 0;
|
||||||
|
let db_control = db_dir.join("control.sqlite");
|
||||||
|
index_once(&root, &db_control, &control);
|
||||||
|
|
||||||
|
// Optimised: the default head covers every small file in the tree.
|
||||||
|
let optimised = Config::default();
|
||||||
|
let db_opt = db_dir.join("optimised.sqlite");
|
||||||
|
index_once(&root, &db_opt, &optimised);
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
content_rows(&db_control),
|
||||||
|
content_rows(&db_opt),
|
||||||
|
"inlining during the walk must not change a single indexed byte"
|
||||||
|
);
|
||||||
|
|
||||||
|
// And the bodies themselves round-trip identically, not just their lengths.
|
||||||
|
for f in ["small.txt", "large.txt", "note.md"] {
|
||||||
|
assert_eq!(
|
||||||
|
stored_text(&db_control, f),
|
||||||
|
stored_text(&db_opt, f),
|
||||||
|
"stored body differs for {}",
|
||||||
|
f
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_head_boundary_decides_inlining_without_changing_the_result() {
|
||||||
|
let root = tmp_dir("inline-boundary");
|
||||||
|
let db_dir = tmp_dir("inline-boundary-db");
|
||||||
|
|
||||||
|
// Exactly at the limit, and one byte past it.
|
||||||
|
let mut config = Config::default();
|
||||||
|
config.processing.hash_length = 64;
|
||||||
|
let at = "x".repeat(64);
|
||||||
|
let past = "y".repeat(65);
|
||||||
|
touch(&root.join("at.txt"), at.as_bytes());
|
||||||
|
touch(&root.join("past.txt"), past.as_bytes());
|
||||||
|
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
|
||||||
|
// Both are fully extracted; the boundary only decides *which pass* did it.
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let pending: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM files WHERE content_state != 1", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(pending, 0, "both sides of the boundary end up extracted");
|
||||||
|
drop(conn);
|
||||||
|
|
||||||
|
assert_eq!(stored_text(&db, "at.txt").as_deref(), Some(at.as_str()));
|
||||||
|
assert_eq!(stored_text(&db, "past.txt").as_deref(), Some(past.as_str()));
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn undecodable_small_files_are_reported_as_failures_not_silently_skipped() {
|
||||||
|
let root = tmp_dir("inline-badutf8");
|
||||||
|
let db_dir = tmp_dir("inline-badutf8-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
|
||||||
|
touch(&root.join("bad.txt"), &[0x68, 0x69, 0xff, 0xfe]);
|
||||||
|
index_once(&root, &db, &Config::default());
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let (state, msg): (i64, Option<String>) = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT content_state, failure_msg FROM files WHERE path LIKE '%bad.txt'",
|
||||||
|
[],
|
||||||
|
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
// Inlining must not swallow the error: the walk declines to record it, so
|
||||||
|
// the content pass still opens the file and stores a reason.
|
||||||
|
assert_eq!(state, 2, "undecodable content is FAILED, not DONE or NA");
|
||||||
|
assert!(
|
||||||
|
msg.unwrap_or_default().contains("bad.txt"),
|
||||||
|
"the failure names the file"
|
||||||
|
);
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_file_is_done_with_no_snippet_sidecar() {
|
||||||
|
let root = tmp_dir("inline-empty");
|
||||||
|
let db_dir = tmp_dir("inline-empty-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
|
||||||
|
touch(&root.join("empty.txt"), b"");
|
||||||
|
index_once(&root, &db, &Config::default());
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let (state, sidecars): (i64, i64) = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT f.content_state, (SELECT COUNT(*) FROM documents_text d WHERE d.file_id = f.id)
|
||||||
|
FROM files f WHERE f.path LIKE '%empty.txt'",
|
||||||
|
[],
|
||||||
|
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(state, 1, "an empty file is extracted, not failed");
|
||||||
|
assert_eq!(sidecars, 0, "no zstd frame for an empty body");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_content_extension_filter_still_excludes_small_text_files() {
|
||||||
|
let root = tmp_dir("inline-filter");
|
||||||
|
let db_dir = tmp_dir("inline-filter-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
|
||||||
|
let mut config = Config::default();
|
||||||
|
config.indexing.content_extensions = vec!["md".into()];
|
||||||
|
touch(&root.join("kept.md"), b"kept quagmire body");
|
||||||
|
touch(&root.join("skipped.txt"), b"skipped xylophone body");
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let states: Vec<(String, i64)> = conn
|
||||||
|
.prepare("SELECT path, content_state FROM files ORDER BY path")
|
||||||
|
.unwrap()
|
||||||
|
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
|
||||||
|
.unwrap()
|
||||||
|
.map(|r| r.unwrap())
|
||||||
|
.collect();
|
||||||
|
for (path, state) in &states {
|
||||||
|
if path.ends_with("kept.md") {
|
||||||
|
assert_eq!(*state, 1, "an allowed extension is extracted");
|
||||||
|
} else {
|
||||||
|
assert_eq!(*state, 3, "a filtered extension is NA, never inlined");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
drop(conn);
|
||||||
|
assert_eq!(stored_text(&db, "skipped.txt"), None, "no body stored for a filtered file");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn contentless_mode_still_indexes_inlined_files_without_storing_bodies() {
|
||||||
|
let root = tmp_dir("inline-contentless");
|
||||||
|
let db_dir = tmp_dir("inline-contentless-db");
|
||||||
|
let db = db_dir.join("index.sqlite");
|
||||||
|
|
||||||
|
let mut config = Config::default();
|
||||||
|
config.processing.store_text_for_snippets = false;
|
||||||
|
touch(&root.join("small.txt"), b"searchable xylophone body");
|
||||||
|
index_once(&root, &db, &config);
|
||||||
|
|
||||||
|
let conn = rusqlite::Connection::open(&db).unwrap();
|
||||||
|
let sidecars: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM documents_text", [], |r| r.get(0))
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(sidecars, 0, "contentless mode stores no bodies");
|
||||||
|
let hits: i64 = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM searchabletext WHERE searchabletext MATCH '\"xylophone\"'",
|
||||||
|
[],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(hits, 1, "an inlined file is still searchable in contentless mode");
|
||||||
|
|
||||||
|
std::fs::remove_dir_all(&root).ok();
|
||||||
|
std::fs::remove_dir_all(&db_dir).ok();
|
||||||
|
}
|
||||||
|
|
@ -207,12 +207,7 @@ fn snippet_paths_perf_comparison() {
|
||||||
let b_reps = 10;
|
let b_reps = 10;
|
||||||
let start_b = Instant::now();
|
let start_b = Instant::now();
|
||||||
let mut rows_b_total = 0usize;
|
let mut rows_b_total = 0usize;
|
||||||
let opts = snippet::Options {
|
let opts = snippet::Options { approx_chars: 64 };
|
||||||
pre: "<b>",
|
|
||||||
post: "</b>",
|
|
||||||
ellipsis: "...",
|
|
||||||
approx_chars: 64,
|
|
||||||
};
|
|
||||||
for _ in 0..b_reps {
|
for _ in 0..b_reps {
|
||||||
for q in QUERIES {
|
for q in QUERIES {
|
||||||
// Contentless FTS5 returns NULL for stored columns (that's the
|
// Contentless FTS5 returns NULL for stored columns (that's the
|
||||||
|
|
@ -245,7 +240,7 @@ fn snippet_paths_perf_comparison() {
|
||||||
}
|
}
|
||||||
None => String::new(),
|
None => String::new(),
|
||||||
};
|
};
|
||||||
let _snip = snippet::render(&text, &[q], &opts);
|
let _snip = snippet::extract(&text, &[q], &opts);
|
||||||
rows_b_total += 1;
|
rows_b_total += 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -2,14 +2,52 @@
|
||||||
name = "quicksearch-gui"
|
name = "quicksearch-gui"
|
||||||
version.workspace = true
|
version.workspace = true
|
||||||
edition.workspace = true
|
edition.workspace = true
|
||||||
|
license.workspace = true
|
||||||
|
authors.workspace = true
|
||||||
|
repository.workspace = true
|
||||||
|
description = "Fast full-text search across your files: desktop app and terminal search tool."
|
||||||
|
|
||||||
[[bin]]
|
[[bin]]
|
||||||
name = "quicksearch"
|
name = "quicksearch"
|
||||||
path = "src/main.rs"
|
path = "src/main.rs"
|
||||||
|
|
||||||
|
# Terminal search lives in its own binary because the two want opposite
|
||||||
|
# Windows subsystems: a GUI built as a console app flashes a console window on
|
||||||
|
# every launch, and a console tool built as a GUI app cannot write to the shell
|
||||||
|
# that invoked it (cmd and PowerShell do not even wait for it). Splitting is
|
||||||
|
# the only arrangement that is correct in both cases. On Unix the distinction
|
||||||
|
# does not exist and `quicksearch` still does both.
|
||||||
|
[[bin]]
|
||||||
|
name = "quicksearch-cli"
|
||||||
|
path = "src/cli_main.rs"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
quicksearch-core = { path = "../quicksearch-core" }
|
quicksearch-core = { path = "../quicksearch-core" }
|
||||||
|
|
||||||
dioxus = { version = "0.5.1", features = ["desktop"] }
|
eframe = { version = "0.32", default-features = false, features = [
|
||||||
dioxus-desktop = "0.5.1"
|
"glow",
|
||||||
tokio = { version = "1.0", features = ["time", "signal"] }
|
"persistence",
|
||||||
|
] }
|
||||||
|
egui = "0.32"
|
||||||
|
egui_extras = "0.32"
|
||||||
|
rfd = "0.15"
|
||||||
|
open = "5"
|
||||||
|
chrono = { version = "0.4", default-features = false, features = ["clock"] }
|
||||||
|
|
||||||
|
# Display backends, which only exist on Linux/BSD. `default-features = false`
|
||||||
|
# has to be repeated: feature resolution unions the two stanzas, so a single
|
||||||
|
# permissive one would switch defaults back on for every target.
|
||||||
|
[target.'cfg(all(unix, not(target_os = "macos")))'.dependencies]
|
||||||
|
eframe = { version = "0.32", default-features = false, features = [
|
||||||
|
"wayland",
|
||||||
|
"x11",
|
||||||
|
] }
|
||||||
|
|
||||||
|
# Console attachment for the GUI binary (which has no stdio when launched from
|
||||||
|
# Explorer) and VT-mode enabling for the CLI binary. 0.59 matches what eframe
|
||||||
|
# and rfd already resolve, so no extra crate is compiled.
|
||||||
|
[target.'cfg(windows)'.dependencies]
|
||||||
|
windows-sys = { version = "0.59", features = [
|
||||||
|
"Win32_Foundation",
|
||||||
|
"Win32_System_Console",
|
||||||
|
] }
|
||||||
|
|
|
||||||
|
|
@ -1,232 +0,0 @@
|
||||||
// QuickSearch Application JavaScript
|
|
||||||
|
|
||||||
// Enhanced UI interactions
|
|
||||||
document.addEventListener('DOMContentLoaded', function() {
|
|
||||||
console.log('QuickSearch UI loaded');
|
|
||||||
|
|
||||||
// Add loading states to buttons
|
|
||||||
function addLoadingState(button, originalText) {
|
|
||||||
button.disabled = true;
|
|
||||||
button.innerHTML = '<span class="loading"></span>' + originalText;
|
|
||||||
}
|
|
||||||
|
|
||||||
function removeLoadingState(button, originalText) {
|
|
||||||
button.disabled = false;
|
|
||||||
button.innerHTML = originalText;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Enhanced form interactions
|
|
||||||
const forms = document.querySelectorAll('form');
|
|
||||||
forms.forEach(form => {
|
|
||||||
form.addEventListener('submit', function(e) {
|
|
||||||
const submitButton = form.querySelector('button[type="submit"]');
|
|
||||||
if (submitButton) {
|
|
||||||
addLoadingState(submitButton, submitButton.textContent);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Keyboard shortcuts
|
|
||||||
document.addEventListener('keydown', function(e) {
|
|
||||||
// Ctrl+F to focus search
|
|
||||||
if (e.ctrlKey && e.key === 'f') {
|
|
||||||
e.preventDefault();
|
|
||||||
const searchInput = document.querySelector('input[type="text"]');
|
|
||||||
if (searchInput) {
|
|
||||||
searchInput.focus();
|
|
||||||
searchInput.select();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Escape to clear search
|
|
||||||
if (e.key === 'Escape') {
|
|
||||||
const searchInput = document.querySelector('input[type="text"]');
|
|
||||||
if (searchInput && searchInput === document.activeElement) {
|
|
||||||
searchInput.value = '';
|
|
||||||
searchInput.blur();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Enhanced table interactions
|
|
||||||
function enhanceTable(table) {
|
|
||||||
// Add click-to-copy functionality for table cells
|
|
||||||
const cells = table.querySelectorAll('td');
|
|
||||||
cells.forEach(cell => {
|
|
||||||
cell.addEventListener('click', function() {
|
|
||||||
const text = cell.textContent.trim();
|
|
||||||
if (text && navigator.clipboard) {
|
|
||||||
navigator.clipboard.writeText(text).then(() => {
|
|
||||||
// Visual feedback
|
|
||||||
cell.style.backgroundColor = '#4CAF50';
|
|
||||||
cell.style.color = 'white';
|
|
||||||
setTimeout(() => {
|
|
||||||
cell.style.backgroundColor = '';
|
|
||||||
cell.style.color = '';
|
|
||||||
}, 200);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Add sortable columns (basic implementation)
|
|
||||||
const headers = table.querySelectorAll('th');
|
|
||||||
headers.forEach((header, index) => {
|
|
||||||
header.style.cursor = 'pointer';
|
|
||||||
header.addEventListener('click', () => sortTable(table, index));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// Simple table sorting
|
|
||||||
function sortTable(table, columnIndex) {
|
|
||||||
const tbody = table.querySelector('tbody');
|
|
||||||
const rows = Array.from(tbody.querySelectorAll('tr'));
|
|
||||||
|
|
||||||
rows.sort((a, b) => {
|
|
||||||
const aVal = a.cells[columnIndex]?.textContent.trim() || '';
|
|
||||||
const bVal = b.cells[columnIndex]?.textContent.trim() || '';
|
|
||||||
|
|
||||||
// Try numeric sort first
|
|
||||||
const aNum = parseFloat(aVal);
|
|
||||||
const bNum = parseFloat(bVal);
|
|
||||||
|
|
||||||
if (!isNaN(aNum) && !isNaN(bNum)) {
|
|
||||||
return aNum - bNum;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fall back to string sort
|
|
||||||
return aVal.localeCompare(bVal);
|
|
||||||
});
|
|
||||||
|
|
||||||
// Clear tbody and re-append sorted rows
|
|
||||||
tbody.innerHTML = '';
|
|
||||||
rows.forEach(row => tbody.appendChild(row));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Auto-enhance any tables that appear
|
|
||||||
const observer = new MutationObserver(function(mutations) {
|
|
||||||
mutations.forEach(function(mutation) {
|
|
||||||
mutation.addedNodes.forEach(function(node) {
|
|
||||||
if (node.nodeType === 1) { // Element node
|
|
||||||
const tables = node.querySelectorAll ? node.querySelectorAll('table') : [];
|
|
||||||
tables.forEach(enhanceTable);
|
|
||||||
|
|
||||||
if (node.tagName === 'TABLE') {
|
|
||||||
enhanceTable(node);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
observer.observe(document.body, { childList: true, subtree: true });
|
|
||||||
|
|
||||||
// Enhance existing tables
|
|
||||||
document.querySelectorAll('table').forEach(enhanceTable);
|
|
||||||
});
|
|
||||||
|
|
||||||
// Utility functions for Rust integration
|
|
||||||
window.QuickSearch = {
|
|
||||||
// Function to show toast notifications
|
|
||||||
showToast: function(message, type = 'info') {
|
|
||||||
const toast = document.createElement('div');
|
|
||||||
toast.className = `toast toast-${type}`;
|
|
||||||
toast.textContent = message;
|
|
||||||
toast.style.cssText = `
|
|
||||||
position: fixed;
|
|
||||||
top: 20px;
|
|
||||||
right: 20px;
|
|
||||||
padding: 12px 20px;
|
|
||||||
border-radius: 6px;
|
|
||||||
color: white;
|
|
||||||
font-weight: 600;
|
|
||||||
z-index: 2000;
|
|
||||||
animation: slideIn 0.3s ease;
|
|
||||||
`;
|
|
||||||
|
|
||||||
// Set background based on type
|
|
||||||
const colors = {
|
|
||||||
info: '#2196F3',
|
|
||||||
success: '#4CAF50',
|
|
||||||
warning: '#FF9800',
|
|
||||||
error: '#f44336'
|
|
||||||
};
|
|
||||||
toast.style.backgroundColor = colors[type] || colors.info;
|
|
||||||
|
|
||||||
document.body.appendChild(toast);
|
|
||||||
|
|
||||||
setTimeout(() => {
|
|
||||||
toast.style.animation = 'slideOut 0.3s ease';
|
|
||||||
setTimeout(() => {
|
|
||||||
document.body.removeChild(toast);
|
|
||||||
}, 300);
|
|
||||||
}, 3000);
|
|
||||||
},
|
|
||||||
|
|
||||||
// Function to update status display
|
|
||||||
updateStatus: function(status) {
|
|
||||||
const statusDisplay = document.querySelector('.status-display');
|
|
||||||
if (statusDisplay) {
|
|
||||||
statusDisplay.textContent = status;
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
// Function to highlight search terms in results
|
|
||||||
highlightSearchTerms: function(searchTerm, container) {
|
|
||||||
if (!searchTerm || !container) return;
|
|
||||||
|
|
||||||
const walker = document.createTreeWalker(
|
|
||||||
container,
|
|
||||||
NodeFilter.SHOW_TEXT,
|
|
||||||
null,
|
|
||||||
false
|
|
||||||
);
|
|
||||||
|
|
||||||
const textNodes = [];
|
|
||||||
let node;
|
|
||||||
while (node = walker.nextNode()) {
|
|
||||||
textNodes.push(node);
|
|
||||||
}
|
|
||||||
|
|
||||||
textNodes.forEach(textNode => {
|
|
||||||
const parent = textNode.parentNode;
|
|
||||||
if (parent.tagName === 'B') return; // Skip already highlighted
|
|
||||||
|
|
||||||
const text = textNode.textContent;
|
|
||||||
const regex = new RegExp(`(${searchTerm})`, 'gi');
|
|
||||||
|
|
||||||
if (regex.test(text)) {
|
|
||||||
const highlightedHTML = text.replace(regex, '<mark>$1</mark>');
|
|
||||||
const wrapper = document.createElement('span');
|
|
||||||
wrapper.innerHTML = highlightedHTML;
|
|
||||||
parent.replaceChild(wrapper, textNode);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Add custom CSS for toasts and animations
|
|
||||||
const style = document.createElement('style');
|
|
||||||
style.textContent = `
|
|
||||||
@keyframes slideIn {
|
|
||||||
from { transform: translateX(100%); opacity: 0; }
|
|
||||||
to { transform: translateX(0); opacity: 1; }
|
|
||||||
}
|
|
||||||
|
|
||||||
@keyframes slideOut {
|
|
||||||
from { transform: translateX(0); opacity: 1; }
|
|
||||||
to { transform: translateX(100%); opacity: 0; }
|
|
||||||
}
|
|
||||||
|
|
||||||
mark {
|
|
||||||
background: #ffeb3b;
|
|
||||||
padding: 2px 4px;
|
|
||||||
border-radius: 3px;
|
|
||||||
font-weight: bold;
|
|
||||||
}
|
|
||||||
|
|
||||||
.toast {
|
|
||||||
box-shadow: 0 4px 12px rgba(0,0,0,0.2);
|
|
||||||
}
|
|
||||||
`;
|
|
||||||
document.head.appendChild(style);
|
|
||||||
BIN
crates/quicksearch-gui/assets/icons/quicksearch-128.png
Normal file
|
After Width: | Height: | Size: 5.9 KiB |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-16.png
Normal file
|
After Width: | Height: | Size: 652 B |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-22.png
Normal file
|
After Width: | Height: | Size: 919 B |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-24.png
Normal file
|
After Width: | Height: | Size: 998 B |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-256.png
Normal file
|
After Width: | Height: | Size: 12 KiB |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-32.png
Normal file
|
After Width: | Height: | Size: 1.4 KiB |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-48.png
Normal file
|
After Width: | Height: | Size: 2.1 KiB |
BIN
crates/quicksearch-gui/assets/icons/quicksearch-64.png
Normal file
|
After Width: | Height: | Size: 2.8 KiB |
14
crates/quicksearch-gui/assets/icons/quicksearch_icon.svg
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
<svg width="240" height="240" viewBox="0 0 240 240" role="img" xmlns="http://www.w3.org/2000/svg"><title style="fill:rgb(0, 0, 0);stroke:none;color:rgb(11, 11, 11);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto">Fast full-text search icon, square crop</title><desc style="fill:rgb(0, 0, 0);stroke:none;color:rgb(11, 11, 11);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto">A square app icon: centered text lines indenting around a magnifying glass with a lightning bolt inside.</desc>
|
||||||
|
<rect x="0" y="0" width="240" height="240" rx="52" fill="#1e293b" style="fill:rgb(30, 41, 59);stroke:none;color:rgb(11, 11, 11);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<g stroke="#475569" stroke-width="10" stroke-linecap="round" style="fill:rgb(0, 0, 0);stroke:rgb(71, 85, 105);color:rgb(11, 11, 11);stroke-width:10px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto">
|
||||||
|
<line x1="42" y1="58" x2="198" y2="58" style="fill:rgb(0, 0, 0);stroke:rgb(71, 85, 105);color:rgb(11, 11, 11);stroke-width:10px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<line x1="42" y1="92" x2="62" y2="92" style="fill:rgb(0, 0, 0);stroke:rgb(71, 85, 105);color:rgb(11, 11, 11);stroke-width:10px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<line x1="42" y1="126" x2="56" y2="126" style="fill:rgb(0, 0, 0);stroke:rgb(71, 85, 105);color:rgb(11, 11, 11);stroke-width:10px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<line x1="42" y1="160" x2="68" y2="160" style="fill:rgb(0, 0, 0);stroke:rgb(71, 85, 105);color:rgb(11, 11, 11);stroke-width:10px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<line x1="42" y1="194" x2="168" y2="194" style="fill:rgb(0, 0, 0);stroke:rgb(71, 85, 105);color:rgb(11, 11, 11);stroke-width:10px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
</g>
|
||||||
|
<circle cx="130" cy="118" r="46" fill="#0ea5e9" fill-opacity="0.18" style="fill:rgb(14, 165, 233);stroke:none;color:rgb(11, 11, 11);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<circle cx="130" cy="118" r="46" fill="none" stroke="#38bdf8" stroke-width="14" style="fill:none;stroke:rgb(56, 189, 248);color:rgb(11, 11, 11);stroke-width:14px;stroke-linecap:butt;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<line x1="163" y1="151" x2="195" y2="183" stroke="#38bdf8" stroke-width="16" stroke-linecap="round" style="fill:rgb(0, 0, 0);stroke:rgb(56, 189, 248);color:rgb(11, 11, 11);stroke-width:16px;stroke-linecap:round;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
<path d="M137 84 L108 125 L128 125 L121 154 L152 111 L132 111 Z" fill="#facc15" style="fill:rgb(250, 204, 21);stroke:none;color:rgb(11, 11, 11);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;opacity:1;font-family:"Anthropic Sans", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;font-size:16px;font-weight:400;text-anchor:start;dominant-baseline:auto"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 5 KiB |
|
|
@ -1,21 +0,0 @@
|
||||||
<!DOCTYPE html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
||||||
<title>QuickSearch - File Indexer & Search</title>
|
|
||||||
|
|
||||||
<!-- External CSS -->
|
|
||||||
<link rel="stylesheet" href="assets://styles.css">
|
|
||||||
|
|
||||||
<!-- Favicon (optional) -->
|
|
||||||
<link rel="icon" type="image/x-icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'><path fill='%234CAF50' d='M15.5 14h-.79l-.28-.27C15.41 12.59 16 11.11 16 9.5 16 5.91 13.09 3 9.5 3S3 5.91 3 9.5 5.91 16 9.5 16c1.61 0 3.09-.59 4.23-1.57l.27.28v.79l5 4.99L20.49 19l-4.99-5zm-6 0C7.01 14 5 11.99 5 9.5S7.01 5 9.5 5 14 7.01 14 9.5 11.99 14 9.5 14z'/></svg>">
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<!-- Main app container - Dioxus will render into this -->
|
|
||||||
<div id="main"></div>
|
|
||||||
|
|
||||||
<!-- External JavaScript -->
|
|
||||||
<script src="assets://app.js"></script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
|
|
@ -1,299 +0,0 @@
|
||||||
/* QuickSearch Application Styles */
|
|
||||||
body {
|
|
||||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
|
||||||
margin: 0;
|
|
||||||
padding: 20px;
|
|
||||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
|
||||||
min-height: 100vh;
|
|
||||||
}
|
|
||||||
|
|
||||||
.app-container {
|
|
||||||
max-width: 1200px;
|
|
||||||
margin: 0 auto;
|
|
||||||
background: white;
|
|
||||||
border-radius: 12px;
|
|
||||||
box-shadow: 0 8px 32px rgba(0,0,0,0.1);
|
|
||||||
overflow: hidden;
|
|
||||||
}
|
|
||||||
|
|
||||||
.app-header {
|
|
||||||
background: linear-gradient(90deg, #4CAF50 0%, #45a049 100%);
|
|
||||||
color: white;
|
|
||||||
padding: 20px;
|
|
||||||
text-align: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.app-content {
|
|
||||||
padding: 20px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.section {
|
|
||||||
background: #f8f9fa;
|
|
||||||
border-radius: 8px;
|
|
||||||
padding: 20px;
|
|
||||||
margin-bottom: 20px;
|
|
||||||
border-left: 4px solid #4CAF50;
|
|
||||||
}
|
|
||||||
|
|
||||||
.form-control {
|
|
||||||
width: 100%;
|
|
||||||
padding: 12px;
|
|
||||||
border: 2px solid #e0e0e0;
|
|
||||||
border-radius: 6px;
|
|
||||||
font-size: 14px;
|
|
||||||
transition: border-color 0.3s ease;
|
|
||||||
box-sizing: border-box;
|
|
||||||
}
|
|
||||||
|
|
||||||
.form-control:focus {
|
|
||||||
outline: none;
|
|
||||||
border-color: #4CAF50;
|
|
||||||
box-shadow: 0 0 0 3px rgba(76, 175, 80, 0.1);
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn {
|
|
||||||
padding: 12px 24px;
|
|
||||||
border: none;
|
|
||||||
border-radius: 6px;
|
|
||||||
cursor: pointer;
|
|
||||||
font-size: 14px;
|
|
||||||
font-weight: 600;
|
|
||||||
text-transform: uppercase;
|
|
||||||
letter-spacing: 0.5px;
|
|
||||||
transition: all 0.3s ease;
|
|
||||||
margin-right: 10px;
|
|
||||||
display: inline-block;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-primary {
|
|
||||||
background: linear-gradient(90deg, #4CAF50 0%, #45a049 100%);
|
|
||||||
color: white;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-primary:hover:not(:disabled) {
|
|
||||||
transform: translateY(-2px);
|
|
||||||
box-shadow: 0 4px 12px rgba(76, 175, 80, 0.3);
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-danger {
|
|
||||||
background: linear-gradient(90deg, #f44336 0%, #d32f2f 100%);
|
|
||||||
color: white;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-danger:hover:not(:disabled) {
|
|
||||||
transform: translateY(-2px);
|
|
||||||
box-shadow: 0 4px 12px rgba(244, 67, 54, 0.3);
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-info {
|
|
||||||
background: linear-gradient(90deg, #2196F3 0%, #1976D2 100%);
|
|
||||||
color: white;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn-info:hover:not(:disabled) {
|
|
||||||
transform: translateY(-2px);
|
|
||||||
box-shadow: 0 4px 12px rgba(33, 150, 243, 0.3);
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn:disabled {
|
|
||||||
opacity: 0.6;
|
|
||||||
cursor: not-allowed;
|
|
||||||
transform: none !important;
|
|
||||||
box-shadow: none !important;
|
|
||||||
}
|
|
||||||
|
|
||||||
.status-display {
|
|
||||||
background: #1a1a1a;
|
|
||||||
color: #00ff00;
|
|
||||||
padding: 15px;
|
|
||||||
border-radius: 6px;
|
|
||||||
font-family: 'Courier New', monospace;
|
|
||||||
white-space: pre-wrap;
|
|
||||||
font-size: 13px;
|
|
||||||
border: 1px solid #333;
|
|
||||||
overflow-x: auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
.search-results {
|
|
||||||
background: white;
|
|
||||||
border-radius: 8px;
|
|
||||||
overflow: hidden;
|
|
||||||
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
|
|
||||||
margin-top: 20px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table {
|
|
||||||
max-height: 400px;
|
|
||||||
overflow: auto;
|
|
||||||
border: 1px solid #ddd;
|
|
||||||
border-radius: 6px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table table {
|
|
||||||
width: 100%;
|
|
||||||
border-collapse: collapse;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table th {
|
|
||||||
background: #4CAF50;
|
|
||||||
color: white;
|
|
||||||
padding: 12px;
|
|
||||||
text-align: left;
|
|
||||||
font-weight: 600;
|
|
||||||
position: sticky;
|
|
||||||
top: 0;
|
|
||||||
border-bottom: 2px solid #45a049;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table td {
|
|
||||||
padding: 10px 12px;
|
|
||||||
border-bottom: 1px solid #e0e0e0;
|
|
||||||
word-break: break-all;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table tbody tr:hover {
|
|
||||||
background: #f5f5f5;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table tbody tr:nth-child(even) {
|
|
||||||
background: #fafafa;
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table tbody tr:nth-child(even):hover {
|
|
||||||
background: #f0f0f0;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Clickable path cells */
|
|
||||||
.path-cell.clickable {
|
|
||||||
cursor: pointer;
|
|
||||||
color: #1976d2;
|
|
||||||
font-weight: 500;
|
|
||||||
transition: all 0.2s ease;
|
|
||||||
position: relative;
|
|
||||||
}
|
|
||||||
|
|
||||||
.path-cell.clickable:hover {
|
|
||||||
background: #e3f2fd !important;
|
|
||||||
color: #0d47a1;
|
|
||||||
text-decoration: underline;
|
|
||||||
}
|
|
||||||
|
|
||||||
.path-cell.clickable:active {
|
|
||||||
background: #bbdefb !important;
|
|
||||||
transform: scale(0.98);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
.error-message {
|
|
||||||
background: #ffebee;
|
|
||||||
color: #c62828;
|
|
||||||
padding: 12px;
|
|
||||||
border-radius: 6px;
|
|
||||||
border-left: 4px solid #f44336;
|
|
||||||
margin: 10px 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.modal-backdrop {
|
|
||||||
position: fixed;
|
|
||||||
top: 0;
|
|
||||||
left: 0;
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
background: rgba(0,0,0,0.5);
|
|
||||||
backdrop-filter: blur(4px);
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
z-index: 1000;
|
|
||||||
}
|
|
||||||
|
|
||||||
.modal-dialog {
|
|
||||||
background: white;
|
|
||||||
padding: 30px;
|
|
||||||
border-radius: 12px;
|
|
||||||
max-width: 600px;
|
|
||||||
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
|
|
||||||
animation: modalSlideIn 0.3s ease;
|
|
||||||
}
|
|
||||||
|
|
||||||
@keyframes modalSlideIn {
|
|
||||||
from {
|
|
||||||
opacity: 0;
|
|
||||||
transform: translateY(-20px);
|
|
||||||
}
|
|
||||||
to {
|
|
||||||
opacity: 1;
|
|
||||||
transform: translateY(0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
h1, h2, h3 {
|
|
||||||
margin-top: 0;
|
|
||||||
color: #333;
|
|
||||||
}
|
|
||||||
|
|
||||||
label {
|
|
||||||
display: block;
|
|
||||||
margin-bottom: 8px;
|
|
||||||
font-weight: 600;
|
|
||||||
color: #555;
|
|
||||||
}
|
|
||||||
|
|
||||||
.form-group {
|
|
||||||
margin-bottom: 20px;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Highlight matched text in search results */
|
|
||||||
.results-table b {
|
|
||||||
background: #ffeb3b;
|
|
||||||
padding: 2px 4px;
|
|
||||||
border-radius: 3px;
|
|
||||||
font-weight: bold;
|
|
||||||
color: #333;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Loading spinner */
|
|
||||||
.loading {
|
|
||||||
display: inline-block;
|
|
||||||
width: 20px;
|
|
||||||
height: 20px;
|
|
||||||
border: 3px solid #f3f3f3;
|
|
||||||
border-top: 3px solid #4CAF50;
|
|
||||||
border-radius: 50%;
|
|
||||||
animation: spin 1s linear infinite;
|
|
||||||
margin-right: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
@keyframes spin {
|
|
||||||
0% { transform: rotate(0deg); }
|
|
||||||
100% { transform: rotate(360deg); }
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Responsive design */
|
|
||||||
@media (max-width: 768px) {
|
|
||||||
body {
|
|
||||||
padding: 10px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.app-container {
|
|
||||||
border-radius: 8px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.app-content {
|
|
||||||
padding: 15px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.form-control {
|
|
||||||
font-size: 16px; /* Prevents zoom on iOS */
|
|
||||||
}
|
|
||||||
|
|
||||||
.results-table {
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.btn {
|
|
||||||
width: 100%;
|
|
||||||
margin-bottom: 10px;
|
|
||||||
margin-right: 0;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
721
crates/quicksearch-gui/src/app.rs
Normal file
|
|
@ -0,0 +1,721 @@
|
||||||
|
//! Application shell: tab strip, per-frame event drains, debounce,
|
||||||
|
//! status bar, and config-change routing.
|
||||||
|
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use quicksearch_core::cli::{index_counts, IndexCounts};
|
||||||
|
use quicksearch_core::config::{diff_actions, nested_roots, Config};
|
||||||
|
use quicksearch_core::coordinator::{IndexMode, IndexerState, WatcherStatus};
|
||||||
|
use quicksearch_core::indexing::{ConfigChange, IndexingStatus, RootPhase};
|
||||||
|
use quicksearch_core::search::SearchOptions;
|
||||||
|
use quicksearch_core::watcher::WatchError;
|
||||||
|
|
||||||
|
use crate::backend::Backend;
|
||||||
|
use crate::duplicates_tab::{DupState, DuplicatesTab};
|
||||||
|
use crate::format::{fmt_interval, group_thousands};
|
||||||
|
use crate::logs_tab::LogsTab;
|
||||||
|
use crate::manage_tab::ManageTab;
|
||||||
|
use crate::options::OptionsWindow;
|
||||||
|
use crate::search_tab::SearchTab;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
enum Tab {
|
||||||
|
Search,
|
||||||
|
Manage,
|
||||||
|
Duplicates,
|
||||||
|
Logs,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct QuickSearchApp {
|
||||||
|
cfg: Config,
|
||||||
|
backend: Backend,
|
||||||
|
tab: Tab,
|
||||||
|
search: SearchTab,
|
||||||
|
manage: ManageTab,
|
||||||
|
dups: DuplicatesTab,
|
||||||
|
logs: LogsTab,
|
||||||
|
options: OptionsWindow,
|
||||||
|
/// Cached idle counts for the status bar, refreshed at most every 5 s.
|
||||||
|
counts: Option<(Instant, IndexCounts)>,
|
||||||
|
/// Set when applying a config that invalidates the stored index.
|
||||||
|
rebuild_prompt: Option<Vec<ConfigChange>>,
|
||||||
|
/// Set while the "delete the index?" confirmation is open.
|
||||||
|
clear_prompt: bool,
|
||||||
|
/// Nested roots found in the loaded config (startup validation); shown
|
||||||
|
/// as a modal over the Manage tab until dismissed.
|
||||||
|
nested_prompt: Option<Vec<(String, String)>>,
|
||||||
|
/// Set when the watcher gave up on the directory budget and live
|
||||||
|
/// updates are off; see [`QuickSearchApp::check_watch_cap_warning`].
|
||||||
|
watch_cap_prompt: Option<WatchError>,
|
||||||
|
config_error: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl QuickSearchApp {
|
||||||
|
/// `initial_query` pre-fills the search box and fires a search on the
|
||||||
|
/// first frame. It carries the positional arguments the binary was given,
|
||||||
|
/// which on Windows is the only thing the GUI can do with them — terminal
|
||||||
|
/// output belongs to `quicksearch-cli` there.
|
||||||
|
pub fn new(
|
||||||
|
cc: &eframe::CreationContext<'_>,
|
||||||
|
cfg: Config,
|
||||||
|
config_error: Option<String>,
|
||||||
|
initial_query: Option<String>,
|
||||||
|
) -> Result<QuickSearchApp, String> {
|
||||||
|
// Compact styling: results density is the whole point.
|
||||||
|
cc.egui_ctx.style_mut(|style| {
|
||||||
|
style.spacing.item_spacing = egui::vec2(6.0, 3.0);
|
||||||
|
style.spacing.button_padding = egui::vec2(6.0, 2.0);
|
||||||
|
});
|
||||||
|
cc.egui_ctx.set_zoom_factor(clamp_scale(cfg.ui.scale));
|
||||||
|
|
||||||
|
let backend = Backend::start(&cfg, cc.egui_ctx.clone())?;
|
||||||
|
let fuzzy = cfg.search.fuzzy_default;
|
||||||
|
// Startup validation: a hand-edited config can nest roots, which
|
||||||
|
// per-root pipelines can't accept. Redirect straight to the folder
|
||||||
|
// list with an explanatory modal; the coordinator refuses runs
|
||||||
|
// until it's fixed.
|
||||||
|
let nested = nested_roots(&cfg.paths.indexing_paths);
|
||||||
|
let (tab, nested_prompt) = if nested.is_empty() {
|
||||||
|
(Tab::Search, None)
|
||||||
|
} else {
|
||||||
|
(Tab::Manage, Some(nested))
|
||||||
|
};
|
||||||
|
let mut search = SearchTab::new(fuzzy);
|
||||||
|
if let Some(query) = initial_query {
|
||||||
|
search.seed(query);
|
||||||
|
}
|
||||||
|
Ok(QuickSearchApp {
|
||||||
|
cfg,
|
||||||
|
backend,
|
||||||
|
tab,
|
||||||
|
search,
|
||||||
|
manage: ManageTab::new(),
|
||||||
|
dups: DuplicatesTab::new(),
|
||||||
|
logs: LogsTab::new(),
|
||||||
|
options: OptionsWindow::new(),
|
||||||
|
counts: None,
|
||||||
|
rebuild_prompt: None,
|
||||||
|
clear_prompt: false,
|
||||||
|
nested_prompt,
|
||||||
|
watch_cap_prompt: None,
|
||||||
|
config_error,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn search_options(&self) -> SearchOptions {
|
||||||
|
SearchOptions {
|
||||||
|
fuzzy: self.search.fuzzy,
|
||||||
|
fuzzy_max_edits: self.cfg.search.fuzzy_max_edits,
|
||||||
|
limit: self.cfg.search.display_limit,
|
||||||
|
batch: self.cfg.search.results_per_page.max(1),
|
||||||
|
session_ignores: self.search.session_ignores.clone(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn start_search(&mut self) {
|
||||||
|
let generation = self
|
||||||
|
.backend
|
||||||
|
.search()
|
||||||
|
.search(&self.search.query, self.search_options());
|
||||||
|
self.search.on_search_started(generation);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn start_duplicates_scan(&mut self, ctx: &egui::Context) {
|
||||||
|
self.dups.state = DupState::Loading;
|
||||||
|
let cfg = self.cfg.clone();
|
||||||
|
self.backend.start_duplicates(&cfg, ctx.clone());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Save + route an edited config to the running services.
|
||||||
|
fn apply_new_config(&mut self, ctx: &egui::Context, mut new: Config) {
|
||||||
|
if let Some((child, parent)) = nested_roots(&new.paths.indexing_paths).first() {
|
||||||
|
self.config_error = Some(format!(
|
||||||
|
"Not applied: indexed folder {} is nested under {}",
|
||||||
|
child, parent
|
||||||
|
));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Warned-root memory only means anything for folders still indexed.
|
||||||
|
// Pruning here is what makes removing and re-adding a folder warn
|
||||||
|
// again rather than staying silently suppressed forever.
|
||||||
|
new.ui
|
||||||
|
.watch_cap_warned_roots
|
||||||
|
.retain(|root| new.paths.indexing_paths.contains(root));
|
||||||
|
let actions = diff_actions(&self.cfg, &new);
|
||||||
|
if let Err(e) = new.save() {
|
||||||
|
self.config_error = Some(e);
|
||||||
|
}
|
||||||
|
if (new.ui.scale - self.cfg.ui.scale).abs() > f32::EPSILON {
|
||||||
|
ctx.set_zoom_factor(clamp_scale(new.ui.scale));
|
||||||
|
}
|
||||||
|
if actions.search_db_changed {
|
||||||
|
self.backend
|
||||||
|
.search()
|
||||||
|
.set_db_path(new.resolved_database_path());
|
||||||
|
self.counts = None;
|
||||||
|
}
|
||||||
|
self.backend.coordinator.apply_config(new.clone());
|
||||||
|
if actions.requires_rebuild {
|
||||||
|
if self.backend.coordinator.state().mode == IndexMode::Auto {
|
||||||
|
// Automatic mode is hands-off: reconcile immediately, no
|
||||||
|
// prompt. Root-only changes need just a full run — the
|
||||||
|
// walk indexes new roots and the stale sweep drops removed
|
||||||
|
// ones. Anything else (tokenizer, hashing, filters, hidden
|
||||||
|
// files) invalidates stored data and gets the real wipe.
|
||||||
|
let roots_only = {
|
||||||
|
let mut probe = new.clone();
|
||||||
|
probe.paths.indexing_paths = self.cfg.paths.indexing_paths.clone();
|
||||||
|
!diff_actions(&self.cfg, &probe).requires_rebuild
|
||||||
|
};
|
||||||
|
if roots_only {
|
||||||
|
self.backend.coordinator.reindex_now();
|
||||||
|
} else {
|
||||||
|
self.backend.coordinator.rebuild_index();
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
let changes = self
|
||||||
|
.backend
|
||||||
|
.coordinator
|
||||||
|
.check_config_validation(&new)
|
||||||
|
.ok()
|
||||||
|
.flatten()
|
||||||
|
.unwrap_or_default();
|
||||||
|
self.rebuild_prompt = Some(changes);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
self.cfg = new;
|
||||||
|
self.manage.invalidate_editors();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn drain_events(&mut self) {
|
||||||
|
// Streamed search results.
|
||||||
|
loop {
|
||||||
|
match self.backend.search_rx.try_recv() {
|
||||||
|
Ok(update) => self
|
||||||
|
.search
|
||||||
|
.apply_update(update, self.cfg.search.display_limit),
|
||||||
|
Err(_) => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Duplicates worker.
|
||||||
|
if let Some(rx) = &self.backend.dup_job {
|
||||||
|
match rx.try_recv() {
|
||||||
|
Ok(Ok(groups)) => {
|
||||||
|
self.dups.state = DupState::Loaded(groups);
|
||||||
|
self.backend.dup_job = None;
|
||||||
|
}
|
||||||
|
Ok(Err(e)) => {
|
||||||
|
self.dups.state = DupState::Error(e);
|
||||||
|
self.backend.dup_job = None;
|
||||||
|
}
|
||||||
|
Err(std::sync::mpsc::TryRecvError::Empty) => {}
|
||||||
|
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
|
||||||
|
self.dups.state = DupState::Error("duplicates scan aborted".into());
|
||||||
|
self.backend.dup_job = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn tick_debounce(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(edited_at) = self.search.pending_edit else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let debounce = Duration::from_millis(self.cfg.search.debounce_ms);
|
||||||
|
let elapsed = edited_at.elapsed();
|
||||||
|
if elapsed >= debounce {
|
||||||
|
self.search.pending_edit = None;
|
||||||
|
self.start_search();
|
||||||
|
} else {
|
||||||
|
ctx.request_repaint_after(debounce - elapsed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Raise the "live updates are disabled" modal when the watcher has
|
||||||
|
/// given up on the directory budget and at least one indexed folder has
|
||||||
|
/// not been warned about yet.
|
||||||
|
///
|
||||||
|
/// Keyed on roots rather than a single dismissed flag: a restart should
|
||||||
|
/// stay quiet, but adding a folder changes the trade-off and deserves
|
||||||
|
/// the warning again.
|
||||||
|
fn check_watch_cap_warning(&mut self, state: &IndexerState) {
|
||||||
|
let WatcherStatus::Disabled { reason } = &state.watcher else {
|
||||||
|
// Recovered (e.g. the user trimmed the folder list) — retract a
|
||||||
|
// modal that is no longer true.
|
||||||
|
self.watch_cap_prompt = None;
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
// Only the budget limits warrant a modal. Other failures are
|
||||||
|
// transient and not the user's to act on; they are named in the
|
||||||
|
// status line's tooltip and logged to the Logs tab.
|
||||||
|
if !matches!(
|
||||||
|
reason,
|
||||||
|
WatchError::TooManyDirectories { .. } | WatchError::KernelLimit { .. }
|
||||||
|
) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if self.watch_cap_prompt.is_some() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let unwarned = self
|
||||||
|
.cfg
|
||||||
|
.paths
|
||||||
|
.indexing_paths
|
||||||
|
.iter()
|
||||||
|
.any(|root| !self.cfg.ui.watch_cap_warned_roots.contains(root));
|
||||||
|
if unwarned {
|
||||||
|
self.watch_cap_prompt = Some(reason.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn status_bar(&mut self, ctx: &egui::Context) {
|
||||||
|
let state = self.backend.coordinator.state();
|
||||||
|
self.manage.observe(&state.activity);
|
||||||
|
self.check_watch_cap_warning(&state);
|
||||||
|
|
||||||
|
egui::TopBottomPanel::bottom("status-bar").show(ctx, |ui| {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
match &state.activity {
|
||||||
|
IndexingStatus::Idle => {
|
||||||
|
let mode = match state.mode {
|
||||||
|
IndexMode::Auto => "Auto",
|
||||||
|
IndexMode::ManualStopped => "Manual",
|
||||||
|
IndexMode::ManualRunning => "Manual",
|
||||||
|
};
|
||||||
|
let stale = self
|
||||||
|
.counts
|
||||||
|
.map(|(at, _)| at.elapsed() > Duration::from_secs(5))
|
||||||
|
.unwrap_or(true);
|
||||||
|
if stale {
|
||||||
|
let db = self.cfg.resolved_database_path();
|
||||||
|
let counts = index_counts(&db.to_string_lossy()).unwrap_or(
|
||||||
|
IndexCounts {
|
||||||
|
files: 0,
|
||||||
|
content_done: 0,
|
||||||
|
content_pending: 0,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
self.counts = Some((Instant::now(), counts));
|
||||||
|
}
|
||||||
|
let files = self.counts.map(|(_, c)| c.files).unwrap_or(0);
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!(
|
||||||
|
"Idle · {} · {} files indexed",
|
||||||
|
mode,
|
||||||
|
group_thousands(files.max(0) as u64)
|
||||||
|
))
|
||||||
|
.small(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
IndexingStatus::Error(e) => {
|
||||||
|
ui.colored_label(
|
||||||
|
ui.visuals().error_fg_color,
|
||||||
|
egui::RichText::new(format!("Indexing error: {}", e)).small(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
IndexingStatus::Stopping => {
|
||||||
|
ui.label(egui::RichText::new("Stopping indexing…").small());
|
||||||
|
}
|
||||||
|
IndexingStatus::Running { roots, .. } => {
|
||||||
|
let done = roots
|
||||||
|
.iter()
|
||||||
|
.filter(|r| r.phase == RootPhase::Done)
|
||||||
|
.count();
|
||||||
|
let processed: usize =
|
||||||
|
roots.iter().map(|r| r.walked + r.extracted).sum();
|
||||||
|
let totals_known = roots.iter().all(|r| r.walk_total.is_some());
|
||||||
|
let denominator: usize = roots
|
||||||
|
.iter()
|
||||||
|
.map(|r| r.walk_total.unwrap_or(0) + r.extract_total)
|
||||||
|
.sum();
|
||||||
|
|
||||||
|
let mut text = if totals_known && denominator > 0 {
|
||||||
|
let frac =
|
||||||
|
(processed as f64 / denominator as f64).min(1.0);
|
||||||
|
format!(
|
||||||
|
"Indexing {} / {} ({:.0}%)",
|
||||||
|
group_thousands(processed as u64),
|
||||||
|
group_thousands(denominator as u64),
|
||||||
|
frac * 100.0
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
format!(
|
||||||
|
"Indexing · {} files",
|
||||||
|
group_thousands(processed as u64)
|
||||||
|
)
|
||||||
|
};
|
||||||
|
if roots.len() > 1 {
|
||||||
|
text.push_str(&format!(" · {}/{} roots done", done, roots.len()));
|
||||||
|
}
|
||||||
|
if let Some(rate) = self.manage.speed.files_per_sec() {
|
||||||
|
text.push_str(&format!(" · {}", crate::format::fmt_rate(rate)));
|
||||||
|
}
|
||||||
|
let active: usize = roots.iter().map(|r| r.active_workers).sum();
|
||||||
|
let total_workers: usize =
|
||||||
|
roots.iter().map(|r| r.total_workers).sum();
|
||||||
|
if total_workers > 0 {
|
||||||
|
text.push_str(&format!(" · {}/{} workers", active, total_workers));
|
||||||
|
}
|
||||||
|
ui.label(egui::RichText::new(text).small());
|
||||||
|
if totals_known && denominator > 0 {
|
||||||
|
let frac =
|
||||||
|
(processed as f32 / denominator as f32).clamp(0.0, 1.0);
|
||||||
|
ui.add(egui::ProgressBar::new(frac).desired_width(120.0));
|
||||||
|
} else {
|
||||||
|
ui.add(egui::Spinner::new().size(12.0));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Right corner: search result count.
|
||||||
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
|
if self.tab == Tab::Search {
|
||||||
|
if let Some(label) = self.search.result_count_label() {
|
||||||
|
ui.label(egui::RichText::new(label).small().weak());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Keep painting while anything is moving.
|
||||||
|
if !matches!(state.activity, IndexingStatus::Idle | IndexingStatus::Error(_)) {
|
||||||
|
ctx.request_repaint_after(Duration::from_millis(250));
|
||||||
|
}
|
||||||
|
// Watcher registration walks every root, so its verdict can land
|
||||||
|
// minutes after startup. Without this the warning would wait for
|
||||||
|
// the user to happen to move the mouse.
|
||||||
|
if matches!(state.watcher, WatcherStatus::Starting) {
|
||||||
|
ctx.request_repaint_after(Duration::from_millis(500));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rebuild_prompt_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(changes) = &self.rebuild_prompt else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let changes = changes.clone();
|
||||||
|
let mut close = false;
|
||||||
|
egui::Window::new("Rebuild index?")
|
||||||
|
.collapsible(false)
|
||||||
|
.resizable(false)
|
||||||
|
.default_width(560.0)
|
||||||
|
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.label("These settings differ from what the index was built with:");
|
||||||
|
ui.add_space(4.0);
|
||||||
|
if changes.is_empty() {
|
||||||
|
ui.monospace("indexing settings changed");
|
||||||
|
}
|
||||||
|
for change in &changes {
|
||||||
|
ui.strong(format!("{}:", change.key));
|
||||||
|
// Multi-line values (roots, patterns, extensions) are
|
||||||
|
// newline-joined — side-by-side columns keep before and
|
||||||
|
// after readable instead of one run-on arrow line.
|
||||||
|
ui.columns(2, |cols| {
|
||||||
|
cols[0].label(
|
||||||
|
egui::RichText::new("index was built with").small().weak(),
|
||||||
|
);
|
||||||
|
cols[0].monospace(display_value(&change.stored));
|
||||||
|
cols[1].label(egui::RichText::new("config now says").small().weak());
|
||||||
|
cols[1].monospace(display_value(&change.current));
|
||||||
|
});
|
||||||
|
ui.add_space(6.0);
|
||||||
|
}
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"A full rebuild applies them everywhere. Until then, existing \
|
||||||
|
entries keep the old settings.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
if ui.button("Rebuild now").clicked() {
|
||||||
|
self.backend.coordinator.rebuild_index();
|
||||||
|
close = true;
|
||||||
|
}
|
||||||
|
if ui.button("Later").clicked() {
|
||||||
|
close = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
if close {
|
||||||
|
self.rebuild_prompt = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl QuickSearchApp {
|
||||||
|
fn nested_prompt_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(pairs) = &self.nested_prompt else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let pairs = pairs.clone();
|
||||||
|
let mut close = false;
|
||||||
|
egui::Window::new("Indexed folders may not be nested")
|
||||||
|
.collapsible(false)
|
||||||
|
.resizable(false)
|
||||||
|
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.label(
|
||||||
|
"Each root is indexed by its own worker pool, so one root \
|
||||||
|
may not contain another. Fix the folder list below:",
|
||||||
|
);
|
||||||
|
for (child, parent) in &pairs {
|
||||||
|
ui.monospace(format!("{} ⊂ {}", child, parent));
|
||||||
|
}
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Indexing stays paused until the overlap is removed and \
|
||||||
|
the list is applied.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
if ui.button("Fix folders").clicked() {
|
||||||
|
close = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if close {
|
||||||
|
self.nested_prompt = None;
|
||||||
|
self.tab = Tab::Manage;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn watch_cap_prompt_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let Some(reason) = &self.watch_cap_prompt else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let reason = reason.clone();
|
||||||
|
let mut close = false;
|
||||||
|
egui::Window::new("Live index updating is disabled")
|
||||||
|
.collapsible(false)
|
||||||
|
.resizable(false)
|
||||||
|
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.set_max_width(420.0);
|
||||||
|
match &reason {
|
||||||
|
WatchError::TooManyDirectories { cap, .. } => {
|
||||||
|
ui.label(format!(
|
||||||
|
"Your indexed folders contain more than {} directories. The \
|
||||||
|
system limits how many folders can be watched for changes at \
|
||||||
|
once, so QuickSearch cannot update the index as files change.",
|
||||||
|
group_thousands(*cap as u64),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
WatchError::KernelLimit { registered } => {
|
||||||
|
ui.label(format!(
|
||||||
|
"The system ran out of folder watches after {} directories, so \
|
||||||
|
QuickSearch cannot update the index as files change.",
|
||||||
|
group_thousands(*registered as u64),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
WatchError::Other(msg) => {
|
||||||
|
ui.label(format!("Live updates are unavailable: {}", msg));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ui.add_space(4.0);
|
||||||
|
ui.label(format!(
|
||||||
|
"The index is rebuilt every {} instead. Searches keep working; \
|
||||||
|
recent changes may take that long to appear.",
|
||||||
|
fmt_interval(self.cfg.indexing.reindex_interval_minutes),
|
||||||
|
));
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"To restore live updates, index fewer folders or exclude large \
|
||||||
|
subfolders under Filters on the Manage Index tab.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
if ui.button("OK").clicked() {
|
||||||
|
close = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if close {
|
||||||
|
self.watch_cap_prompt = None;
|
||||||
|
for root in &self.cfg.paths.indexing_paths {
|
||||||
|
if !self.cfg.ui.watch_cap_warned_roots.contains(root) {
|
||||||
|
self.cfg.ui.watch_cap_warned_roots.push(root.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Err(e) = self.cfg.save() {
|
||||||
|
self.config_error = Some(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn clear_prompt_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
if !self.clear_prompt {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let mut close = false;
|
||||||
|
egui::Window::new("Clear index?")
|
||||||
|
.collapsible(false)
|
||||||
|
.resizable(false)
|
||||||
|
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.label("This deletes the search index database. Your files are not touched.");
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Indexing switches to manual until you start it again or return to automatic mode.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
if ui
|
||||||
|
.button(
|
||||||
|
egui::RichText::new("Delete index")
|
||||||
|
.color(ui.visuals().error_fg_color),
|
||||||
|
)
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
self.backend.coordinator.clear_index();
|
||||||
|
self.counts = None;
|
||||||
|
self.dups.state = DupState::NotLoaded;
|
||||||
|
close = true;
|
||||||
|
}
|
||||||
|
if ui.button("Cancel").clicked() {
|
||||||
|
close = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
if close {
|
||||||
|
self.clear_prompt = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/// A stored/current config value for the rebuild prompt; list values are
|
||||||
|
/// already newline-joined and render as-is, empty means unset.
|
||||||
|
fn display_value(value: &str) -> String {
|
||||||
|
if value.trim().is_empty() {
|
||||||
|
"(none)".to_string()
|
||||||
|
} else {
|
||||||
|
value.to_string()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Keep the configured UI scale within sane, recoverable bounds.
|
||||||
|
fn clamp_scale(scale: f32) -> f32 {
|
||||||
|
if scale.is_finite() {
|
||||||
|
scale.clamp(0.5, 2.5)
|
||||||
|
} else {
|
||||||
|
1.1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl eframe::App for QuickSearchApp {
|
||||||
|
fn update(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
|
||||||
|
self.drain_events();
|
||||||
|
self.tick_debounce(ctx);
|
||||||
|
self.status_bar(ctx);
|
||||||
|
|
||||||
|
let previous_tab = self.tab;
|
||||||
|
egui::TopBottomPanel::top("tab-strip").show(ctx, |ui| {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.selectable_value(&mut self.tab, Tab::Search, "Search");
|
||||||
|
ui.selectable_value(&mut self.tab, Tab::Manage, "Manage Index");
|
||||||
|
ui.selectable_value(&mut self.tab, Tab::Duplicates, "Duplicates");
|
||||||
|
ui.selectable_value(&mut self.tab, Tab::Logs, "Logs");
|
||||||
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
|
if ui.button("⚙").on_hover_text("Options").clicked() {
|
||||||
|
if self.options.open {
|
||||||
|
self.options.open = false;
|
||||||
|
} else {
|
||||||
|
self.options.open_with(&self.cfg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
// Entering the Duplicates tab kicks off a fresh scan.
|
||||||
|
if self.tab == Tab::Duplicates && previous_tab != Tab::Duplicates {
|
||||||
|
self.start_duplicates_scan(ctx);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(err) = &self.config_error {
|
||||||
|
let err = err.clone();
|
||||||
|
egui::TopBottomPanel::top("config-error").show(ctx, |ui| {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.colored_label(
|
||||||
|
ui.visuals().error_fg_color,
|
||||||
|
format!("Config problem: {} (using defaults)", err),
|
||||||
|
);
|
||||||
|
if ui.small_button("Dismiss").clicked() {
|
||||||
|
self.config_error = None;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
egui::CentralPanel::default().show(ctx, |ui| match self.tab {
|
||||||
|
Tab::Search => {
|
||||||
|
let actions = self.search.ui(ui);
|
||||||
|
if let Some(fuzzy) = actions.save_fuzzy_default {
|
||||||
|
self.cfg.search.fuzzy_default = fuzzy;
|
||||||
|
if let Err(e) = self.cfg.save() {
|
||||||
|
self.config_error = Some(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(pattern) = actions.persist_ignore {
|
||||||
|
let mut new_cfg = self.cfg.clone();
|
||||||
|
if !new_cfg.indexing.ignore_patterns.contains(&pattern) {
|
||||||
|
new_cfg.indexing.ignore_patterns.push(pattern);
|
||||||
|
self.apply_new_config(ctx, new_cfg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if actions.rerun {
|
||||||
|
self.start_search();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Tab::Manage => {
|
||||||
|
let state = self.backend.coordinator.state();
|
||||||
|
let actions = self.manage.ui(ui, &state, &self.cfg);
|
||||||
|
if actions.start_now {
|
||||||
|
self.backend.coordinator.reindex_now();
|
||||||
|
}
|
||||||
|
if actions.stop {
|
||||||
|
self.backend.coordinator.set_mode(IndexMode::ManualStopped);
|
||||||
|
}
|
||||||
|
if actions.auto {
|
||||||
|
self.backend.coordinator.set_mode(IndexMode::Auto);
|
||||||
|
}
|
||||||
|
if actions.clear_index {
|
||||||
|
self.clear_prompt = true;
|
||||||
|
}
|
||||||
|
if actions.start_now || actions.stop || actions.auto {
|
||||||
|
// Keep repainting while the command lands so the state
|
||||||
|
// change is visible without wiggling the mouse — fast
|
||||||
|
// runs otherwise flash by between frames.
|
||||||
|
ui.ctx().request_repaint_after(Duration::from_millis(100));
|
||||||
|
}
|
||||||
|
if let Some(new_cfg) = actions.apply_config {
|
||||||
|
self.apply_new_config(ctx, new_cfg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Tab::Duplicates => {
|
||||||
|
let actions = self.dups.ui(ui);
|
||||||
|
if actions.refresh {
|
||||||
|
self.start_duplicates_scan(ctx);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Tab::Logs => self.logs.ui(ui),
|
||||||
|
});
|
||||||
|
|
||||||
|
if let Some(new_cfg) = self.options.ui(ctx, &self.cfg) {
|
||||||
|
self.apply_new_config(ctx, new_cfg);
|
||||||
|
}
|
||||||
|
self.rebuild_prompt_ui(ctx);
|
||||||
|
self.clear_prompt_ui(ctx);
|
||||||
|
self.nested_prompt_ui(ctx);
|
||||||
|
self.watch_cap_prompt_ui(ctx);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_exit(&mut self, _gl: Option<&eframe::glow::Context>) {
|
||||||
|
self.backend.shutdown();
|
||||||
|
}
|
||||||
|
}
|
||||||
71
crates/quicksearch-gui/src/backend.rs
Normal file
|
|
@ -0,0 +1,71 @@
|
||||||
|
//! Wiring between the egui thread and the core services.
|
||||||
|
//!
|
||||||
|
//! All communication is non-blocking from the UI's point of view:
|
||||||
|
//! searches stream over an mpsc receiver drained each frame, indexing
|
||||||
|
//! state is polled, and the duplicates query runs on a throwaway worker
|
||||||
|
//! thread. Core threads wake the UI through `ctx.request_repaint()`.
|
||||||
|
|
||||||
|
use std::sync::{mpsc, Arc};
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
use quicksearch_core::coordinator::IndexCoordinator;
|
||||||
|
use quicksearch_core::search::{DuplicateGroup, SearchService, SearchUpdate};
|
||||||
|
use quicksearch_core::shutdown;
|
||||||
|
|
||||||
|
pub struct Backend {
|
||||||
|
pub coordinator: Arc<IndexCoordinator>,
|
||||||
|
pub search: Option<SearchService>,
|
||||||
|
pub search_rx: mpsc::Receiver<SearchUpdate>,
|
||||||
|
pub dup_job: Option<mpsc::Receiver<Result<Vec<DuplicateGroup>, String>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Backend {
|
||||||
|
pub fn start(config: &Config, ctx: egui::Context) -> Result<Backend, String> {
|
||||||
|
let coordinator = Arc::new(IndexCoordinator::start(config.clone())?);
|
||||||
|
if let Err(e) = shutdown::install_signal_handler(coordinator.clone()) {
|
||||||
|
quicksearch_core::log_warn!("failed to install signal handler: {}", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
let repaint_ctx = ctx.clone();
|
||||||
|
let (search, search_rx) = SearchService::new(
|
||||||
|
config.resolved_database_path(),
|
||||||
|
Arc::new(move || repaint_ctx.request_repaint()),
|
||||||
|
);
|
||||||
|
|
||||||
|
Ok(Backend {
|
||||||
|
coordinator,
|
||||||
|
search: Some(search),
|
||||||
|
search_rx,
|
||||||
|
dup_job: None,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn search(&self) -> &SearchService {
|
||||||
|
self.search.as_ref().expect("search service alive")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Kick off (or restart) the duplicates listing on a worker thread.
|
||||||
|
pub fn start_duplicates(&mut self, config: &Config, ctx: egui::Context) {
|
||||||
|
let (tx, rx) = mpsc::channel();
|
||||||
|
let db = config.resolved_database_path();
|
||||||
|
std::thread::spawn(move || {
|
||||||
|
let result = quicksearch_core::search::find_duplicate_groups(
|
||||||
|
&db.to_string_lossy(),
|
||||||
|
500,
|
||||||
|
0,
|
||||||
|
);
|
||||||
|
let _ = tx.send(result);
|
||||||
|
ctx.request_repaint();
|
||||||
|
});
|
||||||
|
self.dup_job = Some(rx);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Join the search worker and stop the coordinator. Called once from
|
||||||
|
/// `on_exit`.
|
||||||
|
pub fn shutdown(&mut self) {
|
||||||
|
if let Some(search) = self.search.take() {
|
||||||
|
search.shutdown();
|
||||||
|
}
|
||||||
|
self.coordinator.shutdown();
|
||||||
|
}
|
||||||
|
}
|
||||||
230
crates/quicksearch-gui/src/cli.rs
Normal file
|
|
@ -0,0 +1,230 @@
|
||||||
|
//! Terminal query mode: `quicksearch [FLAGS] <query terms...>` runs the
|
||||||
|
//! same ranked cascade the GUI uses and prints results to stdout. With no
|
||||||
|
//! positional arguments the binary opens the GUI instead.
|
||||||
|
|
||||||
|
use std::io::IsTerminal;
|
||||||
|
use std::sync::atomic::AtomicU64;
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
use quicksearch_core::db;
|
||||||
|
use quicksearch_core::query::split::split_for_cascade;
|
||||||
|
use quicksearch_core::search::{cascade, SearchHit, SearchOptions};
|
||||||
|
|
||||||
|
use crate::format::{fmt_mtime, human_size};
|
||||||
|
|
||||||
|
pub(crate) const USAGE: &str = "\
|
||||||
|
QuickSearch: indexed file search
|
||||||
|
|
||||||
|
USAGE:
|
||||||
|
quicksearch open the GUI
|
||||||
|
quicksearch [FLAGS] <query terms> search from the terminal
|
||||||
|
(Windows: quicksearch-cli)
|
||||||
|
|
||||||
|
FLAGS:
|
||||||
|
--fuzzy also run the fuzzy filename/full-text passes
|
||||||
|
--limit <N> maximum results (default: [search].display_limit)
|
||||||
|
--long rank, size, mtime, and snippets instead of bare paths
|
||||||
|
-h, --help this help
|
||||||
|
|
||||||
|
Query syntax matches the GUI: plain words form one phrase; filters like
|
||||||
|
type:Document, modified:>=2024-01-01, path:/dir, mime:application/pdf,
|
||||||
|
name:frag combine with it.";
|
||||||
|
|
||||||
|
/// Parse argv; `Some(exit_code)` when the invocation was CLI-mode (query
|
||||||
|
/// or --help), `None` to open the GUI.
|
||||||
|
///
|
||||||
|
/// Invariant: terminal mode never builds an [`IndexCoordinator`], so it
|
||||||
|
/// starts no filesystem watcher, no background threads, and consumes no
|
||||||
|
/// inotify watches — a one-shot query must not leave anything running or
|
||||||
|
/// compete for the per-user watch budget with a running GUI. It opens the
|
||||||
|
/// database, queries, prints, and exits. Keep it that way: the coordinator
|
||||||
|
/// belongs to the GUI path in `backend.rs` alone.
|
||||||
|
///
|
||||||
|
/// [`IndexCoordinator`]: quicksearch_core::coordinator::IndexCoordinator
|
||||||
|
pub fn maybe_run_cli() -> Option<i32> {
|
||||||
|
let args: Vec<String> = std::env::args().skip(1).collect();
|
||||||
|
|
||||||
|
let mut fuzzy = false;
|
||||||
|
let mut long = false;
|
||||||
|
let mut limit: Option<usize> = None;
|
||||||
|
let mut terms: Vec<String> = Vec::new();
|
||||||
|
|
||||||
|
let mut it = args.into_iter();
|
||||||
|
while let Some(arg) = it.next() {
|
||||||
|
match arg.as_str() {
|
||||||
|
"-h" | "--help" => {
|
||||||
|
println!("{}", USAGE);
|
||||||
|
return Some(0);
|
||||||
|
}
|
||||||
|
"--fuzzy" => fuzzy = true,
|
||||||
|
"--long" => long = true,
|
||||||
|
"--limit" => match it.next().and_then(|v| v.parse().ok()) {
|
||||||
|
Some(n) => limit = Some(n),
|
||||||
|
None => {
|
||||||
|
eprintln!("--limit requires a number\n\n{}", USAGE);
|
||||||
|
return Some(2);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
other if other.starts_with("--limit=") => {
|
||||||
|
match other["--limit=".len()..].parse() {
|
||||||
|
Ok(n) => limit = Some(n),
|
||||||
|
Err(_) => {
|
||||||
|
eprintln!("--limit requires a number\n\n{}", USAGE);
|
||||||
|
return Some(2);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
other if other.starts_with('-') && terms.is_empty() => {
|
||||||
|
// Unknown flags without a query fall through to the GUI
|
||||||
|
// (they may be eframe/winit flags).
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
other => terms.push(other.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if terms.is_empty() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some(run_query(&terms.join(" "), fuzzy, limit, long))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn run_query(query: &str, fuzzy: bool, limit: Option<usize>, long: bool) -> i32 {
|
||||||
|
let config = match Config::load() {
|
||||||
|
Ok(c) => c,
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("config: {}", e);
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let db_path = config.resolved_database_path();
|
||||||
|
// Read-write purely so SQLite may create the WAL shared-memory file
|
||||||
|
// when no other process has the index open; nothing is written.
|
||||||
|
let conn = match db::open_existing(&db_path.to_string_lossy(), true) {
|
||||||
|
Ok(c) => c,
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!(
|
||||||
|
"No usable index at {}; run the GUI once to build it.\n({})",
|
||||||
|
db_path.display(),
|
||||||
|
e
|
||||||
|
);
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let split = match split_for_cascade(query) {
|
||||||
|
Ok(s) => s,
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("query: {}", e);
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if fuzzy {
|
||||||
|
if let Some(warning) = config.search.fuzzy_edits_warning() {
|
||||||
|
eprintln!("warning: {}", warning);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let options = SearchOptions {
|
||||||
|
fuzzy,
|
||||||
|
fuzzy_max_edits: config.search.fuzzy_max_edits,
|
||||||
|
limit: limit.unwrap_or(config.search.display_limit),
|
||||||
|
batch: config.search.results_per_page.max(1),
|
||||||
|
session_ignores: Vec::new(),
|
||||||
|
};
|
||||||
|
let latest = AtomicU64::new(1);
|
||||||
|
let mut hits: Vec<SearchHit> = Vec::new();
|
||||||
|
let outcome = cascade::run(&conn, &split, &options, 1, &latest, &mut |batch| {
|
||||||
|
hits.extend(batch)
|
||||||
|
});
|
||||||
|
|
||||||
|
match outcome {
|
||||||
|
Ok(Some(outcome)) => {
|
||||||
|
let color = long && std::io::stdout().is_terminal() && enable_vt();
|
||||||
|
for hit in &hits {
|
||||||
|
if long {
|
||||||
|
println!(
|
||||||
|
"{:6.3} {:>9} {} {}",
|
||||||
|
hit.rank,
|
||||||
|
human_size(hit.size),
|
||||||
|
fmt_mtime(hit.mtime),
|
||||||
|
hit.path
|
||||||
|
);
|
||||||
|
if let Some(snip) = &hit.snippet {
|
||||||
|
println!(" {}", render_snippet(snip, color));
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
println!("{}", hit.path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if outcome.limited {
|
||||||
|
eprintln!("(truncated at {} results; raise with --limit)", hits.len());
|
||||||
|
}
|
||||||
|
0
|
||||||
|
}
|
||||||
|
Ok(None) => 0, // unreachable: nothing cancels a CLI search
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("search: {}", e);
|
||||||
|
2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether ANSI escapes will actually render.
|
||||||
|
///
|
||||||
|
/// Always true where the terminal is ANSI by nature. On Windows the console
|
||||||
|
/// only interprets escapes once `ENABLE_VIRTUAL_TERMINAL_PROCESSING` is set:
|
||||||
|
/// Windows Terminal and Windows 11 have it already, older conhost needs it
|
||||||
|
/// turned on, and anything that refuses gets plain text rather than a screen
|
||||||
|
/// full of `\x1b[1m`.
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
fn enable_vt() -> bool {
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(windows)]
|
||||||
|
fn enable_vt() -> bool {
|
||||||
|
use windows_sys::Win32::Foundation::INVALID_HANDLE_VALUE;
|
||||||
|
use windows_sys::Win32::System::Console::{
|
||||||
|
GetConsoleMode, GetStdHandle, SetConsoleMode, ENABLE_VIRTUAL_TERMINAL_PROCESSING,
|
||||||
|
STD_OUTPUT_HANDLE,
|
||||||
|
};
|
||||||
|
|
||||||
|
unsafe {
|
||||||
|
let handle = GetStdHandle(STD_OUTPUT_HANDLE);
|
||||||
|
if handle.is_null() || handle == INVALID_HANDLE_VALUE {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let mut mode = 0;
|
||||||
|
if GetConsoleMode(handle, &mut mode) == 0 {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
mode & ENABLE_VIRTUAL_TERMINAL_PROCESSING != 0
|
||||||
|
|| SetConsoleMode(handle, mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING) != 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One-line snippet with matches emphasized (ANSI bold on TTYs).
|
||||||
|
fn render_snippet(snip: &quicksearch_core::snippet::Snippet, color: bool) -> String {
|
||||||
|
let mut out = String::new();
|
||||||
|
if snip.truncated_start {
|
||||||
|
out.push('…');
|
||||||
|
}
|
||||||
|
let mut cursor = 0;
|
||||||
|
for &(start, end) in &snip.ranges {
|
||||||
|
out.push_str(&snip.window[cursor..start]);
|
||||||
|
if color {
|
||||||
|
out.push_str("\x1b[1m");
|
||||||
|
out.push_str(&snip.window[start..end]);
|
||||||
|
out.push_str("\x1b[0m");
|
||||||
|
} else {
|
||||||
|
out.push_str(&snip.window[start..end]);
|
||||||
|
}
|
||||||
|
cursor = end;
|
||||||
|
}
|
||||||
|
out.push_str(&snip.window[cursor..]);
|
||||||
|
if snip.truncated_end {
|
||||||
|
out.push('…');
|
||||||
|
}
|
||||||
|
out.replace(['\n', '\r'], " ")
|
||||||
|
}
|
||||||
21
crates/quicksearch-gui/src/cli_main.rs
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
//! `quicksearch-cli <query>` — terminal search, and nothing else.
|
||||||
|
//!
|
||||||
|
//! A console-subsystem binary, so redirection, pipes, exit codes, and the
|
||||||
|
//! shell waiting for the process all behave normally. `src/cli.rs` and
|
||||||
|
//! `src/format.rs` are shared with the GUI binary by compiling them into both;
|
||||||
|
//! neither touches egui, so there is nothing to split out into a library.
|
||||||
|
|
||||||
|
mod cli;
|
||||||
|
// The GUI uses more of this module than the CLI does.
|
||||||
|
#[allow(dead_code)]
|
||||||
|
mod format;
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
// `maybe_run_cli` returns `None` for "no query given", which the combined
|
||||||
|
// binary treats as "open the GUI". This one has no GUI to fall back to.
|
||||||
|
let code = cli::maybe_run_cli().unwrap_or_else(|| {
|
||||||
|
eprintln!("{}", cli::USAGE);
|
||||||
|
2
|
||||||
|
});
|
||||||
|
std::process::exit(code);
|
||||||
|
}
|
||||||
111
crates/quicksearch-gui/src/duplicates_tab.rs
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
//! The Duplicates tab: groups of files sharing a content hash.
|
||||||
|
|
||||||
|
use quicksearch_core::search::DuplicateGroup;
|
||||||
|
|
||||||
|
use crate::format::{group_thousands, human_size};
|
||||||
|
use crate::platform;
|
||||||
|
|
||||||
|
pub enum DupState {
|
||||||
|
NotLoaded,
|
||||||
|
Loading,
|
||||||
|
Loaded(Vec<DuplicateGroup>),
|
||||||
|
Error(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct DuplicatesTab {
|
||||||
|
pub state: DupState,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the tab asks the app to do after this frame.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct DuplicatesActions {
|
||||||
|
pub refresh: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DuplicatesTab {
|
||||||
|
pub fn new() -> DuplicatesTab {
|
||||||
|
DuplicatesTab {
|
||||||
|
state: DupState::NotLoaded,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn ui(&mut self, ui: &mut egui::Ui) -> DuplicatesActions {
|
||||||
|
let mut actions = DuplicatesActions::default();
|
||||||
|
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
let loading = matches!(self.state, DupState::Loading);
|
||||||
|
if ui.add_enabled(!loading, egui::Button::new("Refresh")).clicked() {
|
||||||
|
actions.refresh = true;
|
||||||
|
}
|
||||||
|
if loading {
|
||||||
|
ui.add(egui::Spinner::new().size(16.0));
|
||||||
|
ui.label("Scanning for duplicates…");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
match &self.state {
|
||||||
|
DupState::NotLoaded => {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new("Press Refresh to scan the index for duplicate files.")
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
DupState::Loading => {}
|
||||||
|
DupState::Error(e) => {
|
||||||
|
ui.colored_label(ui.visuals().error_fg_color, e);
|
||||||
|
}
|
||||||
|
DupState::Loaded(groups) => {
|
||||||
|
if groups.is_empty() {
|
||||||
|
ui.label("No duplicate files found.");
|
||||||
|
return actions;
|
||||||
|
}
|
||||||
|
if groups.len() == 500 {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new("Showing the 500 largest groups.").small().weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
egui::ScrollArea::vertical().auto_shrink([false; 2]).show(ui, |ui| {
|
||||||
|
for (i, group) in groups.iter().enumerate() {
|
||||||
|
let name = group
|
||||||
|
.members
|
||||||
|
.first()
|
||||||
|
.map(|m| m.1.as_str())
|
||||||
|
.unwrap_or("(unknown)");
|
||||||
|
let title = format!(
|
||||||
|
"{} × {}: {} reclaimable ({} total)",
|
||||||
|
group_thousands(group.count as u64),
|
||||||
|
name,
|
||||||
|
human_size(group.redundant_size.max(0) as u64),
|
||||||
|
human_size(group.total_size.max(0) as u64),
|
||||||
|
);
|
||||||
|
egui::CollapsingHeader::new(title).id_salt(i).show(ui, |ui| {
|
||||||
|
for (_, _, path, size, _) in &group.members {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.label(human_size(*size));
|
||||||
|
let response = ui
|
||||||
|
.add(egui::Label::new(egui::RichText::new(path).monospace())
|
||||||
|
.sense(egui::Sense::click()));
|
||||||
|
if response.double_clicked() {
|
||||||
|
platform::open_file(path);
|
||||||
|
}
|
||||||
|
response.context_menu(|ui| {
|
||||||
|
if ui.button("Open").clicked() {
|
||||||
|
platform::open_file(path);
|
||||||
|
ui.close();
|
||||||
|
}
|
||||||
|
if ui.button("Open containing folder").clicked() {
|
||||||
|
platform::reveal_in_folder(path);
|
||||||
|
ui.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
actions
|
||||||
|
}
|
||||||
|
}
|
||||||
202
crates/quicksearch-gui/src/format.rs
Normal file
|
|
@ -0,0 +1,202 @@
|
||||||
|
//! Small display formatters shared across tabs.
|
||||||
|
|
||||||
|
/// Human-readable byte size: `999 B`, `1.2 KB`, `4.7 MB`, `1.3 GB`.
|
||||||
|
pub fn human_size(bytes: u64) -> String {
|
||||||
|
const UNITS: [&str; 5] = ["B", "KB", "MB", "GB", "TB"];
|
||||||
|
let mut value = bytes as f64;
|
||||||
|
let mut unit = 0;
|
||||||
|
while value >= 1000.0 && unit < UNITS.len() - 1 {
|
||||||
|
value /= 1000.0;
|
||||||
|
unit += 1;
|
||||||
|
}
|
||||||
|
if unit == 0 {
|
||||||
|
format!("{} B", bytes)
|
||||||
|
} else {
|
||||||
|
format!("{:.1} {}", value, UNITS[unit])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `YYYY-MM-DD HH:MM` in local time; raw seconds if out of range.
|
||||||
|
pub fn fmt_mtime(unix_secs: i64) -> String {
|
||||||
|
use chrono::TimeZone;
|
||||||
|
match chrono::Local.timestamp_opt(unix_secs, 0) {
|
||||||
|
chrono::LocalResult::Single(dt) => dt.format("%Y-%m-%d %H:%M").to_string(),
|
||||||
|
_ => unix_secs.to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Relative time for recent events, absolute for old ones: "just now",
|
||||||
|
/// "5 min ago", "3 h ago", else `YYYY-MM-DD HH:MM`. Gives instant
|
||||||
|
/// feedback that an action (like a fast index run) actually happened.
|
||||||
|
pub fn fmt_ago(unix_secs: u64) -> String {
|
||||||
|
let now = std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.map(|d| d.as_secs())
|
||||||
|
.unwrap_or(0);
|
||||||
|
let age = now.saturating_sub(unix_secs);
|
||||||
|
if age < 60 {
|
||||||
|
"just now".to_string()
|
||||||
|
} else if age < 3600 {
|
||||||
|
format!("{} min ago", age / 60)
|
||||||
|
} else if age < 86_400 {
|
||||||
|
format!("{} h ago", age / 3600)
|
||||||
|
} else {
|
||||||
|
fmt_mtime(unix_secs as i64)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A configured interval as a phrase to drop after "every": `90 min`,
|
||||||
|
/// `24 h`, `3 days`. Used where the periodic reindex is the only thing
|
||||||
|
/// refreshing the index, so the user can judge how stale it may get.
|
||||||
|
pub fn fmt_interval(minutes: u64) -> String {
|
||||||
|
if minutes == 0 {
|
||||||
|
// The scheduler treats 0 as always-due.
|
||||||
|
return "run".to_string();
|
||||||
|
}
|
||||||
|
if minutes < 60 {
|
||||||
|
return format!("{} min", minutes);
|
||||||
|
}
|
||||||
|
if minutes.is_multiple_of(1440) {
|
||||||
|
let days = minutes / 1440;
|
||||||
|
return if days == 1 {
|
||||||
|
// "24 h" reads better than "1 day" for the shipped default.
|
||||||
|
"24 h".to_string()
|
||||||
|
} else {
|
||||||
|
format!("{} days", days)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if minutes.is_multiple_of(60) {
|
||||||
|
return format!("{} h", minutes / 60);
|
||||||
|
}
|
||||||
|
format!("{} h {} min", minutes / 60, minutes % 60)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Group thousands for counts: `1,234,567`.
|
||||||
|
pub fn group_thousands(n: u64) -> String {
|
||||||
|
let digits = n.to_string();
|
||||||
|
let mut out = String::with_capacity(digits.len() + digits.len() / 3);
|
||||||
|
for (i, c) in digits.chars().enumerate() {
|
||||||
|
if i > 0 && (digits.len() - i) % 3 == 0 {
|
||||||
|
out.push(',');
|
||||||
|
}
|
||||||
|
out.push(c);
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Files/sec display. Never renders a nonzero rate as "0.0": slow rates
|
||||||
|
/// switch to a per-minute figure.
|
||||||
|
pub fn fmt_rate(files_per_sec: f64) -> String {
|
||||||
|
if files_per_sec <= 0.0 {
|
||||||
|
"0 files/s".to_string()
|
||||||
|
} else if files_per_sec >= 10.0 {
|
||||||
|
format!("{:.0} files/s", files_per_sec)
|
||||||
|
} else if files_per_sec >= 1.0 {
|
||||||
|
format!("{:.1} files/s", files_per_sec)
|
||||||
|
} else {
|
||||||
|
format!("{:.0} files/min", (files_per_sec * 60.0).max(1.0))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Search duration: milliseconds under a second, seconds above.
|
||||||
|
pub fn fmt_elapsed(d: std::time::Duration) -> String {
|
||||||
|
let ms = d.as_millis();
|
||||||
|
if ms >= 1000 {
|
||||||
|
format!("{:.1} s", d.as_secs_f64())
|
||||||
|
} else {
|
||||||
|
format!("{} ms", ms)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Middle-truncate a path to at most `max_chars` characters.
|
||||||
|
pub fn middle_truncate(s: &str, max_chars: usize) -> String {
|
||||||
|
let chars: Vec<char> = s.chars().collect();
|
||||||
|
if chars.len() <= max_chars || max_chars < 5 {
|
||||||
|
return s.to_string();
|
||||||
|
}
|
||||||
|
let keep = max_chars - 1;
|
||||||
|
let head = keep / 2;
|
||||||
|
let tail = keep - head;
|
||||||
|
let mut out: String = chars[..head].iter().collect();
|
||||||
|
out.push('…');
|
||||||
|
out.extend(&chars[chars.len() - tail..]);
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sizes() {
|
||||||
|
assert_eq!(human_size(0), "0 B");
|
||||||
|
assert_eq!(human_size(999), "999 B");
|
||||||
|
assert_eq!(human_size(1200), "1.2 KB");
|
||||||
|
assert_eq!(human_size(4_700_000), "4.7 MB");
|
||||||
|
assert_eq!(human_size(1_300_000_000), "1.3 GB");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn intervals() {
|
||||||
|
assert_eq!(fmt_interval(0), "run");
|
||||||
|
assert_eq!(fmt_interval(1), "1 min");
|
||||||
|
assert_eq!(fmt_interval(59), "59 min");
|
||||||
|
assert_eq!(fmt_interval(60), "1 h");
|
||||||
|
assert_eq!(fmt_interval(90), "1 h 30 min");
|
||||||
|
assert_eq!(fmt_interval(120), "2 h");
|
||||||
|
assert_eq!(fmt_interval(1440), "24 h", "the shipped default");
|
||||||
|
assert_eq!(fmt_interval(2880), "2 days");
|
||||||
|
assert_eq!(fmt_interval(10_080), "7 days");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn thousands() {
|
||||||
|
assert_eq!(group_thousands(0), "0");
|
||||||
|
assert_eq!(group_thousands(999), "999");
|
||||||
|
assert_eq!(group_thousands(1000), "1,000");
|
||||||
|
assert_eq!(group_thousands(1_234_567), "1,234,567");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn rates_never_show_zero_for_nonzero() {
|
||||||
|
assert_eq!(fmt_rate(0.0), "0 files/s");
|
||||||
|
assert_eq!(fmt_rate(2543.0), "2543 files/s");
|
||||||
|
// Not 3.14: clippy reads that as a botched `PI` and denies it.
|
||||||
|
assert_eq!(fmt_rate(3.12), "3.1 files/s");
|
||||||
|
assert_eq!(fmt_rate(0.4), "24 files/min");
|
||||||
|
assert_eq!(fmt_rate(0.001), "1 files/min", "floor at 1/min, never 0.0");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn elapsed_units() {
|
||||||
|
use std::time::Duration;
|
||||||
|
assert_eq!(fmt_elapsed(Duration::from_millis(0)), "0 ms");
|
||||||
|
assert_eq!(fmt_elapsed(Duration::from_millis(7)), "7 ms");
|
||||||
|
assert_eq!(fmt_elapsed(Duration::from_millis(999)), "999 ms");
|
||||||
|
assert_eq!(fmt_elapsed(Duration::from_millis(1000)), "1.0 s");
|
||||||
|
assert_eq!(fmt_elapsed(Duration::from_millis(2340)), "2.3 s");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ago_buckets() {
|
||||||
|
let now = std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.unwrap()
|
||||||
|
.as_secs();
|
||||||
|
assert_eq!(fmt_ago(now), "just now");
|
||||||
|
assert_eq!(fmt_ago(now - 59), "just now");
|
||||||
|
assert_eq!(fmt_ago(now - 120), "2 min ago");
|
||||||
|
assert_eq!(fmt_ago(now - 7200), "2 h ago");
|
||||||
|
assert!(fmt_ago(now - 200_000).contains('-'), "old = absolute date");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn truncation() {
|
||||||
|
assert_eq!(middle_truncate("short", 20), "short");
|
||||||
|
let t = middle_truncate("/very/long/path/to/some/file.txt", 15);
|
||||||
|
assert!(t.chars().count() <= 15);
|
||||||
|
assert!(t.contains('…'));
|
||||||
|
assert!(t.starts_with("/very"));
|
||||||
|
assert!(t.ends_with("e.txt"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,385 +0,0 @@
|
||||||
#![allow(non_snake_case)]
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
|
||||||
use std::collections::VecDeque;
|
|
||||||
use std::time::Instant;
|
|
||||||
use dioxus::prelude::*;
|
|
||||||
use quicksearch_core::indexing::{IndexingService, IndexingStatus};
|
|
||||||
use quicksearch_core::config::Config;
|
|
||||||
|
|
||||||
#[derive(Debug, Clone)]
|
|
||||||
struct SpeedDataPoint {
|
|
||||||
timestamp: Instant,
|
|
||||||
files_processed: usize,
|
|
||||||
}
|
|
||||||
|
|
||||||
struct SpeedTracker {
|
|
||||||
data_points: VecDeque<SpeedDataPoint>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl SpeedTracker {
|
|
||||||
fn new() -> Self {
|
|
||||||
Self {
|
|
||||||
data_points: VecDeque::new(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn add_data_point(&mut self, files_processed: usize) {
|
|
||||||
let now = Instant::now();
|
|
||||||
self.data_points.push_back(SpeedDataPoint {
|
|
||||||
timestamp: now,
|
|
||||||
files_processed,
|
|
||||||
});
|
|
||||||
|
|
||||||
// Prune data points older than 1 second
|
|
||||||
while let Some(front) = self.data_points.front() {
|
|
||||||
if now.duration_since(front.timestamp).as_secs_f64() > 1.0 {
|
|
||||||
self.data_points.pop_front();
|
|
||||||
} else {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn calculate_files_per_second(&self) -> Option<f64> {
|
|
||||||
if self.data_points.len() < 2 {
|
|
||||||
return None;
|
|
||||||
}
|
|
||||||
|
|
||||||
let newest = self.data_points.back()?;
|
|
||||||
let oldest = self.data_points.front()?;
|
|
||||||
|
|
||||||
let time_span = newest.timestamp.duration_since(oldest.timestamp).as_secs_f64();
|
|
||||||
if time_span < 0.1 { // Avoid division by very small numbers
|
|
||||||
return None;
|
|
||||||
}
|
|
||||||
|
|
||||||
let files_diff = newest.files_processed.saturating_sub(oldest.files_processed);
|
|
||||||
Some(files_diff as f64 / time_span)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Props, Clone)]
|
|
||||||
pub struct AppProps {
|
|
||||||
pub indexing_service: Arc<IndexingService>,
|
|
||||||
pub config: Config,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl PartialEq for AppProps {
|
|
||||||
fn eq(&self, other: &Self) -> bool {
|
|
||||||
Arc::ptr_eq(&self.indexing_service, &other.indexing_service)
|
|
||||||
&& self.config.paths.indexing_paths == other.config.paths.indexing_paths
|
|
||||||
&& self.config.paths.database_path == other.config.paths.database_path
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn App(props: AppProps) -> Element {
|
|
||||||
// Multi-root support: the GUI's single line edit holds one path per line.
|
|
||||||
// Empty lines are ignored. Core still de-duplicates and handles nested roots.
|
|
||||||
let mut indexing_path = use_signal(|| props.config.paths.indexing_paths.join("\n"));
|
|
||||||
let mut db_path = use_signal(|| props.config.paths.database_path.clone());
|
|
||||||
let mut status_text = use_signal(|| "Idle".to_string());
|
|
||||||
let mut show_config_dialog = use_signal(|| false);
|
|
||||||
let mut config_changes = use_signal(|| Vec::<String>::new());
|
|
||||||
let speed_tracker = use_signal(|| SpeedTracker::new());
|
|
||||||
|
|
||||||
|
|
||||||
let indexing_service_for_start = props.indexing_service.clone();
|
|
||||||
let indexing_service_for_start_dialog = props.indexing_service.clone();
|
|
||||||
let indexing_service_for_stop = props.indexing_service.clone();
|
|
||||||
let indexing_service_for_timer = props.indexing_service.clone();
|
|
||||||
let config_for_start = props.config.clone();
|
|
||||||
let config_for_dialog = props.config.clone();
|
|
||||||
|
|
||||||
|
|
||||||
// Automatic status updates every second
|
|
||||||
{
|
|
||||||
let mut status_text_clone = status_text.clone();
|
|
||||||
let mut speed_tracker_clone = speed_tracker.clone();
|
|
||||||
let service_clone = indexing_service_for_timer.clone();
|
|
||||||
use_future(move || {
|
|
||||||
let service = service_clone.clone();
|
|
||||||
async move {
|
|
||||||
loop {
|
|
||||||
tokio::time::sleep(tokio::time::Duration::from_millis(50)).await;
|
|
||||||
|
|
||||||
let status = service.get_status();
|
|
||||||
let status_str = match status {
|
|
||||||
IndexingStatus::Idle => {
|
|
||||||
// Reset speed tracker when idle
|
|
||||||
speed_tracker_clone.set(SpeedTracker::new());
|
|
||||||
"Idle".to_string()
|
|
||||||
},
|
|
||||||
IndexingStatus::CountingFiles {
|
|
||||||
current_file,
|
|
||||||
start_time,
|
|
||||||
..
|
|
||||||
} => {
|
|
||||||
let elapsed = start_time.elapsed();
|
|
||||||
let current_file_display = current_file
|
|
||||||
.as_ref()
|
|
||||||
.map(|f| format!("{}", f))
|
|
||||||
.unwrap_or_else(|| "...".to_string());
|
|
||||||
format!(
|
|
||||||
"Phase 0 - Counting paths (shell) - {:.1}s elapsed\n{}",
|
|
||||||
elapsed.as_secs_f64(),
|
|
||||||
current_file_display
|
|
||||||
)
|
|
||||||
}
|
|
||||||
IndexingStatus::RunningFileIndex { files_processed, total_files, current_file, start_time } => {
|
|
||||||
// Add data point to speed tracker
|
|
||||||
speed_tracker_clone.with_mut(|tracker| {
|
|
||||||
tracker.add_data_point(files_processed);
|
|
||||||
});
|
|
||||||
|
|
||||||
let elapsed = start_time.elapsed();
|
|
||||||
let current_file_display = current_file
|
|
||||||
.as_ref()
|
|
||||||
.map(|f| format!("Current: {}", f))
|
|
||||||
.unwrap_or_default();
|
|
||||||
|
|
||||||
// Calculate speed
|
|
||||||
let speed_display = speed_tracker_clone.with(|tracker| {
|
|
||||||
tracker.calculate_files_per_second()
|
|
||||||
.map(|fps| format!(" - {:.1} files/sec", fps))
|
|
||||||
.unwrap_or_default()
|
|
||||||
});
|
|
||||||
|
|
||||||
if let Some(total) = total_files {
|
|
||||||
let percentage = if total > 0 {
|
|
||||||
(files_processed as f64 / total as f64 * 100.0) as u32
|
|
||||||
} else { 0 };
|
|
||||||
format!(
|
|
||||||
"Phase 1 - File Index: {}/{} files ({}%) - {:.1}s elapsed{}\n{}",
|
|
||||||
files_processed,
|
|
||||||
total,
|
|
||||||
percentage,
|
|
||||||
elapsed.as_secs_f64(),
|
|
||||||
speed_display,
|
|
||||||
current_file_display
|
|
||||||
)
|
|
||||||
} else {
|
|
||||||
format!(
|
|
||||||
"Phase 1 - File Index: {} files processed - {:.1}s elapsed{}\n{}",
|
|
||||||
files_processed,
|
|
||||||
elapsed.as_secs_f64(),
|
|
||||||
speed_display,
|
|
||||||
current_file_display
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
IndexingStatus::RunningTextIndex { files_processed, current_file, start_time } => {
|
|
||||||
// Add data point to speed tracker
|
|
||||||
speed_tracker_clone.with_mut(|tracker| {
|
|
||||||
tracker.add_data_point(files_processed);
|
|
||||||
});
|
|
||||||
|
|
||||||
let elapsed = start_time.elapsed();
|
|
||||||
let current_file_display = current_file
|
|
||||||
.as_ref()
|
|
||||||
.map(|f| format!("Current: {}", f))
|
|
||||||
.unwrap_or_default();
|
|
||||||
|
|
||||||
// Calculate speed
|
|
||||||
let speed_display = speed_tracker_clone.with(|tracker| {
|
|
||||||
tracker.calculate_files_per_second()
|
|
||||||
.map(|fps| format!(" - {:.1} files/sec", fps))
|
|
||||||
.unwrap_or_default()
|
|
||||||
});
|
|
||||||
format!(
|
|
||||||
"Phase 2 - Text Index: {} files processed - {:.1}s elapsed{}\n{}",
|
|
||||||
files_processed,
|
|
||||||
elapsed.as_secs_f64(),
|
|
||||||
speed_display,
|
|
||||||
current_file_display
|
|
||||||
)
|
|
||||||
}
|
|
||||||
IndexingStatus::Stopping => "Indexing Stopped".to_string(),
|
|
||||||
IndexingStatus::Error(ref e) => format!("Error: {}", e),
|
|
||||||
};
|
|
||||||
status_text_clone.set(status_str);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
rsx! {
|
|
||||||
div {
|
|
||||||
class: "app-container",
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "app-header",
|
|
||||||
h1 { "QuickSearch File Indexer" }
|
|
||||||
}
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "app-content",
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "section",
|
|
||||||
h2 { "Indexing Controls" }
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "form-group",
|
|
||||||
label { "Paths to index (one per line):" }
|
|
||||||
textarea {
|
|
||||||
class: "form-control",
|
|
||||||
rows: "3",
|
|
||||||
value: "{indexing_path}",
|
|
||||||
oninput: move |evt| indexing_path.set(evt.value())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "form-group",
|
|
||||||
label { "Database path:" }
|
|
||||||
input {
|
|
||||||
class: "form-control",
|
|
||||||
r#type: "text",
|
|
||||||
value: "{db_path}",
|
|
||||||
oninput: move |evt| db_path.set(evt.value())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "form-group",
|
|
||||||
button {
|
|
||||||
class: "btn btn-primary",
|
|
||||||
onclick: move |_| {
|
|
||||||
let service = indexing_service_for_start.clone();
|
|
||||||
let config = config_for_start.clone();
|
|
||||||
let path_text = indexing_path().clone();
|
|
||||||
let paths: Vec<String> = path_text
|
|
||||||
.lines()
|
|
||||||
.map(|l| l.trim().to_string())
|
|
||||||
.filter(|l| !l.is_empty())
|
|
||||||
.collect();
|
|
||||||
let db = db_path().clone();
|
|
||||||
|
|
||||||
if paths.is_empty() {
|
|
||||||
status_text.set("Enter at least one path to index (one per line).".to_string());
|
|
||||||
} else {
|
|
||||||
// The config_validation key still stores a single joined string.
|
|
||||||
let joined = paths.join("\n");
|
|
||||||
match service.check_config_validation(&db, &config, &joined) {
|
|
||||||
Ok(Some(changes)) => {
|
|
||||||
config_changes.set(changes);
|
|
||||||
show_config_dialog.set(true);
|
|
||||||
}
|
|
||||||
Ok(None) => {
|
|
||||||
let _ = service.start_indexing(paths, db, config);
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
status_text.set(format!("Configuration validation error: {}", e));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"Start Indexing"
|
|
||||||
}
|
|
||||||
button {
|
|
||||||
class: "btn btn-danger",
|
|
||||||
onclick: move |_| {
|
|
||||||
let _ = indexing_service_for_stop.stop_indexing();
|
|
||||||
},
|
|
||||||
"Stop Indexing"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "section",
|
|
||||||
h2 { "Status" }
|
|
||||||
pre {
|
|
||||||
class: "status-display",
|
|
||||||
"{status_text}"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
crate::search::Search {
|
|
||||||
indexing_service: props.indexing_service.clone(),
|
|
||||||
db_path: db_path().clone()
|
|
||||||
}
|
|
||||||
|
|
||||||
} // Close app-content
|
|
||||||
}
|
|
||||||
|
|
||||||
// Configuration validation dialog
|
|
||||||
if show_config_dialog() {
|
|
||||||
div {
|
|
||||||
class: "modal-backdrop",
|
|
||||||
div {
|
|
||||||
class: "modal-dialog",
|
|
||||||
h3 {
|
|
||||||
style: "margin-top: 0; color: #d32f2f;",
|
|
||||||
"⚠️ Configuration Changes Detected"
|
|
||||||
}
|
|
||||||
p {
|
|
||||||
style: "margin: 15px 0;",
|
|
||||||
"The following configuration changes require deleting and rebuilding the search index:"
|
|
||||||
}
|
|
||||||
ul {
|
|
||||||
style: "margin: 15px 0; padding-left: 20px;",
|
|
||||||
for change in config_changes().iter() {
|
|
||||||
li {
|
|
||||||
style: "margin: 5px 0; font-family: monospace; background-color: #f5f5f5; padding: 5px; border-radius: 3px;",
|
|
||||||
"{change}"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
p {
|
|
||||||
style: "margin: 15px 0; font-weight: bold;",
|
|
||||||
"This will delete the existing index and rebuild it from scratch."
|
|
||||||
}
|
|
||||||
div {
|
|
||||||
style: "display: flex; gap: 10px; margin-top: 20px;",
|
|
||||||
button {
|
|
||||||
style: "padding: 10px 20px; background-color: #d32f2f; color: white; border: none; border-radius: 5px; cursor: pointer;",
|
|
||||||
onclick: move |_| {
|
|
||||||
let service = indexing_service_for_start_dialog.clone();
|
|
||||||
let config = config_for_dialog.clone();
|
|
||||||
let path_text = indexing_path().clone();
|
|
||||||
let paths: Vec<String> = path_text
|
|
||||||
.lines()
|
|
||||||
.map(|l| l.trim().to_string())
|
|
||||||
.filter(|l| !l.is_empty())
|
|
||||||
.collect();
|
|
||||||
let db = db_path().clone();
|
|
||||||
|
|
||||||
show_config_dialog.set(false);
|
|
||||||
status_text.set("Stopping indexing and deleting database...".to_string());
|
|
||||||
|
|
||||||
// Delete database file and restart indexing
|
|
||||||
let service_clone = service.clone();
|
|
||||||
let paths_clone = paths.clone();
|
|
||||||
let db_clone = db.clone();
|
|
||||||
let config_clone = config.clone();
|
|
||||||
let mut status_clone = status_text.clone();
|
|
||||||
|
|
||||||
spawn(async move {
|
|
||||||
match service_clone.delete_index_for_rebuild(&db_clone) {
|
|
||||||
Ok(()) => {
|
|
||||||
status_clone.set("Database deleted. Starting fresh indexing...".to_string());
|
|
||||||
let _ = service_clone.start_indexing(paths_clone, db_clone, config_clone);
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
status_clone.set(format!("Error deleting database: {}", e));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
},
|
|
||||||
"Yes, Rebuild Index"
|
|
||||||
}
|
|
||||||
button {
|
|
||||||
style: "padding: 10px 20px; background-color: #666; color: white; border: none; border-radius: 5px; cursor: pointer;",
|
|
||||||
onclick: move |_| {
|
|
||||||
show_config_dialog.set(false);
|
|
||||||
},
|
|
||||||
"Cancel"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
255
crates/quicksearch-gui/src/logs_tab.rs
Normal file
|
|
@ -0,0 +1,255 @@
|
||||||
|
//! The Logs tab: what the terminal would have shown.
|
||||||
|
//!
|
||||||
|
//! Everything here comes from [`quicksearch_core::log`], which background
|
||||||
|
//! threads write through instead of printing. Launched from a desktop
|
||||||
|
//! launcher there is no terminal to read, and this is the only place a
|
||||||
|
//! "cannot read that folder" warning is visible.
|
||||||
|
|
||||||
|
use quicksearch_core::log::{self, Level, LogLine};
|
||||||
|
|
||||||
|
use crate::format::group_thousands;
|
||||||
|
|
||||||
|
/// How often to repaint while the tab is open. Log lines arrive on indexer
|
||||||
|
/// and watcher threads, which have no reason to wake the UI, so an idle
|
||||||
|
/// window would otherwise sit on a stale list until the mouse moved.
|
||||||
|
const REFRESH_MS: u64 = 500;
|
||||||
|
|
||||||
|
pub struct LogsTab {
|
||||||
|
/// Copy of the ring, refreshed only when the recorded count moves —
|
||||||
|
/// cloning a few thousand lines every frame would be silly.
|
||||||
|
lines: Vec<LogLine>,
|
||||||
|
/// [`log::recorded`] as of the last refresh.
|
||||||
|
seen: u64,
|
||||||
|
dropped: u64,
|
||||||
|
filter: String,
|
||||||
|
warnings_only: bool,
|
||||||
|
/// Keep the newest line in view. Scrolling up releases the view anyway
|
||||||
|
/// (egui unsticks a scroll area the user moves, and re-sticks it when
|
||||||
|
/// they return to the bottom); unticking this stops it following at all.
|
||||||
|
follow: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl LogsTab {
|
||||||
|
pub fn new() -> LogsTab {
|
||||||
|
LogsTab {
|
||||||
|
lines: Vec::new(),
|
||||||
|
seen: 0,
|
||||||
|
dropped: 0,
|
||||||
|
filter: String::new(),
|
||||||
|
warnings_only: false,
|
||||||
|
follow: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn refresh(&mut self) {
|
||||||
|
self.lines = log::snapshot();
|
||||||
|
self.seen = log::recorded();
|
||||||
|
self.dropped = log::dropped();
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn ui(&mut self, ui: &mut egui::Ui) {
|
||||||
|
if log::recorded() != self.seen {
|
||||||
|
self.refresh();
|
||||||
|
}
|
||||||
|
ui.ctx()
|
||||||
|
.request_repaint_after(std::time::Duration::from_millis(REFRESH_MS));
|
||||||
|
|
||||||
|
// Indices rather than references: the control row below takes the
|
||||||
|
// filter and follow flags mutably, and a borrow of `self.lines`
|
||||||
|
// held across it would conflict. One frame of lag after a
|
||||||
|
// keystroke, which repaints immediately anyway.
|
||||||
|
let needle = self.filter.to_lowercase();
|
||||||
|
let shown: Vec<usize> = self
|
||||||
|
.lines
|
||||||
|
.iter()
|
||||||
|
.enumerate()
|
||||||
|
.filter(|(_, l)| keep(l, &needle, self.warnings_only))
|
||||||
|
.map(|(i, _)| i)
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
let mut cleared = false;
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.checkbox(&mut self.follow, "Follow")
|
||||||
|
.on_hover_text("Scroll to the newest line as it arrives");
|
||||||
|
ui.checkbox(&mut self.warnings_only, "Warnings only");
|
||||||
|
ui.add(
|
||||||
|
egui::TextEdit::singleline(&mut self.filter)
|
||||||
|
.desired_width(200.0)
|
||||||
|
.hint_text("Filter"),
|
||||||
|
);
|
||||||
|
if ui
|
||||||
|
.add_enabled(!shown.is_empty(), egui::Button::new("Copy"))
|
||||||
|
.on_hover_text("Copy the lines shown below to the clipboard")
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
let joined = shown
|
||||||
|
.iter()
|
||||||
|
.map(|&i| self.lines[i].text.as_str())
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join("\n");
|
||||||
|
ui.ctx().copy_text(joined);
|
||||||
|
}
|
||||||
|
if ui
|
||||||
|
.add_enabled(!self.lines.is_empty(), egui::Button::new("Clear"))
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
log::clear();
|
||||||
|
cleared = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
|
let count = if shown.len() == self.lines.len() {
|
||||||
|
format!("{} lines", group_thousands(self.lines.len() as u64))
|
||||||
|
} else {
|
||||||
|
format!(
|
||||||
|
"{} of {} lines",
|
||||||
|
group_thousands(shown.len() as u64),
|
||||||
|
group_thousands(self.lines.len() as u64)
|
||||||
|
)
|
||||||
|
};
|
||||||
|
ui.label(egui::RichText::new(count).small().weak());
|
||||||
|
});
|
||||||
|
});
|
||||||
|
if cleared {
|
||||||
|
// `shown` indexes lines that no longer exist.
|
||||||
|
self.refresh();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if self.dropped > 0 {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!(
|
||||||
|
"{} earlier lines were dropped; the newest {} are kept.",
|
||||||
|
group_thousands(self.dropped),
|
||||||
|
group_thousands(log::CAPACITY as u64),
|
||||||
|
))
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
if self.lines.is_empty() {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Nothing logged yet. Warnings from indexing, watching folders and \
|
||||||
|
opening files appear here — the same lines the terminal would show.",
|
||||||
|
)
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if shown.is_empty() {
|
||||||
|
ui.label(egui::RichText::new("No lines match the filter.").weak());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Long paths extend into a horizontal scroll rather than wrapping:
|
||||||
|
// `show_rows` only draws the visible slice, and that costs nothing
|
||||||
|
// only while every row is exactly one line tall.
|
||||||
|
ui.style_mut().wrap_mode = Some(egui::TextWrapMode::Extend);
|
||||||
|
let row_height = ui.text_style_height(&egui::TextStyle::Monospace);
|
||||||
|
egui::ScrollArea::both()
|
||||||
|
.auto_shrink([false; 2])
|
||||||
|
.stick_to_bottom(self.follow)
|
||||||
|
.show_rows(ui, row_height, shown.len(), |ui, range| {
|
||||||
|
for &i in &shown[range] {
|
||||||
|
let line = &self.lines[i];
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(fmt_clock(line.at))
|
||||||
|
.monospace()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
let text = egui::RichText::new(&line.text).monospace();
|
||||||
|
match line.level {
|
||||||
|
Level::Warn => {
|
||||||
|
ui.colored_label(ui.visuals().warn_fg_color, text);
|
||||||
|
}
|
||||||
|
Level::Info => {
|
||||||
|
ui.label(text);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a line survives the tab's two filters. `needle` is expected
|
||||||
|
/// already lowercased — it is the same for every line, so folding it once
|
||||||
|
/// per frame beats folding it per line.
|
||||||
|
fn keep(line: &LogLine, needle: &str, warnings_only: bool) -> bool {
|
||||||
|
if warnings_only && line.level != Level::Warn {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
needle.is_empty() || line.text.to_lowercase().contains(needle)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `HH:MM:SS` local time. The date is deliberately absent: these lines are
|
||||||
|
/// read while something is going wrong now, and a full stamp on every row
|
||||||
|
/// would crowd out the message.
|
||||||
|
fn fmt_clock(unix_secs: u64) -> String {
|
||||||
|
use chrono::TimeZone;
|
||||||
|
// Saturating rather than `as`: that cast wraps a huge value into a
|
||||||
|
// negative one, which is a perfectly valid 1969 timestamp and would
|
||||||
|
// render as a plausible time instead of falling back.
|
||||||
|
let secs = i64::try_from(unix_secs).unwrap_or(i64::MAX);
|
||||||
|
match chrono::Local.timestamp_opt(secs, 0) {
|
||||||
|
chrono::LocalResult::Single(dt) => dt.format("%H:%M:%S").to_string(),
|
||||||
|
_ => "--:--:--".to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::{fmt_clock, keep};
|
||||||
|
use quicksearch_core::log::{Level, LogLine};
|
||||||
|
|
||||||
|
fn line(level: Level, text: &str) -> LogLine {
|
||||||
|
LogLine {
|
||||||
|
at: 1_700_000_000,
|
||||||
|
level,
|
||||||
|
text: text.to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_filter_keeps_everything() {
|
||||||
|
assert!(keep(&line(Level::Info, "anything"), "", false));
|
||||||
|
assert!(keep(&line(Level::Warn, "anything"), "", false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_filter_ignores_case_on_both_sides() {
|
||||||
|
let l = line(Level::Warn, "Warning: cannot read /Home/Photos");
|
||||||
|
assert!(keep(&l, "photos", false), "needle case must not matter");
|
||||||
|
assert!(keep(&l, "cannot read", false), "nor the line's");
|
||||||
|
assert!(!keep(&l, "videos", false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn warnings_only_hides_informational_lines() {
|
||||||
|
assert!(!keep(&line(Level::Info, "shutting down"), "", true));
|
||||||
|
assert!(keep(&line(Level::Warn, "cannot read"), "", true));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Both filters apply, not either.
|
||||||
|
#[test]
|
||||||
|
fn the_two_filters_compose() {
|
||||||
|
assert!(!keep(&line(Level::Info, "cannot read"), "cannot", true));
|
||||||
|
assert!(!keep(&line(Level::Warn, "cannot read"), "missing", true));
|
||||||
|
assert!(keep(&line(Level::Warn, "cannot read"), "cannot", true));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_clock_stamp_is_fixed_width() {
|
||||||
|
assert_eq!(fmt_clock(0).len(), 8, "epoch renders as a time, not a date");
|
||||||
|
assert_eq!(fmt_clock(1_700_000_000).len(), 8);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Beyond what a local calendar can represent, the row still lines up.
|
||||||
|
#[test]
|
||||||
|
fn an_out_of_range_stamp_falls_back() {
|
||||||
|
assert_eq!(fmt_clock(u64::MAX), "--:--:--");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,54 +1,95 @@
|
||||||
use std::sync::{Arc, OnceLock};
|
//! QuickSearch binary: `quicksearch <query>` searches from the terminal;
|
||||||
use dioxus::prelude::*;
|
//! without a query it opens the egui desktop app.
|
||||||
use quicksearch_core::{config, indexing, shutdown};
|
//!
|
||||||
mod frontend;
|
//! On Windows this is the GUI only, built as a window-subsystem app so no
|
||||||
mod search;
|
//! console flashes behind it. Terminal search there is `quicksearch-cli`,
|
||||||
|
//! which is a console app and so keeps working pipes, exit codes, and a shell
|
||||||
|
//! that waits for it. A query passed here still does something useful: it
|
||||||
|
//! seeds the search box.
|
||||||
|
#![cfg_attr(windows, windows_subsystem = "windows")]
|
||||||
|
|
||||||
static INDEXING_SERVICE: OnceLock<Arc<indexing::IndexingService>> = OnceLock::new();
|
mod app;
|
||||||
|
mod backend;
|
||||||
|
#[cfg(not(windows))]
|
||||||
|
mod cli;
|
||||||
|
mod duplicates_tab;
|
||||||
|
mod format;
|
||||||
|
mod logs_tab;
|
||||||
|
mod manage_tab;
|
||||||
|
mod options;
|
||||||
|
mod platform;
|
||||||
|
mod query_highlight;
|
||||||
|
mod search_tab;
|
||||||
|
mod tracker;
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
|
||||||
|
/// The window icon, shown in the titlebar, taskbar and alt-tab switcher.
|
||||||
|
///
|
||||||
|
/// X11 takes these pixels directly via `_NET_WM_ICON`. Wayland ignores them and
|
||||||
|
/// instead looks up the app id in `/usr/share/applications/`, so the id below has
|
||||||
|
/// to match the installed `quicksearch.desktop` for the icon to appear there.
|
||||||
|
fn app_icon() -> egui::IconData {
|
||||||
|
eframe::icon_data::from_png_bytes(include_bytes!("../assets/icons/quicksearch-256.png"))
|
||||||
|
.expect("bundled icon is a valid PNG")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Leftover positional arguments, joined — used to seed the search box.
|
||||||
|
///
|
||||||
|
/// Flags are dropped rather than parsed: eframe and winit take some of their
|
||||||
|
/// own, and a stray `--foo` should not end up in the query.
|
||||||
|
fn seed_query() -> Option<String> {
|
||||||
|
let terms: Vec<String> = std::env::args()
|
||||||
|
.skip(1)
|
||||||
|
.filter(|a| !a.starts_with('-'))
|
||||||
|
.collect();
|
||||||
|
if terms.is_empty() {
|
||||||
|
None
|
||||||
|
} else {
|
||||||
|
Some(terms.join(" "))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn main() {
|
fn main() {
|
||||||
let indexing_service = Arc::new(indexing::IndexingService::new());
|
// Must come first: anything below may print, and printing without a
|
||||||
INDEXING_SERVICE
|
// stdio handle panics rather than failing quietly.
|
||||||
.set(indexing_service.clone())
|
#[cfg(windows)]
|
||||||
.expect("Failed to set global indexing service");
|
platform::redirect_null_stdio();
|
||||||
|
|
||||||
if let Err(e) = shutdown::install_signal_handler(indexing_service.clone()) {
|
#[cfg(not(windows))]
|
||||||
eprintln!("Warning: failed to install signal handler: {}", e);
|
if let Some(code) = cli::maybe_run_cli() {
|
||||||
|
std::process::exit(code);
|
||||||
}
|
}
|
||||||
|
|
||||||
LaunchBuilder::desktop()
|
// A broken config file should never keep the window from opening —
|
||||||
.with_cfg(
|
// surface the error in-app and run on defaults.
|
||||||
dioxus_desktop::Config::new()
|
let (config, config_error) = match Config::load() {
|
||||||
.with_custom_head(format!("<style>{}</style>", include_str!("../assets/styles.css")))
|
Ok(c) => (c, None),
|
||||||
.with_window(dioxus_desktop::WindowBuilder::new()
|
Err(e) => (Config::default(), Some(e)),
|
||||||
.with_title("QuickSearch - File Indexer & Search")
|
};
|
||||||
.with_resizable(true)
|
let initial_query = seed_query();
|
||||||
.with_inner_size(dioxus_desktop::LogicalSize::new(1000.0, 700.0))
|
|
||||||
)
|
|
||||||
)
|
|
||||||
.launch(app);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
let native_options = eframe::NativeOptions {
|
||||||
fn app() -> Element {
|
viewport: egui::ViewportBuilder::default()
|
||||||
let cfg = match config::Config::load() {
|
.with_title("QuickSearch")
|
||||||
Ok(c) => c,
|
.with_app_id("quicksearch")
|
||||||
Err(e) => {
|
.with_icon(app_icon())
|
||||||
eprintln!("Failed to load config: {}", e);
|
.with_inner_size([1000.0, 700.0])
|
||||||
return rsx! { div { "Failed to load configuration" } };
|
.with_min_inner_size([640.0, 400.0]),
|
||||||
}
|
..Default::default()
|
||||||
};
|
};
|
||||||
|
|
||||||
let indexing_service = INDEXING_SERVICE
|
let result = eframe::run_native(
|
||||||
.get()
|
"QuickSearch",
|
||||||
.expect("Indexing service not initialized")
|
native_options,
|
||||||
.clone();
|
Box::new(move |cc| {
|
||||||
|
app::QuickSearchApp::new(cc, config, config_error, initial_query)
|
||||||
rsx! {
|
.map(|app| Box::new(app) as Box<dyn eframe::App>)
|
||||||
frontend::App {
|
.map_err(|e| e.into())
|
||||||
indexing_service: indexing_service,
|
}),
|
||||||
config: cfg
|
);
|
||||||
|
if let Err(e) = result {
|
||||||
|
eprintln!("failed to start GUI: {}", e);
|
||||||
|
std::process::exit(1);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
|
|
|
||||||
447
crates/quicksearch-gui/src/manage_tab.rs
Normal file
|
|
@ -0,0 +1,447 @@
|
||||||
|
//! The Manage Index tab: detailed status, mode controls, indexed roots,
|
||||||
|
//! and the content/ignore filter editors.
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
use quicksearch_core::coordinator::{IndexMode, IndexerState, WatcherStatus};
|
||||||
|
use quicksearch_core::indexing::{IndexingStatus, RootPhase, RootProgress};
|
||||||
|
|
||||||
|
use crate::format::{fmt_interval, fmt_rate, group_thousands, middle_truncate};
|
||||||
|
use crate::options::{config_editor_ui, Section};
|
||||||
|
use crate::tracker::SpeedTracker;
|
||||||
|
|
||||||
|
/// What the tab asks the app to do after this frame.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct ManageActions {
|
||||||
|
pub start_now: bool,
|
||||||
|
pub stop: bool,
|
||||||
|
pub auto: bool,
|
||||||
|
/// Ask the app to confirm and delete the index.
|
||||||
|
pub clear_index: bool,
|
||||||
|
/// A full edited config to apply (roots / filters / indexing knobs).
|
||||||
|
pub apply_config: Option<Config>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct ManageTab {
|
||||||
|
pub speed: SpeedTracker,
|
||||||
|
/// Multiline editors, one entry per line; synced from config on tab
|
||||||
|
/// entry and parsed back on Apply.
|
||||||
|
ext_filter_text: String,
|
||||||
|
ignore_filter_text: String,
|
||||||
|
new_root: String,
|
||||||
|
/// Inline error from a rejected root add (nested/duplicate).
|
||||||
|
root_error: Option<String>,
|
||||||
|
editors_synced: bool,
|
||||||
|
/// Draft of the indexing/processing knobs edited in-place.
|
||||||
|
draft: Option<Config>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ManageTab {
|
||||||
|
pub fn new() -> ManageTab {
|
||||||
|
ManageTab {
|
||||||
|
speed: SpeedTracker::new(),
|
||||||
|
ext_filter_text: String::new(),
|
||||||
|
ignore_filter_text: String::new(),
|
||||||
|
new_root: String::new(),
|
||||||
|
root_error: None,
|
||||||
|
editors_synced: false,
|
||||||
|
draft: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Feed the tracker from the polled status (called every frame, on
|
||||||
|
/// every tab, so the status bar rate stays live).
|
||||||
|
pub fn observe(&mut self, status: &IndexingStatus) {
|
||||||
|
match status {
|
||||||
|
IndexingStatus::Running { roots, .. } => {
|
||||||
|
// Monotonic within a run: walks and extractions only grow.
|
||||||
|
let total: usize = roots.iter().map(|r| r.walked + r.extracted).sum();
|
||||||
|
self.speed.record(total);
|
||||||
|
}
|
||||||
|
IndexingStatus::Idle | IndexingStatus::Error(_) => self.speed.reset(),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn sync_editors(&mut self, config: &Config) {
|
||||||
|
if !self.editors_synced {
|
||||||
|
self.ext_filter_text = config.indexing.content_extensions.join("\n");
|
||||||
|
self.ignore_filter_text = config.indexing.ignore_patterns.join("\n");
|
||||||
|
self.draft = Some(config.clone());
|
||||||
|
self.editors_synced = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Force a re-sync next frame (config changed elsewhere).
|
||||||
|
pub fn invalidate_editors(&mut self) {
|
||||||
|
self.editors_synced = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn ui(
|
||||||
|
&mut self,
|
||||||
|
ui: &mut egui::Ui,
|
||||||
|
state: &IndexerState,
|
||||||
|
config: &Config,
|
||||||
|
) -> ManageActions {
|
||||||
|
let mut actions = ManageActions::default();
|
||||||
|
self.sync_editors(config);
|
||||||
|
|
||||||
|
egui::ScrollArea::vertical().auto_shrink([false; 2]).show(ui, |ui| {
|
||||||
|
// --- Status ---------------------------------------------------
|
||||||
|
ui.heading("Status");
|
||||||
|
status_panel(ui, state, &self.speed);
|
||||||
|
watch_panel(ui, state, config);
|
||||||
|
ui.add_space(8.0);
|
||||||
|
|
||||||
|
// --- Controls -------------------------------------------------
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
let running = !matches!(
|
||||||
|
state.activity,
|
||||||
|
IndexingStatus::Idle | IndexingStatus::Error(_)
|
||||||
|
);
|
||||||
|
if ui.add_enabled(!running, egui::Button::new("Start indexing now")).clicked() {
|
||||||
|
actions.start_now = true;
|
||||||
|
}
|
||||||
|
if ui.add_enabled(running || state.mode == IndexMode::Auto, egui::Button::new("Stop")).clicked() {
|
||||||
|
actions.stop = true;
|
||||||
|
}
|
||||||
|
if ui
|
||||||
|
.add_enabled(state.mode != IndexMode::Auto, egui::Button::new("Return to Automatic"))
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
actions.auto = true;
|
||||||
|
}
|
||||||
|
let mode = match state.mode {
|
||||||
|
IndexMode::Auto => "Automatic",
|
||||||
|
IndexMode::ManualStopped => "Manual (stopped)",
|
||||||
|
IndexMode::ManualRunning => "Manual (running)",
|
||||||
|
};
|
||||||
|
ui.label(egui::RichText::new(format!("Mode: {}", mode)).weak());
|
||||||
|
ui.separator();
|
||||||
|
if ui
|
||||||
|
.button(egui::RichText::new("Clear index…").color(ui.visuals().error_fg_color))
|
||||||
|
.on_hover_text("Delete the index database (asks for confirmation)")
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
actions.clear_index = true;
|
||||||
|
}
|
||||||
|
if state.queued_events > 0 {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!("{} changes queued", state.queued_events))
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
// --- Indexed roots ---------------------------------------------
|
||||||
|
ui.heading("Indexed folders");
|
||||||
|
let draft = self.draft.as_mut().expect("synced");
|
||||||
|
let mut remove: Option<usize> = None;
|
||||||
|
for (i, root) in draft.paths.indexing_paths.clone().iter().enumerate() {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
// Controls claim the right edge first so a long path can
|
||||||
|
// never push them out of view; the path truncates into
|
||||||
|
// whatever width remains (full path on hover).
|
||||||
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
|
if ui.small_button("Remove").clicked() {
|
||||||
|
remove = Some(i);
|
||||||
|
}
|
||||||
|
// Per-root walker override; 0 = auto (4 local / 16
|
||||||
|
// network, detected per root). Applies on the next run.
|
||||||
|
let mut workers =
|
||||||
|
draft.indexing.root_workers.get(root).copied().unwrap_or(0);
|
||||||
|
let response = ui
|
||||||
|
.add(
|
||||||
|
egui::DragValue::new(&mut workers)
|
||||||
|
.range(0..=64)
|
||||||
|
.custom_formatter(|n, _| {
|
||||||
|
if n == 0.0 {
|
||||||
|
"auto".to_string()
|
||||||
|
} else {
|
||||||
|
format!("{:.0}", n)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.custom_parser(|s| {
|
||||||
|
let s = s.trim();
|
||||||
|
if s.is_empty() || s.eq_ignore_ascii_case("auto") {
|
||||||
|
Some(0.0)
|
||||||
|
} else {
|
||||||
|
s.parse().ok()
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.on_hover_text(
|
||||||
|
"Walker threads for this folder. auto = 4 on local \
|
||||||
|
storage, 16 on network mounts. Takes effect on \
|
||||||
|
the next indexing run.",
|
||||||
|
);
|
||||||
|
if response.changed() {
|
||||||
|
if workers == 0 {
|
||||||
|
draft.indexing.root_workers.remove(root);
|
||||||
|
} else {
|
||||||
|
draft.indexing.root_workers.insert(root.clone(), workers);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ui.label(egui::RichText::new("workers:").small().weak());
|
||||||
|
|
||||||
|
// Path label takes the leftover width, middle-truncated.
|
||||||
|
ui.with_layout(
|
||||||
|
egui::Layout::left_to_right(egui::Align::Center),
|
||||||
|
|ui| {
|
||||||
|
let font_id = egui::TextStyle::Monospace.resolve(ui.style());
|
||||||
|
let char_width =
|
||||||
|
ui.fonts(|f| f.glyph_width(&font_id, '0')).max(1.0);
|
||||||
|
let budget =
|
||||||
|
((ui.available_width() / char_width) as usize).max(16);
|
||||||
|
ui.monospace(middle_truncate(root, budget))
|
||||||
|
.on_hover_text(root);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if let Some(i) = remove {
|
||||||
|
let removed = draft.paths.indexing_paths.remove(i);
|
||||||
|
draft.indexing.root_workers.remove(&removed);
|
||||||
|
}
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
if ui.button("Add folder…").clicked() {
|
||||||
|
if let Some(dir) = rfd::FileDialog::new().pick_folder() {
|
||||||
|
let path = dir.to_string_lossy().into_owned();
|
||||||
|
try_add_root(draft, path, &mut self.root_error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ui.add(
|
||||||
|
egui::TextEdit::singleline(&mut self.new_root)
|
||||||
|
.desired_width(240.0)
|
||||||
|
.hint_text("or type a path"),
|
||||||
|
);
|
||||||
|
if ui.button("Add").clicked() && !self.new_root.trim().is_empty() {
|
||||||
|
let path = self.new_root.trim().to_string();
|
||||||
|
if try_add_root(draft, path, &mut self.root_error) {
|
||||||
|
self.new_root.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if let Some(err) = &self.root_error {
|
||||||
|
ui.colored_label(ui.visuals().error_fg_color, err);
|
||||||
|
}
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
// --- Filters ---------------------------------------------------
|
||||||
|
ui.heading("Content filters");
|
||||||
|
ui.columns(2, |cols| {
|
||||||
|
cols[0].label("Full-text extensions (empty = all supported):");
|
||||||
|
cols[0].add(
|
||||||
|
egui::TextEdit::multiline(&mut self.ext_filter_text)
|
||||||
|
.desired_rows(4)
|
||||||
|
.desired_width(f32::INFINITY)
|
||||||
|
.hint_text("txt\nmd\npdf"),
|
||||||
|
);
|
||||||
|
cols[1].label("Ignore patterns (excluded entirely):");
|
||||||
|
cols[1].add(
|
||||||
|
egui::TextEdit::multiline(&mut self.ignore_filter_text)
|
||||||
|
.desired_rows(4)
|
||||||
|
.desired_width(f32::INFINITY)
|
||||||
|
.hint_text(".git\nnode_modules\n*.tmp"),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
// --- Indexing options -------------------------------------------
|
||||||
|
ui.heading("Indexing options");
|
||||||
|
config_editor_ui(ui, draft, Section::Indexing);
|
||||||
|
ui.add_space(4.0);
|
||||||
|
config_editor_ui(ui, draft, Section::Processing);
|
||||||
|
ui.add_space(8.0);
|
||||||
|
|
||||||
|
if ui.button("Apply & Save").clicked() {
|
||||||
|
let mut new_config = draft.clone();
|
||||||
|
new_config.indexing.content_extensions = parse_lines(&self.ext_filter_text);
|
||||||
|
new_config.indexing.ignore_patterns = parse_lines(&self.ignore_filter_text);
|
||||||
|
let roots = new_config.paths.indexing_paths.clone();
|
||||||
|
new_config
|
||||||
|
.indexing
|
||||||
|
.root_workers
|
||||||
|
.retain(|root, _| roots.contains(root));
|
||||||
|
actions.apply_config = Some(new_config);
|
||||||
|
self.editors_synced = false;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
actions
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Append a root to the draft unless it would duplicate or nest with an
|
||||||
|
/// existing one; the rejection reason lands in `error`.
|
||||||
|
fn try_add_root(draft: &mut Config, candidate: String, error: &mut Option<String>) -> bool {
|
||||||
|
if draft.paths.indexing_paths.contains(&candidate) {
|
||||||
|
*error = Some(format!("{} is already in the list", candidate));
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let mut probe = draft.paths.indexing_paths.clone();
|
||||||
|
probe.push(candidate.clone());
|
||||||
|
if let Some((child, parent)) = quicksearch_core::config::nested_roots(&probe).first() {
|
||||||
|
*error = Some(format!(
|
||||||
|
"Not added: {} is nested under {}; indexed folders may not overlap",
|
||||||
|
child, parent
|
||||||
|
));
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
draft.paths.indexing_paths.push(candidate);
|
||||||
|
*error = None;
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_lines(text: &str) -> Vec<String> {
|
||||||
|
text.lines()
|
||||||
|
.map(str::trim)
|
||||||
|
.filter(|l| !l.is_empty())
|
||||||
|
.map(str::to_string)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Live-update health. Permanent counterpart to the one-time modal: the
|
||||||
|
/// modal is dismissed and remembered per root, but "live updates are off"
|
||||||
|
/// stays true and must remain discoverable.
|
||||||
|
fn watch_panel(ui: &mut egui::Ui, state: &IndexerState, config: &Config) {
|
||||||
|
match &state.watcher {
|
||||||
|
// Manual mode already says "stopped" in the controls row; repeating
|
||||||
|
// it here would be noise.
|
||||||
|
WatcherStatus::Off => {}
|
||||||
|
WatcherStatus::Starting => {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new("Setting up live updates…")
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
WatcherStatus::Active { dirs } => {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!(
|
||||||
|
"Live updates on, watching {} folders",
|
||||||
|
group_thousands(*dirs as u64)
|
||||||
|
))
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
WatcherStatus::Disabled { reason } => {
|
||||||
|
ui.colored_label(
|
||||||
|
ui.visuals().warn_fg_color,
|
||||||
|
format!(
|
||||||
|
"⚠ Live updates off; reindexing every {}",
|
||||||
|
fmt_interval(config.indexing.reindex_interval_minutes)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.on_hover_text(reason.to_string());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn status_panel(ui: &mut egui::Ui, state: &IndexerState, speed: &SpeedTracker) {
|
||||||
|
match &state.activity {
|
||||||
|
IndexingStatus::Idle => {
|
||||||
|
// Relative wording makes even a milliseconds-fast run visibly
|
||||||
|
// register ("just now") instead of looking like a dead button.
|
||||||
|
let last = state
|
||||||
|
.last_full_index
|
||||||
|
.map(crate::format::fmt_ago)
|
||||||
|
.unwrap_or_else(|| "never".to_string());
|
||||||
|
ui.label(format!("Idle; last full index: {}", last));
|
||||||
|
}
|
||||||
|
IndexingStatus::Error(e) => {
|
||||||
|
ui.colored_label(ui.visuals().error_fg_color, format!("Error: {}", e));
|
||||||
|
}
|
||||||
|
IndexingStatus::Stopping => {
|
||||||
|
ui.label("Stopping…");
|
||||||
|
}
|
||||||
|
IndexingStatus::Running { roots, .. } => {
|
||||||
|
for root in roots {
|
||||||
|
root_row(ui, root);
|
||||||
|
}
|
||||||
|
if let Some(rate) = speed.files_per_sec() {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!("overall: {}", fmt_rate(rate)))
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One root's progress: path, phase, bar, counters, current file.
|
||||||
|
fn root_row(ui: &mut egui::Ui, r: &RootProgress) {
|
||||||
|
// Weak "|" separators split the row into folder | status | numbers.
|
||||||
|
let divider = |ui: &mut egui::Ui| {
|
||||||
|
ui.label(egui::RichText::new("|").weak());
|
||||||
|
};
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.monospace(middle_truncate(&r.root, 48));
|
||||||
|
divider(ui);
|
||||||
|
match r.phase {
|
||||||
|
RootPhase::Walking => {
|
||||||
|
ui.label("indexing");
|
||||||
|
divider(ui);
|
||||||
|
let workers = format!("{}/{} workers", r.active_workers, r.total_workers);
|
||||||
|
match r.walk_total {
|
||||||
|
Some(total) if total > 0 => {
|
||||||
|
let frac = (r.walked as f32 / total as f32).clamp(0.0, 1.0);
|
||||||
|
ui.label(format!(
|
||||||
|
"{} / {} ({:.0}%) · {}",
|
||||||
|
group_thousands(r.walked as u64),
|
||||||
|
group_thousands(total as u64),
|
||||||
|
frac * 100.0,
|
||||||
|
workers
|
||||||
|
));
|
||||||
|
ui.add(egui::ProgressBar::new(frac).desired_width(160.0));
|
||||||
|
}
|
||||||
|
_ => {
|
||||||
|
ui.label(format!(
|
||||||
|
"{} files · {}",
|
||||||
|
group_thousands(r.walked as u64),
|
||||||
|
workers
|
||||||
|
));
|
||||||
|
ui.add(egui::ProgressBar::new(0.0).animate(true).desired_width(160.0));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
RootPhase::Extracting => {
|
||||||
|
ui.label("extracting text for search");
|
||||||
|
divider(ui);
|
||||||
|
let frac = if r.extract_total > 0 {
|
||||||
|
(r.extracted as f32 / r.extract_total as f32).clamp(0.0, 1.0)
|
||||||
|
} else {
|
||||||
|
1.0
|
||||||
|
};
|
||||||
|
ui.label(format!(
|
||||||
|
"{} / {} ({:.0}%)",
|
||||||
|
group_thousands(r.extracted as u64),
|
||||||
|
group_thousands(r.extract_total as u64),
|
||||||
|
frac * 100.0
|
||||||
|
));
|
||||||
|
ui.add(egui::ProgressBar::new(frac).desired_width(160.0));
|
||||||
|
}
|
||||||
|
RootPhase::Done => {
|
||||||
|
// Whole-root totals: `walked` covers every file the walk
|
||||||
|
// saw (including unchanged, skipped ones) and `extracted`
|
||||||
|
// covers all rows with searchable text, not just this
|
||||||
|
// run's new work.
|
||||||
|
ui.label("done");
|
||||||
|
divider(ui);
|
||||||
|
ui.label(format!(
|
||||||
|
"indexed {}, extracted {}",
|
||||||
|
group_thousands(r.walked as u64),
|
||||||
|
group_thousands(r.extracted as u64)
|
||||||
|
));
|
||||||
|
ui.add(egui::ProgressBar::new(1.0).desired_width(160.0));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if let Some(f) = &r.current_file {
|
||||||
|
ui.label(egui::RichText::new(middle_truncate(f, 90)).small().weak());
|
||||||
|
}
|
||||||
|
}
|
||||||
251
crates/quicksearch-gui/src/options.rs
Normal file
|
|
@ -0,0 +1,251 @@
|
||||||
|
//! The Options window and the shared config editor used by both the
|
||||||
|
//! window and the Manage Index tab. Edits happen on a draft; Apply
|
||||||
|
//! validates, saves, and hands the new config to the app.
|
||||||
|
|
||||||
|
use quicksearch_core::config::Config;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum Section {
|
||||||
|
Indexing,
|
||||||
|
Processing,
|
||||||
|
Search,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct OptionsWindow {
|
||||||
|
pub open: bool,
|
||||||
|
draft: Option<Config>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OptionsWindow {
|
||||||
|
pub fn new() -> OptionsWindow {
|
||||||
|
OptionsWindow {
|
||||||
|
open: false,
|
||||||
|
draft: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn open_with(&mut self, current: &Config) {
|
||||||
|
self.open = true;
|
||||||
|
self.draft = Some(current.clone());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Render; returns a new config when the user applied changes.
|
||||||
|
pub fn ui(&mut self, ctx: &egui::Context, current: &Config) -> Option<Config> {
|
||||||
|
if !self.open {
|
||||||
|
self.draft = None;
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
if self.draft.is_none() {
|
||||||
|
self.draft = Some(current.clone());
|
||||||
|
}
|
||||||
|
let mut applied = None;
|
||||||
|
let mut open = self.open;
|
||||||
|
let draft = self.draft.as_mut().unwrap();
|
||||||
|
|
||||||
|
egui::Window::new("Options")
|
||||||
|
.open(&mut open)
|
||||||
|
.resizable(false)
|
||||||
|
.default_width(420.0)
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
egui::ScrollArea::vertical().max_height(480.0).show(ui, |ui| {
|
||||||
|
ui.heading("Paths");
|
||||||
|
egui::Grid::new("opt-paths").num_columns(2).show(ui, |ui| {
|
||||||
|
ui.label("Database file");
|
||||||
|
ui.add(
|
||||||
|
egui::TextEdit::singleline(&mut draft.paths.database_path)
|
||||||
|
.desired_width(260.0),
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
});
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Indexed folders are managed on the Manage Index tab.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
ui.heading("Indexing");
|
||||||
|
config_editor_ui(ui, draft, Section::Indexing);
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
ui.heading("Processing");
|
||||||
|
config_editor_ui(ui, draft, Section::Processing);
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
ui.heading("Search");
|
||||||
|
config_editor_ui(ui, draft, Section::Search);
|
||||||
|
ui.separator();
|
||||||
|
|
||||||
|
ui.heading("Interface");
|
||||||
|
egui::Grid::new("opt-ui").num_columns(2).show(ui, |ui| {
|
||||||
|
ui.label("UI scale");
|
||||||
|
ui.add(
|
||||||
|
egui::Slider::new(&mut draft.ui.scale, 0.5..=2.5)
|
||||||
|
.step_by(0.05)
|
||||||
|
.fixed_decimals(2),
|
||||||
|
)
|
||||||
|
.on_hover_text(
|
||||||
|
"Zooms the whole interface: fonts, spacing, and \
|
||||||
|
widgets. Ctrl +/- and Ctrl 0 adjust it temporarily \
|
||||||
|
at runtime.",
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
ui.separator();
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
if ui.button("Apply & Save").clicked() {
|
||||||
|
applied = Some(draft.clone());
|
||||||
|
}
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Changes to tokenizer, filters, hidden files, or hashing \
|
||||||
|
prompt an index rebuild.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
self.open = open;
|
||||||
|
if !self.open {
|
||||||
|
self.draft = None;
|
||||||
|
}
|
||||||
|
applied
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One implementation of the per-section config controls, shared by the
|
||||||
|
/// Options window and the Manage tab.
|
||||||
|
pub fn config_editor_ui(ui: &mut egui::Ui, config: &mut Config, section: Section) {
|
||||||
|
match section {
|
||||||
|
Section::Indexing => {
|
||||||
|
egui::Grid::new("cfg-indexing").num_columns(2).show(ui, |ui| {
|
||||||
|
ui.label("Automatic indexing");
|
||||||
|
ui.checkbox(&mut config.indexing.auto_index, "watchers + periodic reindex");
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Full reindex every");
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.add(
|
||||||
|
egui::DragValue::new(&mut config.indexing.reindex_interval_minutes)
|
||||||
|
.range(5..=60 * 24 * 30),
|
||||||
|
);
|
||||||
|
ui.label("minutes");
|
||||||
|
});
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Follow symlinks");
|
||||||
|
ui.checkbox(&mut config.indexing.follow_symlinks, "");
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Include hidden files");
|
||||||
|
ui.checkbox(&mut config.indexing.include_hidden, "");
|
||||||
|
ui.end_row();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
Section::Processing => {
|
||||||
|
egui::Grid::new("cfg-processing").num_columns(2).show(ui, |ui| {
|
||||||
|
ui.label("Tokenizer");
|
||||||
|
egui::ComboBox::from_id_salt("cfg-tokenize")
|
||||||
|
.selected_text(&config.processing.tokenize)
|
||||||
|
.show_ui(ui, |ui| {
|
||||||
|
for opt in ["trigram", "unicode61", "porter"] {
|
||||||
|
ui.selectable_value(
|
||||||
|
&mut config.processing.tokenize,
|
||||||
|
opt.to_string(),
|
||||||
|
opt,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Hash sample size (bytes)");
|
||||||
|
ui.add(egui::DragValue::new(&mut config.processing.hash_length).range(512..=1_048_576));
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Max stored text (bytes)");
|
||||||
|
ui.add(
|
||||||
|
egui::DragValue::new(&mut config.processing.maximum_text_size)
|
||||||
|
.range(1024..=16_777_216),
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Max text file size (bytes)");
|
||||||
|
ui.add(
|
||||||
|
egui::DragValue::new(&mut config.processing.maximum_text_file_size)
|
||||||
|
.range(1024..=1_073_741_824),
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Batch size");
|
||||||
|
ui.add(egui::DragValue::new(&mut config.processing.batch_size).range(10..=100_000));
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Store text for snippets");
|
||||||
|
ui.checkbox(&mut config.processing.store_text_for_snippets, "")
|
||||||
|
.on_hover_text(
|
||||||
|
"Off: smaller index, but no previews, occurrence ranking, \
|
||||||
|
case verification, or fuzzy full-text search",
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
Section::Search => {
|
||||||
|
egui::Grid::new("cfg-search").num_columns(2).show(ui, |ui| {
|
||||||
|
ui.label("Fuzzy stages on by default");
|
||||||
|
ui.checkbox(&mut config.search.fuzzy_default, "");
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Fuzzy edit distance");
|
||||||
|
ui.vertical(|ui| {
|
||||||
|
ui.add(egui::DragValue::new(&mut config.search.fuzzy_max_edits).range(0..=8))
|
||||||
|
.on_hover_text(
|
||||||
|
"Ceiling on the typo budget. The allowance grows with the \
|
||||||
|
search term, one edit per three characters, up to this \
|
||||||
|
value, so 2 means \"1 edit for short terms, 2 for longer \
|
||||||
|
ones\". 0 turns the fuzzy stages off.",
|
||||||
|
);
|
||||||
|
if let Some(warning) = config.search.fuzzy_edits_warning() {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(warning)
|
||||||
|
.small()
|
||||||
|
.color(egui::Color32::from_rgb(220, 150, 40)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Display limit");
|
||||||
|
ui.add(egui::DragValue::new(&mut config.search.display_limit).range(50..=100_000));
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Stream batch size");
|
||||||
|
ui.add(
|
||||||
|
egui::DragValue::new(&mut config.search.results_per_page).range(10..=10_000),
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Debounce (ms)");
|
||||||
|
ui.add(egui::DragValue::new(&mut config.search.debounce_ms).range(0..=2000));
|
||||||
|
ui.end_row();
|
||||||
|
|
||||||
|
ui.label("Fuzzy max edits");
|
||||||
|
ui.add(egui::DragValue::new(&mut config.search.fuzzy_max_edits).range(0..=8))
|
||||||
|
.on_hover_text(
|
||||||
|
"Ceiling on fuzzy edit distance (the budget grows one \
|
||||||
|
edit per three characters of the term). 0 disables \
|
||||||
|
the fuzzy passes.",
|
||||||
|
);
|
||||||
|
ui.end_row();
|
||||||
|
});
|
||||||
|
if let Some(warning) = config.search.fuzzy_edits_warning() {
|
||||||
|
ui.colored_label(ui.visuals().warn_fg_color, warning);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
134
crates/quicksearch-gui/src/platform.rs
Normal file
|
|
@ -0,0 +1,134 @@
|
||||||
|
//! Opening files, revealing them in the system file manager, and the one bit
|
||||||
|
//! of process setup that has to happen before anything prints.
|
||||||
|
|
||||||
|
use std::process::Command;
|
||||||
|
|
||||||
|
/// Give the process somewhere to write when it has no stdio.
|
||||||
|
///
|
||||||
|
/// A window-subsystem binary launched from Explorer has NULL standard handles,
|
||||||
|
/// and `println!`/`eprintln!` *panic* when the write fails rather than
|
||||||
|
/// dropping the output. Pointing the handles at `NUL` makes those writes
|
||||||
|
/// succeed and go nowhere.
|
||||||
|
///
|
||||||
|
/// Background reporting no longer depends on this — it goes through
|
||||||
|
/// [`quicksearch_core::log`], which ignores a failed stderr write and keeps
|
||||||
|
/// the line for the Logs tab — but the remaining direct prints (a startup
|
||||||
|
/// failure, a panic message) still reach a handle that accepts them.
|
||||||
|
///
|
||||||
|
/// Handles inherited from a real console are left alone, so running the binary
|
||||||
|
/// from a shell still prints normally.
|
||||||
|
#[cfg(windows)]
|
||||||
|
pub fn redirect_null_stdio() {
|
||||||
|
use std::os::windows::io::IntoRawHandle;
|
||||||
|
use windows_sys::Win32::Foundation::INVALID_HANDLE_VALUE;
|
||||||
|
use windows_sys::Win32::System::Console::{
|
||||||
|
GetStdHandle, SetStdHandle, STD_ERROR_HANDLE, STD_OUTPUT_HANDLE,
|
||||||
|
};
|
||||||
|
|
||||||
|
for id in [STD_OUTPUT_HANDLE, STD_ERROR_HANDLE] {
|
||||||
|
let existing = unsafe { GetStdHandle(id) };
|
||||||
|
if !existing.is_null() && existing != INVALID_HANDLE_VALUE {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if let Ok(file) = std::fs::OpenOptions::new().write(true).open("NUL") {
|
||||||
|
// Deliberately leaked: the handle has to outlive every later
|
||||||
|
// write, which means the whole process.
|
||||||
|
unsafe { SetStdHandle(id, file.into_raw_handle() as _) };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Open a file with the desktop's default application, detached.
|
||||||
|
pub fn open_file(path: &str) {
|
||||||
|
if let Err(e) = open::that_detached(path) {
|
||||||
|
quicksearch_core::log_warn!("open {}: {}", path, e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reveal a file in the system file manager with the file selected.
|
||||||
|
///
|
||||||
|
/// Linux: `org.freedesktop.FileManager1.ShowItems` over the session bus
|
||||||
|
/// (supported by every mainstream file manager) via `dbus-send` — no
|
||||||
|
/// D-Bus library dependency for one call. Falls back to opening the
|
||||||
|
/// parent directory. Windows/macOS use their native select verbs.
|
||||||
|
pub fn reveal_in_folder(path: &str) {
|
||||||
|
#[cfg(target_os = "windows")]
|
||||||
|
{
|
||||||
|
use std::os::windows::process::CommandExt;
|
||||||
|
/// Keep a console window from flashing behind the spawn.
|
||||||
|
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
|
||||||
|
|
||||||
|
// explorer.exe parses its own command line rather than using the
|
||||||
|
// standard argv splitting, and wants `/select,` glued to the path as a
|
||||||
|
// single token with quotes around the path only. Passed as two
|
||||||
|
// arguments it ignores the selection and just opens the folder, and
|
||||||
|
// std's quoting would wrap the whole token. `raw_arg` is the only way
|
||||||
|
// to say exactly this.
|
||||||
|
//
|
||||||
|
// Forward slashes are valid everywhere else on Windows but not here,
|
||||||
|
// so normalize first. The exit code is not worth checking: explorer
|
||||||
|
// returns 1 even on success.
|
||||||
|
let native = path.replace('/', "\\");
|
||||||
|
let _ = Command::new("explorer.exe")
|
||||||
|
.raw_arg(format!("/select,\"{}\"", native))
|
||||||
|
.creation_flags(CREATE_NO_WINDOW)
|
||||||
|
.spawn();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
#[cfg(target_os = "macos")]
|
||||||
|
{
|
||||||
|
let _ = Command::new("open").arg("-R").arg(path).spawn();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
#[cfg(all(unix, not(target_os = "macos")))]
|
||||||
|
{
|
||||||
|
use std::path::Path;
|
||||||
|
let uri = format!("file://{}", uri_escape_path(path));
|
||||||
|
let ok = Command::new("dbus-send")
|
||||||
|
.args([
|
||||||
|
"--session",
|
||||||
|
"--print-reply",
|
||||||
|
"--dest=org.freedesktop.FileManager1",
|
||||||
|
"/org/freedesktop/FileManager1",
|
||||||
|
"org.freedesktop.FileManager1.ShowItems",
|
||||||
|
&format!("array:string:{}", uri),
|
||||||
|
"string:",
|
||||||
|
])
|
||||||
|
.output()
|
||||||
|
.map(|out| out.status.success())
|
||||||
|
.unwrap_or(false);
|
||||||
|
if !ok {
|
||||||
|
let parent = Path::new(path).parent().unwrap_or(Path::new("/"));
|
||||||
|
let _ = Command::new("xdg-open").arg(parent).spawn();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Percent-encode a filesystem path for a file:// URI, keeping `/`.
|
||||||
|
#[cfg(all(unix, not(target_os = "macos")))]
|
||||||
|
fn uri_escape_path(path: &str) -> String {
|
||||||
|
let mut out = String::with_capacity(path.len());
|
||||||
|
for byte in path.bytes() {
|
||||||
|
match byte {
|
||||||
|
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'/' | b'-' | b'_' | b'.' | b'~' => {
|
||||||
|
out.push(byte as char)
|
||||||
|
}
|
||||||
|
_ => out.push_str(&format!("%{:02X}", byte)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
#[cfg(all(unix, not(target_os = "macos")))]
|
||||||
|
#[test]
|
||||||
|
fn uri_escaping() {
|
||||||
|
use super::uri_escape_path;
|
||||||
|
assert_eq!(uri_escape_path("/plain/path.txt"), "/plain/path.txt");
|
||||||
|
assert_eq!(
|
||||||
|
uri_escape_path("/with space/ünïcode&.txt"),
|
||||||
|
"/with%20space/%C3%BCn%C3%AFcode%26.txt"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
739
crates/quicksearch-gui/src/query_highlight.rs
Normal file
|
|
@ -0,0 +1,739 @@
|
||||||
|
//! Syntax highlighting for the search box.
|
||||||
|
//!
|
||||||
|
//! [`classify`] is a pure token walk over [`tokenize_spanned`] output that
|
||||||
|
//! mirrors `split_for_cascade` branch for branch — it must never claim
|
||||||
|
//! something is a filter (or a wildcard) that the engine treats as plain
|
||||||
|
//! text. The egui layer at the bottom turns its segments into a `Galley`
|
||||||
|
//! for `TextEdit::layouter`.
|
||||||
|
//!
|
||||||
|
//! Color scheme: recognized keywords red, their arguments blue, syntax
|
||||||
|
//! characters (operators, quotes, live wildcards) green, invalid arguments
|
||||||
|
//! in the error color, everything else plain. A complete recognized filter
|
||||||
|
//! additionally gets a tinted background chip.
|
||||||
|
|
||||||
|
use std::ops::Range;
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use egui::text::{LayoutJob, TextFormat};
|
||||||
|
use egui::{Color32, Galley, Stroke};
|
||||||
|
use quicksearch_core::query::ast::Op;
|
||||||
|
use quicksearch_core::query::lexer::{tokenize_spanned, Token};
|
||||||
|
use quicksearch_core::query::pattern::RegexQuery;
|
||||||
|
use quicksearch_core::query::translator::{build_filter, is_filter_key};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum Class {
|
||||||
|
Plain,
|
||||||
|
/// The key word of a recognized filter (`type`, `name`, `regex`, …).
|
||||||
|
Keyword,
|
||||||
|
/// Syntax characters doing work: filter operators (`:`, `:>=`, …),
|
||||||
|
/// quote delimiters, and `*` where it is a live wildcard.
|
||||||
|
Operator,
|
||||||
|
/// The value of a recognized filter.
|
||||||
|
Argument,
|
||||||
|
/// The value of a recognized filter that the engine would reject
|
||||||
|
/// (unknown type name, bad date, invalid regex).
|
||||||
|
InvalidArg,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
|
pub struct Seg {
|
||||||
|
pub range: Range<usize>,
|
||||||
|
pub class: Class,
|
||||||
|
/// Part of a complete recognized filter — drawn on the chip tint.
|
||||||
|
pub chip: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Classify `text` into contiguous segments tiling `0..text.len()`.
|
||||||
|
pub fn classify(text: &str) -> Vec<Seg> {
|
||||||
|
let (tokens, err) = tokenize_spanned(text);
|
||||||
|
let mut em = Emitter {
|
||||||
|
text,
|
||||||
|
cursor: 0,
|
||||||
|
segs: Vec::new(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let tok = |i: usize| tokens.get(i).map(|(t, _)| t);
|
||||||
|
let span = |i: usize| tokens[i].1.clone();
|
||||||
|
|
||||||
|
let mut i = 0usize;
|
||||||
|
// The engine allows one `regex:` per query; later ones are errors.
|
||||||
|
let mut regex_seen = false;
|
||||||
|
|
||||||
|
while i < tokens.len() {
|
||||||
|
match &tokens[i].0 {
|
||||||
|
Token::Word(word) => {
|
||||||
|
if let Some(Token::Op(op1)) = tok(i + 1) {
|
||||||
|
// Candidate filter: Word(key) Op [Op] (Word|Quoted),
|
||||||
|
// exactly as split_for_cascade sees it.
|
||||||
|
let (op, op_end_idx, value_idx) = match tok(i + 2) {
|
||||||
|
Some(Token::Op(op2)) => (*op2, i + 2, i + 3),
|
||||||
|
_ => (*op1, i + 1, i + 2),
|
||||||
|
};
|
||||||
|
let value = match tok(value_idx) {
|
||||||
|
Some(Token::Word(v)) | Some(Token::Quoted(v)) => Some(v.clone()),
|
||||||
|
_ => None,
|
||||||
|
};
|
||||||
|
let is_regex = word.eq_ignore_ascii_case("regex");
|
||||||
|
if let Some(value) = value {
|
||||||
|
let value_is_word = matches!(tok(value_idx), Some(Token::Word(_)));
|
||||||
|
if is_regex || is_filter_key(word) {
|
||||||
|
let valid = if is_regex {
|
||||||
|
let first = !regex_seen;
|
||||||
|
regex_seen = true;
|
||||||
|
first
|
||||||
|
&& op == Op::Contains
|
||||||
|
&& RegexQuery::new(&value).is_ok()
|
||||||
|
} else {
|
||||||
|
build_filter(word, op, &value, value_is_word).is_ok()
|
||||||
|
};
|
||||||
|
em.emit(span(i), Class::Keyword, true, false);
|
||||||
|
for op_idx in (i + 1)..=op_end_idx {
|
||||||
|
em.emit(span(op_idx), Class::Operator, true, true);
|
||||||
|
}
|
||||||
|
let vspan = span(value_idx);
|
||||||
|
if !valid {
|
||||||
|
// One uniform error run reads better than
|
||||||
|
// error-with-green-sprinkles.
|
||||||
|
em.emit(vspan, Class::InvalidArg, true, true);
|
||||||
|
} else if !value_is_word {
|
||||||
|
em.emit_quoted(vspan, Class::Argument, true);
|
||||||
|
} else if glob_value_key(word) {
|
||||||
|
em.emit_word(vspan, Class::Argument, true, true);
|
||||||
|
} else {
|
||||||
|
// Stars in other filter values are literal
|
||||||
|
// characters — no wildcard color.
|
||||||
|
em.emit(vspan, Class::Argument, true, true);
|
||||||
|
}
|
||||||
|
i = value_idx + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Unrecognized key: the engine reassembles the whole
|
||||||
|
// chain verbatim (stars stay literal), so everything
|
||||||
|
// renders plain — that absence of color is how the
|
||||||
|
// user learns `foo:` is not a filter.
|
||||||
|
em.emit(span(i), Class::Plain, false, false);
|
||||||
|
for op_idx in (i + 1)..=op_end_idx {
|
||||||
|
em.emit(span(op_idx), Class::Plain, false, false);
|
||||||
|
}
|
||||||
|
em.emit_glued_value(span(value_idx), tok(value_idx));
|
||||||
|
i = value_idx + 1;
|
||||||
|
while let Some(Token::Op(_)) = tok(i) {
|
||||||
|
em.emit(span(i), Class::Plain, false, false);
|
||||||
|
i += 1;
|
||||||
|
if let Some(Token::Word(_)) | Some(Token::Quoted(_)) = tok(i) {
|
||||||
|
em.emit_glued_value(span(i), tok(i));
|
||||||
|
i += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Key + op with no value yet (mid-typing `type:`).
|
||||||
|
// Recognized keys color optimistically — instant
|
||||||
|
// feedback that the key landed — but earn no chip
|
||||||
|
// until the filter is complete.
|
||||||
|
let known = is_regex || is_filter_key(word);
|
||||||
|
let (key_class, op_class) = if known {
|
||||||
|
(Class::Keyword, Class::Operator)
|
||||||
|
} else {
|
||||||
|
(Class::Plain, Class::Plain)
|
||||||
|
};
|
||||||
|
em.emit(span(i), key_class, false, false);
|
||||||
|
for op_idx in (i + 1)..=op_end_idx {
|
||||||
|
em.emit(span(op_idx), op_class, false, false);
|
||||||
|
}
|
||||||
|
i = op_end_idx + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// A plain word: unquoted stars are live wildcards.
|
||||||
|
em.emit_word(span(i), Class::Plain, false, false);
|
||||||
|
}
|
||||||
|
Token::Quoted(_) => em.emit_quoted(span(i), Class::Plain, false),
|
||||||
|
// Demoted to plain text by the live search path — coloring
|
||||||
|
// them as operators would lie.
|
||||||
|
Token::And | Token::Or | Token::LParen | Token::RParen | Token::Op(_) => {
|
||||||
|
em.emit(span(i), Class::Plain, false, false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
i += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Trailing lex error: an unterminated quote is a quote-in-progress,
|
||||||
|
// not a mistake — green delimiter, plain tail.
|
||||||
|
if let Some(err) = err {
|
||||||
|
if err.offset < text.len() && text.as_bytes()[err.offset] == b'"' {
|
||||||
|
em.emit(err.offset..err.offset + 1, Class::Operator, false, false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
em.finish(text.len())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Keys whose word-form values interpret `*` as a wildcard (or, for
|
||||||
|
/// `regex`, as live pattern syntax).
|
||||||
|
fn glob_value_key(key: &str) -> bool {
|
||||||
|
matches!(
|
||||||
|
key.to_ascii_lowercase().as_str(),
|
||||||
|
"name" | "filename" | "regex"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Emitter<'a> {
|
||||||
|
text: &'a str,
|
||||||
|
cursor: usize,
|
||||||
|
segs: Vec<Seg>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Emitter<'_> {
|
||||||
|
/// Fill the gap (whitespace the lexer skipped) up to `pos`.
|
||||||
|
fn gap_to(&mut self, pos: usize, chip: bool) {
|
||||||
|
if pos > self.cursor {
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: self.cursor..pos,
|
||||||
|
class: Class::Plain,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
self.cursor = pos;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Emit one span. `gap_chip` tints the whitespace before it — true for
|
||||||
|
/// the interior of a filter (`type : Audio` chips as one run).
|
||||||
|
fn emit(&mut self, range: Range<usize>, class: Class, chip: bool, gap_chip: bool) {
|
||||||
|
self.gap_to(range.start, gap_chip);
|
||||||
|
if range.end > range.start {
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: range.clone(),
|
||||||
|
class,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
self.cursor = range.end;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Emit a word span with each `*` as a green wildcard and the pieces
|
||||||
|
/// between in `base`.
|
||||||
|
fn emit_word(&mut self, range: Range<usize>, base: Class, chip: bool, gap_chip: bool) {
|
||||||
|
self.gap_to(range.start, gap_chip);
|
||||||
|
let bytes = self.text.as_bytes();
|
||||||
|
let mut piece_start = range.start;
|
||||||
|
for pos in range.clone() {
|
||||||
|
if bytes[pos] == b'*' {
|
||||||
|
if pos > piece_start {
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: piece_start..pos,
|
||||||
|
class: base,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: pos..pos + 1,
|
||||||
|
class: Class::Operator,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
piece_start = pos + 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if range.end > piece_start {
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: piece_start..range.end,
|
||||||
|
class: base,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
self.cursor = self.cursor.max(range.end);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Emit a quoted span (delimiters included): quotes green, content in
|
||||||
|
/// `content`. Inner `""` escapes are just content bytes — no offset
|
||||||
|
/// math needed.
|
||||||
|
fn emit_quoted(&mut self, range: Range<usize>, content: Class, chip: bool) {
|
||||||
|
self.gap_to(range.start, chip);
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: range.start..range.start + 1,
|
||||||
|
class: Class::Operator,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
if range.end - range.start > 2 {
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: range.start + 1..range.end - 1,
|
||||||
|
class: content,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if range.end - range.start >= 2 {
|
||||||
|
self.segs.push(Seg {
|
||||||
|
range: range.end - 1..range.end,
|
||||||
|
class: Class::Operator,
|
||||||
|
chip,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
self.cursor = self.cursor.max(range.end);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A value inside unrecognized-key glue: plain, except quote
|
||||||
|
/// delimiters, which still did real tokenizing work.
|
||||||
|
fn emit_glued_value(&mut self, range: Range<usize>, token: Option<&Token>) {
|
||||||
|
match token {
|
||||||
|
Some(Token::Quoted(_)) => self.emit_quoted(range, Class::Plain, false),
|
||||||
|
_ => self.emit(range, Class::Plain, false, false),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn finish(mut self, len: usize) -> Vec<Seg> {
|
||||||
|
self.gap_to(len, false);
|
||||||
|
self.segs
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// egui layer
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
struct QueryPalette {
|
||||||
|
keyword: Color32,
|
||||||
|
argument: Color32,
|
||||||
|
operator: Color32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// GitHub Primer syntax colors — readable on egui's near-black and white
|
||||||
|
/// text-field backgrounds. Same convention as `rank_tier_color`.
|
||||||
|
fn query_palette(dark_mode: bool) -> QueryPalette {
|
||||||
|
if dark_mode {
|
||||||
|
QueryPalette {
|
||||||
|
keyword: Color32::from_rgb(255, 123, 114),
|
||||||
|
argument: Color32::from_rgb(121, 192, 255),
|
||||||
|
operator: Color32::from_rgb(126, 231, 135),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
QueryPalette {
|
||||||
|
keyword: Color32::from_rgb(207, 34, 46),
|
||||||
|
argument: Color32::from_rgb(5, 80, 174),
|
||||||
|
operator: Color32::from_rgb(26, 127, 55),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct QueryFormats {
|
||||||
|
plain: TextFormat,
|
||||||
|
keyword: TextFormat,
|
||||||
|
operator: TextFormat,
|
||||||
|
argument: TextFormat,
|
||||||
|
invalid: TextFormat,
|
||||||
|
chip_bg: Color32,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn query_formats(ui: &egui::Ui) -> QueryFormats {
|
||||||
|
let font_id = egui::TextStyle::Body.resolve(ui.style());
|
||||||
|
let palette = query_palette(ui.visuals().dark_mode);
|
||||||
|
let base = |color: Color32| TextFormat {
|
||||||
|
font_id: font_id.clone(),
|
||||||
|
color,
|
||||||
|
..Default::default()
|
||||||
|
};
|
||||||
|
let error = ui.visuals().error_fg_color;
|
||||||
|
QueryFormats {
|
||||||
|
plain: base(ui.visuals().text_color()),
|
||||||
|
keyword: base(palette.keyword),
|
||||||
|
operator: base(palette.operator),
|
||||||
|
argument: base(palette.argument),
|
||||||
|
// The keyword red and the error red are near neighbors in dark
|
||||||
|
// mode; the underline disambiguates at a glance.
|
||||||
|
invalid: TextFormat {
|
||||||
|
underline: Stroke::new(1.0, error),
|
||||||
|
..base(error)
|
||||||
|
},
|
||||||
|
// Slightly weaker than the snippet highlight's 0.4 so the colored
|
||||||
|
// text on top stays crisp.
|
||||||
|
chip_bg: ui.visuals().selection.bg_fill.gamma_multiply(0.35),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl QueryFormats {
|
||||||
|
fn format_for(&self, seg: &Seg) -> TextFormat {
|
||||||
|
let mut fmt = match seg.class {
|
||||||
|
Class::Plain => self.plain.clone(),
|
||||||
|
Class::Keyword => self.keyword.clone(),
|
||||||
|
Class::Operator => self.operator.clone(),
|
||||||
|
Class::Argument => self.argument.clone(),
|
||||||
|
Class::InvalidArg => self.invalid.clone(),
|
||||||
|
};
|
||||||
|
if seg.chip {
|
||||||
|
fmt.background = self.chip_bg;
|
||||||
|
}
|
||||||
|
fmt
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Classification cache: tokenizing is cheap but validating a `regex:`
|
||||||
|
/// argument compiles the regex, and the layouter runs every frame — so
|
||||||
|
/// segments are recomputed only when the text changes. The `LayoutJob` is
|
||||||
|
/// rebuilt each frame (colors follow the live theme) and epaint's own
|
||||||
|
/// galley cache dedupes the actual layout work by job hash.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct HighlightCache {
|
||||||
|
text: String,
|
||||||
|
segs: Vec<Seg>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn galley(ui: &egui::Ui, cache: &mut HighlightCache, text: &str) -> Arc<Galley> {
|
||||||
|
if cache.text != text {
|
||||||
|
cache.text = text.to_owned();
|
||||||
|
cache.segs = classify(text);
|
||||||
|
}
|
||||||
|
let fmts = query_formats(ui);
|
||||||
|
let mut job = LayoutJob::default();
|
||||||
|
for seg in &cache.segs {
|
||||||
|
job.append(&text[seg.range.clone()], 0.0, fmts.format_for(seg));
|
||||||
|
}
|
||||||
|
ui.fonts(|f| f.layout_job(job))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
/// Readable projection: (slice, class, chip) per segment.
|
||||||
|
fn segs(text: &str) -> Vec<(String, Class, bool)> {
|
||||||
|
assert_tiles(text);
|
||||||
|
classify(text)
|
||||||
|
.into_iter()
|
||||||
|
.map(|s| (text[s.range.clone()].to_string(), s.class, s.chip))
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Segments must tile 0..len exactly: contiguous, ascending, complete.
|
||||||
|
fn assert_tiles(text: &str) {
|
||||||
|
let segs = classify(text);
|
||||||
|
let mut cursor = 0usize;
|
||||||
|
for s in &segs {
|
||||||
|
assert_eq!(s.range.start, cursor, "gap or overlap in {:?}: {:?}", text, segs);
|
||||||
|
assert!(s.range.end > s.range.start, "empty seg in {:?}", text);
|
||||||
|
cursor = s.range.end;
|
||||||
|
}
|
||||||
|
assert_eq!(cursor, text.len(), "segments must cover {:?}", text);
|
||||||
|
}
|
||||||
|
|
||||||
|
use Class::*;
|
||||||
|
|
||||||
|
fn owned(v: Vec<(&str, Class, bool)>) -> Vec<(String, Class, bool)> {
|
||||||
|
v.into_iter().map(|(s, c, b)| (s.to_string(), c, b)).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn empty_input_yields_no_segments() {
|
||||||
|
assert!(classify("").is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn plain_words_stay_plain() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("budget report"),
|
||||||
|
owned(vec![
|
||||||
|
("budget", Plain, false),
|
||||||
|
(" ", Plain, false),
|
||||||
|
("report", Plain, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_recognized_filter_chips() {
|
||||||
|
for input in [
|
||||||
|
"type:Audio",
|
||||||
|
"modified:>=2024-01-01",
|
||||||
|
"mtime:<2023-12-01",
|
||||||
|
"path:/home/me",
|
||||||
|
"folder:/x",
|
||||||
|
"includefolder:/x",
|
||||||
|
"name:report",
|
||||||
|
"filename:report",
|
||||||
|
"mime:application/pdf",
|
||||||
|
"regex:foo",
|
||||||
|
] {
|
||||||
|
let all = segs(input);
|
||||||
|
assert!(
|
||||||
|
all.iter().all(|(_, _, chip)| *chip),
|
||||||
|
"{:?}: whole filter must chip: {:?}",
|
||||||
|
input,
|
||||||
|
all
|
||||||
|
);
|
||||||
|
assert_eq!(all[0].1, Keyword, "{:?}", input);
|
||||||
|
assert_eq!(all[1].1, Operator, "{:?}", input);
|
||||||
|
assert!(
|
||||||
|
all[2..].iter().all(|(_, c, _)| *c == Argument || *c == Operator),
|
||||||
|
"{:?}: {:?}",
|
||||||
|
input,
|
||||||
|
all
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn keys_are_case_insensitive() {
|
||||||
|
assert_eq!(segs("TYPE:Audio")[0], ("TYPE".to_string(), Keyword, true));
|
||||||
|
assert_eq!(
|
||||||
|
segs("Modified:>=2024-01-01")[0],
|
||||||
|
("Modified".to_string(), Keyword, true)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn colon_comparator_runs_are_one_green_stretch() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("modified:>=2024-01-01"),
|
||||||
|
owned(vec![
|
||||||
|
("modified", Keyword, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
(">=", Operator, true),
|
||||||
|
("2024-01-01", Argument, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unrecognized_keys_stay_plain() {
|
||||||
|
for input in ["foo:bar", "12:30", "foo:bar:baz"] {
|
||||||
|
assert!(
|
||||||
|
segs(input).iter().all(|(_, c, chip)| *c == Plain && !chip),
|
||||||
|
"{:?}: {:?}",
|
||||||
|
input,
|
||||||
|
segs(input)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// Stars in glue are literal to the engine — no green.
|
||||||
|
assert!(segs("foo:ba*r").iter().all(|(_, c, _)| *c == Plain));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn drive_letters_do_not_split() {
|
||||||
|
assert_eq!(
|
||||||
|
segs(r"path:C:\Users\me"),
|
||||||
|
owned(vec![
|
||||||
|
("path", Keyword, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
(r"C:\Users\me", Argument, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
segs(r"C:\data"),
|
||||||
|
owned(vec![(r"C:\data", Plain, false)])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_phrases_get_green_delimiters() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("\"exact phrase\""),
|
||||||
|
owned(vec![
|
||||||
|
("\"", Operator, false),
|
||||||
|
("exact phrase", Plain, false),
|
||||||
|
("\"", Operator, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
// Inner "" escapes are content bytes.
|
||||||
|
assert_eq!(
|
||||||
|
segs("\"a\"\"b\""),
|
||||||
|
owned(vec![
|
||||||
|
("\"", Operator, false),
|
||||||
|
("a\"\"b", Plain, false),
|
||||||
|
("\"", Operator, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
// Quoted stars are literal — content stays plain.
|
||||||
|
assert!(segs("\"a*b\"").iter().all(|(s, c, _)| s == "\""
|
||||||
|
|| *c == Plain));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quoted_filter_values_are_blue_with_green_quotes() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("path:\"/home/me/My Docs\""),
|
||||||
|
owned(vec![
|
||||||
|
("path", Keyword, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
("\"", Operator, true),
|
||||||
|
("/home/me/My Docs", Argument, true),
|
||||||
|
("\"", Operator, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
// Empty quoted value: two delimiters, no content seg, no panic.
|
||||||
|
assert_eq!(
|
||||||
|
segs("path:\"\""),
|
||||||
|
owned(vec![
|
||||||
|
("path", Keyword, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
("\"", Operator, true),
|
||||||
|
("\"", Operator, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unterminated_quote_is_a_quote_in_progress() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("\"unclosed phrase"),
|
||||||
|
owned(vec![
|
||||||
|
("\"", Operator, false),
|
||||||
|
("unclosed phrase", Plain, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
// Filters before the open quote keep their colors.
|
||||||
|
let all = segs("type:Audio \"x");
|
||||||
|
assert_eq!(all[0], ("type".to_string(), Keyword, true));
|
||||||
|
assert_eq!(all[4], ("\"".to_string(), Operator, false));
|
||||||
|
assert_eq!(all[5], ("x".to_string(), Plain, false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn trailing_bare_keys_color_optimistically_without_chip() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("type:"),
|
||||||
|
owned(vec![("type", Keyword, false), (":", Operator, false)])
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
segs("modified:>="),
|
||||||
|
owned(vec![
|
||||||
|
("modified", Keyword, false),
|
||||||
|
(":", Operator, false),
|
||||||
|
(">=", Operator, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
segs("foo:"),
|
||||||
|
owned(vec![("foo", Plain, false), (":", Plain, false)])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn stars_in_words_and_name_values_go_green() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("rep*ort"),
|
||||||
|
owned(vec![
|
||||||
|
("rep", Plain, false),
|
||||||
|
("*", Operator, false),
|
||||||
|
("ort", Plain, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
segs("name:re*.txt"),
|
||||||
|
owned(vec![
|
||||||
|
("name", Keyword, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
("re", Argument, true),
|
||||||
|
("*", Operator, true),
|
||||||
|
(".txt", Argument, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
// Edge and doubled stars keep tiling intact.
|
||||||
|
assert_tiles("*foo");
|
||||||
|
assert_tiles("foo*");
|
||||||
|
assert_tiles("**");
|
||||||
|
assert_tiles("*");
|
||||||
|
// In non-glob filter values the star is a literal character.
|
||||||
|
assert_eq!(
|
||||||
|
segs("path:/da*ta")[2],
|
||||||
|
("/da*ta".to_string(), Argument, true)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn invalid_arguments_go_error_uniformly() {
|
||||||
|
// (`regex:(` is not here: `(` lexes as a paren, so that input is an
|
||||||
|
// *incomplete* filter — bare-key optimism applies, not an error.)
|
||||||
|
for input in ["type:NotAThing", "modified:>=tomorrow", "regex:[", "type:Doc*"] {
|
||||||
|
let all = segs(input);
|
||||||
|
assert_eq!(all[0].1, Keyword, "{:?}", input);
|
||||||
|
let last = all.last().unwrap();
|
||||||
|
assert_eq!(last.1, InvalidArg, "{:?}: {:?}", input, all);
|
||||||
|
assert!(last.2, "invalid values keep the chip: {:?}", input);
|
||||||
|
}
|
||||||
|
// name:= is an unsupported op → its value is invalid too.
|
||||||
|
let all = segs("name=x");
|
||||||
|
assert_eq!(all.last().unwrap().1, InvalidArg);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn valid_regex_argument_is_blue_with_green_stars() {
|
||||||
|
assert_eq!(
|
||||||
|
segs("regex:foo.*bar"),
|
||||||
|
owned(vec![
|
||||||
|
("regex", Keyword, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
("foo.", Argument, true),
|
||||||
|
("*", Operator, true),
|
||||||
|
("bar", Argument, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_second_regex_filter_is_invalid() {
|
||||||
|
let all = segs("regex:foo regex:bar");
|
||||||
|
assert_eq!(all[2], ("foo".to_string(), Argument, true));
|
||||||
|
assert_eq!(all.last().unwrap(), &("bar".to_string(), InvalidArg, true));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn multi_filter_queries_chip_separately() {
|
||||||
|
let all = segs("type:Document budget modified:>=2024-01-01");
|
||||||
|
// The word and the whitespace around it stay un-chipped.
|
||||||
|
assert_eq!(
|
||||||
|
all.iter()
|
||||||
|
.filter(|(_, _, chip)| !chip)
|
||||||
|
.map(|(s, _, _)| s.as_str())
|
||||||
|
.collect::<Vec<_>>(),
|
||||||
|
vec![" ", "budget", " "]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn spaced_filters_chip_their_interior_gaps() {
|
||||||
|
// `type : Audio` is still a filter to the lexer/splitter.
|
||||||
|
assert_eq!(
|
||||||
|
segs("type : Audio"),
|
||||||
|
owned(vec![
|
||||||
|
("type", Keyword, true),
|
||||||
|
(" ", Plain, true),
|
||||||
|
(":", Operator, true),
|
||||||
|
(" ", Plain, true),
|
||||||
|
("Audio", Argument, true),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn demoted_operators_stay_plain() {
|
||||||
|
assert!(
|
||||||
|
segs("(alpha AND beta) OR gamma")
|
||||||
|
.iter()
|
||||||
|
.all(|(_, c, chip)| *c == Plain && !chip)
|
||||||
|
);
|
||||||
|
// Dangling comparators are literal text.
|
||||||
|
assert!(segs("a > b").iter().all(|(_, c, _)| *c == Plain));
|
||||||
|
// Leading operator, nothing else.
|
||||||
|
assert!(segs(">foo").iter().all(|(_, c, _)| *c == Plain));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn adjacency_between_filter_and_quote() {
|
||||||
|
// `Audio"q"`: the word ends at the quote; the filter is complete
|
||||||
|
// and the quoted phrase stands alone.
|
||||||
|
let all = segs("type:Audio\"q\"");
|
||||||
|
assert_eq!(all[2], ("Audio".to_string(), Argument, true));
|
||||||
|
assert_eq!(all[3], ("\"".to_string(), Operator, false));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unicode_offsets_hold_up() {
|
||||||
|
assert_tiles("\"José\" type:Audio naïve*file");
|
||||||
|
let all = segs("naïve*café");
|
||||||
|
assert_eq!(
|
||||||
|
all,
|
||||||
|
owned(vec![
|
||||||
|
("naïve", Plain, false),
|
||||||
|
("*", Operator, false),
|
||||||
|
("café", Plain, false),
|
||||||
|
])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,527 +0,0 @@
|
||||||
#![allow(non_snake_case)]
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
|
||||||
use std::time::Instant;
|
|
||||||
use dioxus::prelude::*;
|
|
||||||
use quicksearch_core::indexing::{IndexingService, SearchResult};
|
|
||||||
use quicksearch_core::search_sql::{build_count, build_select, SearchArgs};
|
|
||||||
|
|
||||||
/// One page of results. Tuned to keep DOM size bounded — rendering ten
|
|
||||||
/// thousand `<tr>` nodes wedges WebKit for tens of seconds.
|
|
||||||
const PAGE_SIZE: u32 = 50;
|
|
||||||
|
|
||||||
#[derive(Props, Clone)]
|
|
||||||
pub struct SearchProps {
|
|
||||||
pub indexing_service: Arc<IndexingService>,
|
|
||||||
pub db_path: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl PartialEq for SearchProps {
|
|
||||||
fn eq(&self, other: &Self) -> bool {
|
|
||||||
Arc::ptr_eq(&self.indexing_service, &other.indexing_service) && self.db_path == other.db_path
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn Search(props: SearchProps) -> Element {
|
|
||||||
let mut search_type = use_signal(|| "fulltext".to_string());
|
|
||||||
let mut search_term = use_signal(|| String::new());
|
|
||||||
let mut fulltext_exact = use_signal(|| false);
|
|
||||||
let mut fulltext_case_sensitive = use_signal(|| false);
|
|
||||||
let mut search_results = use_signal(|| Vec::<SearchResult>::new());
|
|
||||||
let mut search_error = use_signal(|| None::<String>);
|
|
||||||
let mut is_searching = use_signal(|| false);
|
|
||||||
let mut last_search_time = use_signal(|| None::<f64>);
|
|
||||||
let mut show_corruption_dialog = use_signal(|| false);
|
|
||||||
let mut current_page = use_signal(|| 1u32);
|
|
||||||
let mut total_count = use_signal(|| None::<u64>);
|
|
||||||
let mut last_args = use_signal(|| None::<SearchArgs>);
|
|
||||||
let mut goto_input = use_signal(|| String::new());
|
|
||||||
|
|
||||||
let service = props.indexing_service.clone();
|
|
||||||
let db_path = props.db_path.clone();
|
|
||||||
|
|
||||||
// Spawn a search task. `refresh_count` is true for fresh searches and
|
|
||||||
// false for in-place page navigation (the cached total still applies).
|
|
||||||
let run_query = {
|
|
||||||
let service = service.clone();
|
|
||||||
let db_path = db_path.clone();
|
|
||||||
move |args: SearchArgs, page: u32, refresh_count: bool| {
|
|
||||||
let service = service.clone();
|
|
||||||
let db_path = db_path.clone();
|
|
||||||
spawn(async move {
|
|
||||||
is_searching.set(true);
|
|
||||||
search_error.set(None);
|
|
||||||
last_search_time.set(None);
|
|
||||||
let start = Instant::now();
|
|
||||||
|
|
||||||
let count_sql = if refresh_count {
|
|
||||||
match build_count(&args) {
|
|
||||||
Ok(s) => Some(s),
|
|
||||||
Err(e) => {
|
|
||||||
search_error.set(Some(e));
|
|
||||||
is_searching.set(false);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
None
|
|
||||||
};
|
|
||||||
|
|
||||||
let offset = page.saturating_sub(1).saturating_mul(PAGE_SIZE);
|
|
||||||
// Validate early for the filename/duplicates branch so we
|
|
||||||
// surface parse errors before dispatching the blocking task.
|
|
||||||
let precomputed_select_sql = if args.search_type == "fulltext" {
|
|
||||||
None
|
|
||||||
} else {
|
|
||||||
match build_select(&args, PAGE_SIZE, offset) {
|
|
||||||
Ok(s) => Some(s),
|
|
||||||
Err(e) => {
|
|
||||||
search_error.set(Some(e));
|
|
||||||
is_searching.set(false);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Fulltext takes the snippet-aware path (decompresses
|
|
||||||
// documents_text and highlights in Rust); filename +
|
|
||||||
// duplicates go through the plain SQL executor.
|
|
||||||
let svc1 = service.clone();
|
|
||||||
let db1 = db_path.clone();
|
|
||||||
let args_for_select = args.clone();
|
|
||||||
let select_handle = tokio::task::spawn_blocking(move || {
|
|
||||||
if args_for_select.search_type == "fulltext" {
|
|
||||||
svc1.execute_fulltext_search(&db1, &args_for_select, PAGE_SIZE, offset)
|
|
||||||
} else {
|
|
||||||
let sql = precomputed_select_sql
|
|
||||||
.expect("non-fulltext select SQL was prebuilt above");
|
|
||||||
svc1.execute_search(&db1, &sql)
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
let count_handle = count_sql.map(|sql| {
|
|
||||||
let svc2 = service.clone();
|
|
||||||
let db2 = db_path.clone();
|
|
||||||
tokio::task::spawn_blocking(move || svc2.execute_search(&db2, &sql))
|
|
||||||
});
|
|
||||||
|
|
||||||
let select_run = select_handle.await;
|
|
||||||
let count_run = match count_handle {
|
|
||||||
Some(h) => Some(h.await),
|
|
||||||
None => None,
|
|
||||||
};
|
|
||||||
|
|
||||||
let elapsed = start.elapsed().as_secs_f64();
|
|
||||||
|
|
||||||
if let Some(c) = count_run {
|
|
||||||
match c {
|
|
||||||
Ok(Ok(rs)) => {
|
|
||||||
let n = rs
|
|
||||||
.first()
|
|
||||||
.and_then(|r| r.rows.first())
|
|
||||||
.and_then(|r| r.values.first())
|
|
||||||
.and_then(|s| s.parse::<u64>().ok())
|
|
||||||
.unwrap_or(0);
|
|
||||||
total_count.set(Some(n));
|
|
||||||
}
|
|
||||||
Ok(Err(e)) => {
|
|
||||||
handle_query_error(
|
|
||||||
e,
|
|
||||||
elapsed,
|
|
||||||
search_error,
|
|
||||||
show_corruption_dialog,
|
|
||||||
last_search_time,
|
|
||||||
is_searching,
|
|
||||||
);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
search_error.set(Some(format!("Task execution error: {}", e)));
|
|
||||||
last_search_time.set(Some(elapsed));
|
|
||||||
is_searching.set(false);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
match select_run {
|
|
||||||
Ok(Ok(rs)) => {
|
|
||||||
search_results.set(rs);
|
|
||||||
current_page.set(page);
|
|
||||||
last_args.set(Some(args));
|
|
||||||
last_search_time.set(Some(elapsed));
|
|
||||||
}
|
|
||||||
Ok(Err(e)) => {
|
|
||||||
handle_query_error(
|
|
||||||
e,
|
|
||||||
elapsed,
|
|
||||||
search_error,
|
|
||||||
show_corruption_dialog,
|
|
||||||
last_search_time,
|
|
||||||
is_searching,
|
|
||||||
);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
search_error.set(Some(format!("Task execution error: {}", e)));
|
|
||||||
last_search_time.set(Some(elapsed));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
is_searching.set(false);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
let do_fresh_search = {
|
|
||||||
let run_query = run_query.clone();
|
|
||||||
move || {
|
|
||||||
let args = SearchArgs {
|
|
||||||
search_type: search_type(),
|
|
||||||
term: search_term(),
|
|
||||||
fulltext_exact: fulltext_exact(),
|
|
||||||
fulltext_case_sensitive: fulltext_case_sensitive(),
|
|
||||||
};
|
|
||||||
run_query(args, 1, true);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
let do_goto_page = {
|
|
||||||
let run_query = run_query.clone();
|
|
||||||
move |target: u32| {
|
|
||||||
if let Some(args) = last_args() {
|
|
||||||
let total = total_count()
|
|
||||||
.map(|n| pages_for(n))
|
|
||||||
.unwrap_or(1)
|
|
||||||
.max(1);
|
|
||||||
let clamped = target.clamp(1, total);
|
|
||||||
if clamped != current_page() {
|
|
||||||
run_query(args, clamped, false);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
let total_pages = total_count().map(pages_for).unwrap_or(0);
|
|
||||||
|
|
||||||
rsx! {
|
|
||||||
div {
|
|
||||||
class: "section",
|
|
||||||
h2 { "Search Database" }
|
|
||||||
|
|
||||||
div {
|
|
||||||
class: "form-group",
|
|
||||||
label { "Search Type: " }
|
|
||||||
select {
|
|
||||||
class: "form-control",
|
|
||||||
value: "{search_type}",
|
|
||||||
onchange: move |evt| search_type.set(evt.value()),
|
|
||||||
option { value: "fulltext", "Full Text Search" }
|
|
||||||
option { value: "filename", "Filename Search" }
|
|
||||||
option { value: "duplicates", "Find Duplicate Files" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if search_type() == "fulltext" {
|
|
||||||
div {
|
|
||||||
class: "form-group",
|
|
||||||
style: "display: flex; flex-direction: column; gap: 6px;",
|
|
||||||
span { style: "font-weight: 600;", "Full text options" }
|
|
||||||
label {
|
|
||||||
style: "display: flex; align-items: center; gap: 8px; cursor: pointer;",
|
|
||||||
input {
|
|
||||||
r#type: "checkbox",
|
|
||||||
checked: fulltext_exact(),
|
|
||||||
onchange: move |evt| fulltext_exact.set(evt.checked()),
|
|
||||||
}
|
|
||||||
"Exact phrase match"
|
|
||||||
}
|
|
||||||
label {
|
|
||||||
style: "display: flex; align-items: center; gap: 8px; cursor: pointer;",
|
|
||||||
input {
|
|
||||||
r#type: "checkbox",
|
|
||||||
checked: fulltext_case_sensitive(),
|
|
||||||
onchange: move |evt| fulltext_case_sensitive.set(evt.checked()),
|
|
||||||
}
|
|
||||||
"Case-sensitive match"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if search_type() != "duplicates" {
|
|
||||||
div {
|
|
||||||
class: "form-group",
|
|
||||||
label { "Search Term: " }
|
|
||||||
input {
|
|
||||||
class: "form-control",
|
|
||||||
r#type: "text",
|
|
||||||
value: "{search_term}",
|
|
||||||
oninput: move |evt| search_term.set(evt.value()),
|
|
||||||
onkeydown: {
|
|
||||||
let do_fresh_search = do_fresh_search.clone();
|
|
||||||
move |evt| {
|
|
||||||
if evt.code() == dioxus::events::Code::Enter {
|
|
||||||
do_fresh_search();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
div {
|
|
||||||
style: "display: flex; align-items: center; gap: 10px;",
|
|
||||||
button {
|
|
||||||
class: "btn btn-info",
|
|
||||||
disabled: is_searching(),
|
|
||||||
onclick: {
|
|
||||||
let do_fresh_search = do_fresh_search.clone();
|
|
||||||
move |_| { do_fresh_search(); }
|
|
||||||
},
|
|
||||||
"Search"
|
|
||||||
}
|
|
||||||
|
|
||||||
if is_searching() {
|
|
||||||
div {
|
|
||||||
class: "loading",
|
|
||||||
title: "Searching..."
|
|
||||||
}
|
|
||||||
} else if let Some(elapsed) = last_search_time() {
|
|
||||||
span {
|
|
||||||
style: "color: #666; font-size: 0.9em;",
|
|
||||||
"Search completed in {elapsed:.3}s"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(error) = search_error() {
|
|
||||||
div {
|
|
||||||
class: "error-message",
|
|
||||||
"Error: {error}"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Results panel: pagination header + bounded table. Only renders
|
|
||||||
// when at least one search has completed (last_args is Some).
|
|
||||||
if last_args().is_some() {
|
|
||||||
div {
|
|
||||||
class: "search-results",
|
|
||||||
{
|
|
||||||
let total_str = match total_count() {
|
|
||||||
Some(n) => format!("{}", n),
|
|
||||||
None => "?".to_string(),
|
|
||||||
};
|
|
||||||
let page_first = (current_page().saturating_sub(1) as u64) * PAGE_SIZE as u64 + 1;
|
|
||||||
let page_last_calc = page_first + search_results().first().map(|r| r.rows.len() as u64).unwrap_or(0).saturating_sub(1);
|
|
||||||
let header = if total_count() == Some(0) {
|
|
||||||
"No results.".to_string()
|
|
||||||
} else {
|
|
||||||
format!(
|
|
||||||
"Showing {}-{} of {} (page {} of {})",
|
|
||||||
page_first,
|
|
||||||
page_last_calc,
|
|
||||||
total_str,
|
|
||||||
current_page(),
|
|
||||||
total_pages
|
|
||||||
)
|
|
||||||
};
|
|
||||||
rsx! { h3 { "{header}" } }
|
|
||||||
}
|
|
||||||
|
|
||||||
// Pagination controls. Hidden if there's only one page.
|
|
||||||
if total_pages > 1 {
|
|
||||||
div {
|
|
||||||
style: "display: flex; align-items: center; gap: 8px; margin: 8px 0;",
|
|
||||||
button {
|
|
||||||
class: "btn",
|
|
||||||
disabled: is_searching() || current_page() <= 1,
|
|
||||||
onclick: {
|
|
||||||
let do_goto_page = do_goto_page.clone();
|
|
||||||
move |_| do_goto_page(1)
|
|
||||||
},
|
|
||||||
"« First"
|
|
||||||
}
|
|
||||||
button {
|
|
||||||
class: "btn",
|
|
||||||
disabled: is_searching() || current_page() <= 1,
|
|
||||||
onclick: {
|
|
||||||
let do_goto_page = do_goto_page.clone();
|
|
||||||
move |_| do_goto_page(current_page().saturating_sub(1))
|
|
||||||
},
|
|
||||||
"‹ Prev"
|
|
||||||
}
|
|
||||||
button {
|
|
||||||
class: "btn",
|
|
||||||
disabled: is_searching() || current_page() >= total_pages,
|
|
||||||
onclick: {
|
|
||||||
let do_goto_page = do_goto_page.clone();
|
|
||||||
move |_| do_goto_page(current_page().saturating_add(1))
|
|
||||||
},
|
|
||||||
"Next ›"
|
|
||||||
}
|
|
||||||
button {
|
|
||||||
class: "btn",
|
|
||||||
disabled: is_searching() || current_page() >= total_pages,
|
|
||||||
onclick: {
|
|
||||||
let do_goto_page = do_goto_page.clone();
|
|
||||||
move |_| do_goto_page(total_pages)
|
|
||||||
},
|
|
||||||
"Last »"
|
|
||||||
}
|
|
||||||
span { "Go to:" }
|
|
||||||
input {
|
|
||||||
r#type: "number",
|
|
||||||
style: "width: 70px;",
|
|
||||||
value: "{goto_input}",
|
|
||||||
oninput: move |evt| goto_input.set(evt.value()),
|
|
||||||
onkeydown: {
|
|
||||||
let do_goto_page = do_goto_page.clone();
|
|
||||||
move |evt| {
|
|
||||||
if evt.code() == dioxus::events::Code::Enter {
|
|
||||||
if let Ok(p) = goto_input().trim().parse::<u32>() {
|
|
||||||
do_goto_page(p);
|
|
||||||
goto_input.set(String::new());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !search_results().is_empty() && !search_results()[0].rows.is_empty() {
|
|
||||||
div {
|
|
||||||
class: "results-table",
|
|
||||||
table {
|
|
||||||
thead {
|
|
||||||
tr {
|
|
||||||
for column in search_results()[0].columns.iter() {
|
|
||||||
th { "{column}" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
tbody {
|
|
||||||
for row in search_results()[0].rows.iter() {
|
|
||||||
tr {
|
|
||||||
for (col_index, value) in row.values.iter().enumerate() {
|
|
||||||
if search_results()[0].columns.get(col_index).map(|s| s.as_str()) == Some("path") {
|
|
||||||
{
|
|
||||||
let value_owned = value.clone();
|
|
||||||
let service_owned = props.indexing_service.clone();
|
|
||||||
rsx! {
|
|
||||||
td {
|
|
||||||
class: "path-cell clickable",
|
|
||||||
onclick: move |_| {
|
|
||||||
let path = value_owned.clone();
|
|
||||||
let service_clone = service_owned.clone();
|
|
||||||
spawn(async move {
|
|
||||||
if let Err(e) = service_clone.open_file_explorer(&path) {
|
|
||||||
eprintln!("Failed to open file explorer: {}", e);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
},
|
|
||||||
title: "Click to open in file explorer",
|
|
||||||
dangerous_inner_html: "{value}"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
td {
|
|
||||||
dangerous_inner_html: "{value}"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if show_corruption_dialog() {
|
|
||||||
div {
|
|
||||||
class: "modal-backdrop",
|
|
||||||
div {
|
|
||||||
class: "modal-dialog",
|
|
||||||
h3 {
|
|
||||||
style: "margin-top: 0; color: #d32f2f;",
|
|
||||||
"⚠️ Database Corruption Detected"
|
|
||||||
}
|
|
||||||
p {
|
|
||||||
style: "margin: 15px 0;",
|
|
||||||
"The database appears to be corrupted or malformed. This can happen due to unexpected shutdowns or disk issues."
|
|
||||||
}
|
|
||||||
p {
|
|
||||||
style: "margin: 15px 0; font-weight: bold;",
|
|
||||||
"Would you like to delete the corrupted database and create a new one? This will require re-indexing your files."
|
|
||||||
}
|
|
||||||
div {
|
|
||||||
style: "display: flex; gap: 10px; margin-top: 20px;",
|
|
||||||
button {
|
|
||||||
style: "padding: 10px 20px; background-color: #d32f2f; color: white; border: none; border-radius: 5px; cursor: pointer;",
|
|
||||||
onclick: move |_| {
|
|
||||||
let service = props.indexing_service.clone();
|
|
||||||
let db = props.db_path.clone();
|
|
||||||
|
|
||||||
show_corruption_dialog.set(false);
|
|
||||||
search_error.set(Some("Deleting corrupted database...".to_string()));
|
|
||||||
|
|
||||||
spawn(async move {
|
|
||||||
match service.delete_index_for_rebuild(&db) {
|
|
||||||
Ok(()) => {
|
|
||||||
search_error.set(Some("Database deleted. You can now start indexing again.".to_string()));
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
search_error.set(Some(format!("Error deleting database: {}", e)));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
},
|
|
||||||
"Yes, Delete & Rebuild"
|
|
||||||
}
|
|
||||||
button {
|
|
||||||
style: "padding: 10px 20px; background-color: #666; color: white; border: none; border-radius: 5px; cursor: pointer;",
|
|
||||||
onclick: move |_| {
|
|
||||||
show_corruption_dialog.set(false);
|
|
||||||
},
|
|
||||||
"Cancel"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Number of pages required to fit `total` rows at [`PAGE_SIZE`] per page.
|
|
||||||
/// Returns 0 for 0 rows so callers can branch on "no results yet".
|
|
||||||
fn pages_for(total: u64) -> u32 {
|
|
||||||
if total == 0 {
|
|
||||||
0
|
|
||||||
} else {
|
|
||||||
((total - 1) / PAGE_SIZE as u64) as u32 + 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Centralized error-router for the two query branches that can fail
|
|
||||||
/// identically (count and select). Sets the error/timing/spinner signals
|
|
||||||
/// and pops the corruption dialog when warranted.
|
|
||||||
fn handle_query_error(
|
|
||||||
e: String,
|
|
||||||
elapsed: f64,
|
|
||||||
mut search_error: Signal<Option<String>>,
|
|
||||||
mut show_corruption_dialog: Signal<bool>,
|
|
||||||
mut last_search_time: Signal<Option<f64>>,
|
|
||||||
mut is_searching: Signal<bool>,
|
|
||||||
) {
|
|
||||||
if e.starts_with("DATABASE_CORRUPTED:") {
|
|
||||||
search_error.set(Some("Database appears to be corrupted".into()));
|
|
||||||
show_corruption_dialog.set(true);
|
|
||||||
} else {
|
|
||||||
search_error.set(Some(e));
|
|
||||||
}
|
|
||||||
last_search_time.set(Some(elapsed));
|
|
||||||
is_searching.set(false);
|
|
||||||
}
|
|
||||||
947
crates/quicksearch-gui/src/search_tab.rs
Normal file
|
|
@ -0,0 +1,947 @@
|
||||||
|
//! The Search tab: query strip, streaming results table, snippet
|
||||||
|
//! preview, context menu, ignore-filter dialog, and syntax help.
|
||||||
|
|
||||||
|
use std::time::Instant;
|
||||||
|
|
||||||
|
use egui::text::{LayoutJob, TextFormat};
|
||||||
|
use egui_extras::{Column, TableBuilder};
|
||||||
|
use quicksearch_core::search::{SearchHit, SearchUpdate};
|
||||||
|
use quicksearch_core::snippet::Snippet;
|
||||||
|
|
||||||
|
use crate::format::{fmt_elapsed, fmt_mtime, human_size};
|
||||||
|
use crate::platform;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum SortKey {
|
||||||
|
Rank,
|
||||||
|
Name,
|
||||||
|
Path,
|
||||||
|
Size,
|
||||||
|
Modified,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct IgnoreDialog {
|
||||||
|
pub source_name: String,
|
||||||
|
pub source_path: String,
|
||||||
|
pub pattern: String,
|
||||||
|
pub persist: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the tab asks the app to do after this frame.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct SearchActions {
|
||||||
|
/// Re-run the search (query/fuzzy/session filters changed *now*, not
|
||||||
|
/// debounced — e.g. a chip was removed).
|
||||||
|
pub rerun: bool,
|
||||||
|
/// Persist an ignore pattern into the config.
|
||||||
|
pub persist_ignore: Option<String>,
|
||||||
|
/// The fuzzy toggle changed; remember it in the config.
|
||||||
|
pub save_fuzzy_default: Option<bool>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct SearchTab {
|
||||||
|
pub query: String,
|
||||||
|
pub fuzzy: bool,
|
||||||
|
/// Set on every edit; the app fires the search after the debounce.
|
||||||
|
pub pending_edit: Option<Instant>,
|
||||||
|
pub generation: u64,
|
||||||
|
pub results: Vec<SearchHit>,
|
||||||
|
/// The next search's hits, held back while the old table fades out;
|
||||||
|
/// swapped into `results` at zero opacity. Prevents the empty-refill
|
||||||
|
/// strobe while typing.
|
||||||
|
staging: Vec<SearchHit>,
|
||||||
|
staging_has_snippets: bool,
|
||||||
|
/// True from search start until the staged set has been swapped in.
|
||||||
|
swap_pending: bool,
|
||||||
|
/// Display permutation over `results`.
|
||||||
|
order: Vec<u32>,
|
||||||
|
sort: (SortKey, bool),
|
||||||
|
sort_dirty: bool,
|
||||||
|
pub selected: Option<u32>,
|
||||||
|
pub running: bool,
|
||||||
|
/// When the in-flight search was submitted.
|
||||||
|
search_started: Option<Instant>,
|
||||||
|
/// Wall time of the last completed search (all cascade passes).
|
||||||
|
elapsed: Option<std::time::Duration>,
|
||||||
|
pub limited: bool,
|
||||||
|
pub error: Option<String>,
|
||||||
|
pub session_ignores: Vec<String>,
|
||||||
|
pub ignore_dialog: Option<IgnoreDialog>,
|
||||||
|
pub help_open: bool,
|
||||||
|
has_snippets: bool,
|
||||||
|
focus_query: bool,
|
||||||
|
/// Query syntax-highlight segments, cached per text.
|
||||||
|
highlight: crate::query_highlight::HighlightCache,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SearchTab {
|
||||||
|
pub fn new(fuzzy_default: bool) -> SearchTab {
|
||||||
|
SearchTab {
|
||||||
|
query: String::new(),
|
||||||
|
fuzzy: fuzzy_default,
|
||||||
|
pending_edit: None,
|
||||||
|
generation: 0,
|
||||||
|
results: Vec::new(),
|
||||||
|
staging: Vec::new(),
|
||||||
|
staging_has_snippets: false,
|
||||||
|
swap_pending: false,
|
||||||
|
order: Vec::new(),
|
||||||
|
sort: (SortKey::Rank, true),
|
||||||
|
sort_dirty: false,
|
||||||
|
selected: None,
|
||||||
|
running: false,
|
||||||
|
search_started: None,
|
||||||
|
elapsed: None,
|
||||||
|
limited: false,
|
||||||
|
error: None,
|
||||||
|
session_ignores: Vec::new(),
|
||||||
|
ignore_dialog: None,
|
||||||
|
help_open: false,
|
||||||
|
has_snippets: false,
|
||||||
|
focus_query: true,
|
||||||
|
highlight: Default::default(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pre-fill the query and let the normal debounce path run it, so a
|
||||||
|
/// command-line query lands the user on results rather than an empty box.
|
||||||
|
pub fn seed(&mut self, query: String) {
|
||||||
|
self.query = query;
|
||||||
|
self.pending_edit = Some(Instant::now());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A new search was submitted under `generation`. The previous
|
||||||
|
/// results stay on screen (fading out); the new ones stage until the
|
||||||
|
/// fade reaches zero.
|
||||||
|
pub fn on_search_started(&mut self, generation: u64) {
|
||||||
|
self.generation = generation;
|
||||||
|
self.staging.clear();
|
||||||
|
self.staging_has_snippets = false;
|
||||||
|
self.swap_pending = true;
|
||||||
|
self.running = true;
|
||||||
|
self.search_started = Some(Instant::now());
|
||||||
|
self.elapsed = None;
|
||||||
|
self.limited = false;
|
||||||
|
self.error = None;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn apply_update(&mut self, update: SearchUpdate, display_limit: usize) {
|
||||||
|
if update.generation() != self.generation {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
match update {
|
||||||
|
SearchUpdate::Started { .. } => {}
|
||||||
|
SearchUpdate::Hits { hits, .. } => {
|
||||||
|
if self.swap_pending {
|
||||||
|
// Old results are still fading out; hold the new ones.
|
||||||
|
for hit in hits {
|
||||||
|
if self.staging.len() >= display_limit {
|
||||||
|
self.limited = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
self.staging_has_snippets |= hit.snippet.is_some();
|
||||||
|
self.staging.push(hit);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Post-swap stream: later cascade passes append live.
|
||||||
|
for hit in hits {
|
||||||
|
if self.results.len() >= display_limit {
|
||||||
|
self.limited = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
self.has_snippets |= hit.snippet.is_some();
|
||||||
|
self.results.push(hit);
|
||||||
|
}
|
||||||
|
// Arrival order *is* rank order, so the default sort
|
||||||
|
// needs no work; anything else re-sorts on the set.
|
||||||
|
if self.sort != (SortKey::Rank, true) {
|
||||||
|
self.sort_dirty = true;
|
||||||
|
} else {
|
||||||
|
self.order = (0..self.results.len() as u32).collect();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
SearchUpdate::Completed { limited, .. } => {
|
||||||
|
self.running = false;
|
||||||
|
self.elapsed = self.search_started.map(|t| t.elapsed());
|
||||||
|
self.limited |= limited;
|
||||||
|
}
|
||||||
|
SearchUpdate::Error { message, .. } => {
|
||||||
|
self.running = false;
|
||||||
|
self.elapsed = self.search_started.map(|t| t.elapsed());
|
||||||
|
self.error = Some(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn result_count_label(&self) -> Option<String> {
|
||||||
|
if self.query.trim().is_empty() && self.results.is_empty() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some(if self.limited {
|
||||||
|
format!("{}+ results (truncated)", self.results.len())
|
||||||
|
} else {
|
||||||
|
format!("{} results", self.results.len())
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resort(&mut self) {
|
||||||
|
let (key, ascending) = self.sort;
|
||||||
|
let selected_id = self
|
||||||
|
.selected
|
||||||
|
.and_then(|i| self.results.get(i as usize))
|
||||||
|
.map(|h| h.file_id);
|
||||||
|
self.order = (0..self.results.len() as u32).collect();
|
||||||
|
let results = &self.results;
|
||||||
|
self.order.sort_by(|&a, &b| {
|
||||||
|
let (a, b) = (&results[a as usize], &results[b as usize]);
|
||||||
|
let ord = match key {
|
||||||
|
SortKey::Rank => a.rank.partial_cmp(&b.rank).unwrap_or(std::cmp::Ordering::Equal),
|
||||||
|
SortKey::Name => a.name.cmp(&b.name),
|
||||||
|
SortKey::Path => a.path.cmp(&b.path),
|
||||||
|
SortKey::Size => a.size.cmp(&b.size),
|
||||||
|
SortKey::Modified => a.mtime.cmp(&b.mtime),
|
||||||
|
};
|
||||||
|
if ascending {
|
||||||
|
ord
|
||||||
|
} else {
|
||||||
|
ord.reverse()
|
||||||
|
}
|
||||||
|
});
|
||||||
|
// Selection follows the file, not the visual slot.
|
||||||
|
self.selected = selected_id.and_then(|id| {
|
||||||
|
self.results
|
||||||
|
.iter()
|
||||||
|
.position(|h| h.file_id == id)
|
||||||
|
.map(|i| i as u32)
|
||||||
|
});
|
||||||
|
self.sort_dirty = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A sortable column header: the whole cell is the click target, the
|
||||||
|
/// label is centered, and the sort indicator is a painter-drawn
|
||||||
|
/// triangle (the default egui fonts have no ▲/▼ glyphs — they render
|
||||||
|
/// as boxes).
|
||||||
|
fn sort_header(&mut self, ui: &mut egui::Ui, key: SortKey, label: &str) {
|
||||||
|
let (cur, asc) = self.sort;
|
||||||
|
let selected = cur == key;
|
||||||
|
let (rect, response) =
|
||||||
|
ui.allocate_exact_size(ui.available_size(), egui::Sense::click());
|
||||||
|
if ui.is_rect_visible(rect) {
|
||||||
|
if response.hovered() {
|
||||||
|
ui.painter()
|
||||||
|
.rect_filled(rect, 2.0, ui.visuals().widgets.hovered.weak_bg_fill);
|
||||||
|
}
|
||||||
|
let font_id = egui::TextStyle::Body.resolve(ui.style());
|
||||||
|
let color = ui.visuals().strong_text_color();
|
||||||
|
let galley = ui
|
||||||
|
.painter()
|
||||||
|
.layout_no_wrap(label.to_string(), font_id, color);
|
||||||
|
let text_size = galley.size();
|
||||||
|
let arrow_space = if selected { 11.0 } else { 0.0 };
|
||||||
|
let text_pos = egui::pos2(
|
||||||
|
rect.center().x - (text_size.x + arrow_space) / 2.0,
|
||||||
|
rect.center().y - text_size.y / 2.0,
|
||||||
|
);
|
||||||
|
ui.painter().galley(text_pos, galley, color);
|
||||||
|
if selected {
|
||||||
|
let cx = text_pos.x + text_size.x + 7.0;
|
||||||
|
let cy = rect.center().y;
|
||||||
|
let (w, h) = (3.5, 3.0);
|
||||||
|
let points = if asc {
|
||||||
|
vec![
|
||||||
|
egui::pos2(cx, cy - h),
|
||||||
|
egui::pos2(cx - w, cy + h),
|
||||||
|
egui::pos2(cx + w, cy + h),
|
||||||
|
]
|
||||||
|
} else {
|
||||||
|
vec![
|
||||||
|
egui::pos2(cx, cy + h),
|
||||||
|
egui::pos2(cx - w, cy - h),
|
||||||
|
egui::pos2(cx + w, cy - h),
|
||||||
|
]
|
||||||
|
};
|
||||||
|
ui.painter().add(egui::Shape::convex_polygon(
|
||||||
|
points,
|
||||||
|
color,
|
||||||
|
egui::Stroke::NONE,
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if response.clicked() {
|
||||||
|
self.sort = if selected { (key, !asc) } else { (key, true) };
|
||||||
|
self.sort_dirty = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn ui(&mut self, ui: &mut egui::Ui) -> SearchActions {
|
||||||
|
let mut actions = SearchActions::default();
|
||||||
|
|
||||||
|
// --- Query strip -------------------------------------------------
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
let show_elapsed =
|
||||||
|
!self.running && self.elapsed.is_some() && !self.query.trim().is_empty();
|
||||||
|
let slot_room = if self.running {
|
||||||
|
24.0
|
||||||
|
} else if show_elapsed {
|
||||||
|
60.0
|
||||||
|
} else {
|
||||||
|
0.0
|
||||||
|
};
|
||||||
|
let width = ui.available_width() - 170.0 - slot_room;
|
||||||
|
let highlight = &mut self.highlight;
|
||||||
|
let mut layouter = move |ui: &egui::Ui, buf: &dyn egui::TextBuffer, _wrap: f32| {
|
||||||
|
crate::query_highlight::galley(ui, highlight, buf.as_str())
|
||||||
|
};
|
||||||
|
let response = ui.add(
|
||||||
|
egui::TextEdit::singleline(&mut self.query)
|
||||||
|
.desired_width(width.max(120.0))
|
||||||
|
.hint_text("Search names and contents… (type:Document regex:… budget*)")
|
||||||
|
.layouter(&mut layouter),
|
||||||
|
);
|
||||||
|
if self.focus_query {
|
||||||
|
response.request_focus();
|
||||||
|
self.focus_query = false;
|
||||||
|
}
|
||||||
|
if response.changed() {
|
||||||
|
self.pending_edit = Some(Instant::now());
|
||||||
|
}
|
||||||
|
// One slot right of the box: spinner while searching, then the
|
||||||
|
// total wall time of all cascade passes once it lands.
|
||||||
|
if self.running {
|
||||||
|
ui.add(egui::Spinner::new().size(16.0));
|
||||||
|
} else if show_elapsed {
|
||||||
|
if let Some(elapsed) = self.elapsed {
|
||||||
|
ui.label(egui::RichText::new(fmt_elapsed(elapsed)).small().weak())
|
||||||
|
.on_hover_text("Time to run all search passes");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ui
|
||||||
|
.checkbox(&mut self.fuzzy, "Fuzzy")
|
||||||
|
.on_hover_text("Also run fuzzy filename and full-text passes (slower)")
|
||||||
|
.changed()
|
||||||
|
{
|
||||||
|
actions.save_fuzzy_default = Some(self.fuzzy);
|
||||||
|
actions.rerun = true;
|
||||||
|
}
|
||||||
|
if ui.button("?").on_hover_text("Query syntax help").clicked() {
|
||||||
|
self.help_open = !self.help_open;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Session ignore chips.
|
||||||
|
if !self.session_ignores.is_empty() {
|
||||||
|
ui.horizontal_wrapped(|ui| {
|
||||||
|
ui.label(egui::RichText::new("Ignoring:").small().weak());
|
||||||
|
let mut remove: Option<usize> = None;
|
||||||
|
for (i, pattern) in self.session_ignores.iter().enumerate() {
|
||||||
|
if ui
|
||||||
|
.small_button(format!("{} ✕", pattern))
|
||||||
|
.on_hover_text("Remove this session filter")
|
||||||
|
.clicked()
|
||||||
|
{
|
||||||
|
remove = Some(i);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(i) = remove {
|
||||||
|
self.session_ignores.remove(i);
|
||||||
|
actions.rerun = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Notices.
|
||||||
|
if let Some(err) = &self.error {
|
||||||
|
ui.colored_label(ui.visuals().error_fg_color, err);
|
||||||
|
} else if self.limited {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!(
|
||||||
|
"Showing first {} matches; refine the query (limit configurable in Options).",
|
||||||
|
self.results.len()
|
||||||
|
))
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
} else if !self.running
|
||||||
|
&& !self.swap_pending
|
||||||
|
&& self.results.is_empty()
|
||||||
|
&& !self.query.trim().is_empty()
|
||||||
|
&& self.error.is_none()
|
||||||
|
{
|
||||||
|
ui.label(egui::RichText::new("No results.").small().weak());
|
||||||
|
}
|
||||||
|
|
||||||
|
// Result-set transitions pulse instead of strobing: the old table
|
||||||
|
// fades out over 0.25 s while the new hits stage, the sets swap at
|
||||||
|
// zero opacity, and the new table fades back in over 0.25 s.
|
||||||
|
// `animate_value_with_time` keeps requesting repaints until the
|
||||||
|
// value settles.
|
||||||
|
let fade_target = if self.swap_pending { 0.0 } else { 1.0 };
|
||||||
|
let fade = ui.ctx().animate_value_with_time(
|
||||||
|
egui::Id::new("qs-results-fade"),
|
||||||
|
fade_target,
|
||||||
|
0.25,
|
||||||
|
);
|
||||||
|
if self.swap_pending && fade <= 0.01 {
|
||||||
|
self.results = std::mem::take(&mut self.staging);
|
||||||
|
self.has_snippets = self.staging_has_snippets;
|
||||||
|
self.selected = None;
|
||||||
|
self.swap_pending = false;
|
||||||
|
if self.sort == (SortKey::Rank, true) {
|
||||||
|
self.order = (0..self.results.len() as u32).collect();
|
||||||
|
} else {
|
||||||
|
self.sort_dirty = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if self.sort_dirty {
|
||||||
|
self.resort();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fade covers the table and the preview strip below it; the modal
|
||||||
|
// windows and notices render at full opacity on their own layers.
|
||||||
|
ui.set_opacity(fade);
|
||||||
|
|
||||||
|
// --- Results table ------------------------------------------------
|
||||||
|
// Reserve room for the selected-row snippet preview strip. Only
|
||||||
|
// content matches get one — a filename match's "snippet" is the
|
||||||
|
// name, already on screen.
|
||||||
|
let preview_snippet: Option<Snippet> = self
|
||||||
|
.selected
|
||||||
|
.and_then(|i| self.results.get(i as usize))
|
||||||
|
.filter(|h| matches!(h.stage, 5 | 6 | 8))
|
||||||
|
.and_then(|h| h.snippet.clone());
|
||||||
|
let preview_height = if preview_snippet.is_some() { 44.0 } else { 0.0 };
|
||||||
|
let table_height = (ui.available_height() - preview_height).max(60.0);
|
||||||
|
|
||||||
|
let text_height = egui::TextStyle::Body.resolve(ui.style()).size + 4.0;
|
||||||
|
let mut open_ignore_dialog: Option<usize> = None;
|
||||||
|
|
||||||
|
ui.push_id("results", |ui| {
|
||||||
|
let mut table = TableBuilder::new(ui)
|
||||||
|
.striped(true)
|
||||||
|
.resizable(true)
|
||||||
|
.sense(egui::Sense::click())
|
||||||
|
.max_scroll_height(table_height)
|
||||||
|
.min_scrolled_height(60.0)
|
||||||
|
.column(Column::initial(220.0).at_least(80.0).clip(true)) // name
|
||||||
|
.column(Column::remainder().at_least(120.0).clip(true)); // path
|
||||||
|
if self.has_snippets {
|
||||||
|
table = table.column(Column::remainder().at_least(120.0).clip(true));
|
||||||
|
}
|
||||||
|
table = table
|
||||||
|
.column(Column::exact(72.0)) // size
|
||||||
|
.column(Column::exact(110.0)) // modified
|
||||||
|
.column(Column::exact(52.0)); // rank
|
||||||
|
|
||||||
|
table
|
||||||
|
.header(text_height + 4.0, |mut header| {
|
||||||
|
header.col(|ui| self.sort_header(ui, SortKey::Name, "Name"));
|
||||||
|
header.col(|ui| self.sort_header(ui, SortKey::Path, "Path"));
|
||||||
|
if self.has_snippets {
|
||||||
|
header.col(|ui| {
|
||||||
|
ui.with_layout(
|
||||||
|
egui::Layout::centered_and_justified(
|
||||||
|
egui::Direction::LeftToRight,
|
||||||
|
),
|
||||||
|
|ui| {
|
||||||
|
ui.label(egui::RichText::new("Match").strong());
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
header.col(|ui| self.sort_header(ui, SortKey::Size, "Size"));
|
||||||
|
header.col(|ui| self.sort_header(ui, SortKey::Modified, "Modified"));
|
||||||
|
header.col(|ui| self.sort_header(ui, SortKey::Rank, "Rank"));
|
||||||
|
})
|
||||||
|
.body(|body| {
|
||||||
|
let order = self.order.clone();
|
||||||
|
body.rows(text_height, order.len(), |mut row| {
|
||||||
|
let result_ix = order[row.index()] as usize;
|
||||||
|
let hit = &self.results[result_ix];
|
||||||
|
row.set_selected(self.selected == Some(result_ix as u32));
|
||||||
|
|
||||||
|
row.col(|ui| {
|
||||||
|
ui.label(&hit.name);
|
||||||
|
});
|
||||||
|
row.col(|ui| {
|
||||||
|
ui.label(egui::RichText::new(&hit.path).weak());
|
||||||
|
});
|
||||||
|
if self.has_snippets {
|
||||||
|
let snippet = hit.snippet.clone();
|
||||||
|
// Name and path matches show a whole field, so
|
||||||
|
// they render bracketed: [matched field].
|
||||||
|
let whole_field =
|
||||||
|
hit.stage <= 4 || hit.stage == 7 || hit.stage >= 9;
|
||||||
|
row.col(|ui| {
|
||||||
|
if let Some(snip) = &snippet {
|
||||||
|
let width = ui.available_width();
|
||||||
|
let job = centered_match_job(ui, snip, width, whole_field);
|
||||||
|
let response = ui
|
||||||
|
.with_layout(
|
||||||
|
egui::Layout::centered_and_justified(
|
||||||
|
egui::Direction::LeftToRight,
|
||||||
|
),
|
||||||
|
|ui| ui.label(job),
|
||||||
|
)
|
||||||
|
.inner;
|
||||||
|
if !snip.ranges.is_empty() {
|
||||||
|
let hover = snip.clone();
|
||||||
|
response.on_hover_ui(|ui| {
|
||||||
|
ui.set_max_width(520.0);
|
||||||
|
let job = snippet_job(ui, &hover, 10);
|
||||||
|
ui.label(job);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
row.col(|ui| {
|
||||||
|
ui.with_layout(
|
||||||
|
egui::Layout::centered_and_justified(
|
||||||
|
egui::Direction::LeftToRight,
|
||||||
|
),
|
||||||
|
|ui| {
|
||||||
|
ui.label(human_size(hit.size));
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
row.col(|ui| {
|
||||||
|
let color = recency_color(ui, hit.mtime);
|
||||||
|
ui.with_layout(
|
||||||
|
egui::Layout::centered_and_justified(
|
||||||
|
egui::Direction::LeftToRight,
|
||||||
|
),
|
||||||
|
|ui| {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(fmt_mtime(hit.mtime)).color(color),
|
||||||
|
);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
row.col(|ui| {
|
||||||
|
ui.with_layout(
|
||||||
|
egui::Layout::centered_and_justified(
|
||||||
|
egui::Direction::LeftToRight,
|
||||||
|
),
|
||||||
|
|ui| {
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(format!(" {:.2} ", hit.rank))
|
||||||
|
.background_color(rank_tier_color(hit.stage))
|
||||||
|
.color(egui::Color32::from_rgb(32, 32, 32)),
|
||||||
|
);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
let response = row.response();
|
||||||
|
if response.clicked() {
|
||||||
|
self.selected = Some(result_ix as u32);
|
||||||
|
}
|
||||||
|
if response.double_clicked() {
|
||||||
|
platform::open_file(&self.results[result_ix].path);
|
||||||
|
}
|
||||||
|
response.context_menu(|ui| {
|
||||||
|
let path = self.results[result_ix].path.clone();
|
||||||
|
if ui.button("Open").clicked() {
|
||||||
|
platform::open_file(&path);
|
||||||
|
ui.close();
|
||||||
|
}
|
||||||
|
if ui.button("Open containing folder").clicked() {
|
||||||
|
platform::reveal_in_folder(&path);
|
||||||
|
ui.close();
|
||||||
|
}
|
||||||
|
ui.separator();
|
||||||
|
if ui.button("Build ignore filter…").clicked() {
|
||||||
|
open_ignore_dialog = Some(result_ix);
|
||||||
|
ui.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
if let Some(ix) = open_ignore_dialog {
|
||||||
|
let hit = &self.results[ix];
|
||||||
|
self.ignore_dialog = Some(IgnoreDialog {
|
||||||
|
source_name: hit.name.clone(),
|
||||||
|
source_path: hit.path.clone(),
|
||||||
|
pattern: hit.name.clone(),
|
||||||
|
persist: false,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Selected-row preview strip: the full snippet, wrapped.
|
||||||
|
if let Some(snip) = &preview_snippet {
|
||||||
|
ui.separator();
|
||||||
|
let job = snippet_job(ui, snip, 2);
|
||||||
|
ui.label(job);
|
||||||
|
}
|
||||||
|
|
||||||
|
self.ignore_dialog_ui(ui.ctx(), &mut actions);
|
||||||
|
self.help_window_ui(ui.ctx());
|
||||||
|
actions
|
||||||
|
}
|
||||||
|
|
||||||
|
fn ignore_dialog_ui(&mut self, ctx: &egui::Context, actions: &mut SearchActions) {
|
||||||
|
let Some(dialog) = &mut self.ignore_dialog else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let mut apply = false;
|
||||||
|
let mut cancel = false;
|
||||||
|
egui::Window::new("Ignore filter")
|
||||||
|
.collapsible(false)
|
||||||
|
.resizable(false)
|
||||||
|
.anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0])
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.label(format!("From: {}", dialog.source_path));
|
||||||
|
ui.add_space(4.0);
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
if ui.button("This name").clicked() {
|
||||||
|
dialog.pattern = dialog.source_name.clone();
|
||||||
|
}
|
||||||
|
if let Some(ext) = std::path::Path::new(&dialog.source_name)
|
||||||
|
.extension()
|
||||||
|
.and_then(|e| e.to_str())
|
||||||
|
{
|
||||||
|
if ui.button(format!("*.{}", ext)).clicked() {
|
||||||
|
dialog.pattern = format!("*.{}", ext);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(parent) = std::path::Path::new(&dialog.source_path)
|
||||||
|
.parent()
|
||||||
|
.and_then(|p| p.to_str())
|
||||||
|
{
|
||||||
|
if ui.button("This directory").clicked() {
|
||||||
|
dialog.pattern = format!("{}/*", parent);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.add(
|
||||||
|
egui::TextEdit::singleline(&mut dialog.pattern)
|
||||||
|
.desired_width(360.0)
|
||||||
|
.hint_text("glob pattern"),
|
||||||
|
);
|
||||||
|
ui.checkbox(&mut dialog.persist, "Persist to config");
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Session filters hide results immediately. Persisted filters also \
|
||||||
|
exclude files from the index at the next reindex.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
if ui.button("Apply").clicked() {
|
||||||
|
apply = true;
|
||||||
|
}
|
||||||
|
if ui.button("Cancel").clicked() {
|
||||||
|
cancel = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
if apply {
|
||||||
|
let dialog = self.ignore_dialog.take().unwrap();
|
||||||
|
let pattern = dialog.pattern.trim().to_string();
|
||||||
|
if !pattern.is_empty() {
|
||||||
|
if !self.session_ignores.contains(&pattern) {
|
||||||
|
self.session_ignores.push(pattern.clone());
|
||||||
|
}
|
||||||
|
if dialog.persist {
|
||||||
|
actions.persist_ignore = Some(pattern);
|
||||||
|
}
|
||||||
|
actions.rerun = true;
|
||||||
|
}
|
||||||
|
} else if cancel {
|
||||||
|
self.ignore_dialog = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn help_window_ui(&mut self, ctx: &egui::Context) {
|
||||||
|
let mut open = self.help_open;
|
||||||
|
egui::Window::new("Query syntax")
|
||||||
|
.open(&mut open)
|
||||||
|
.resizable(false)
|
||||||
|
.default_width(540.0)
|
||||||
|
.show(ctx, |ui| {
|
||||||
|
ui.label(
|
||||||
|
"Everything that is not a filter is matched as one phrase, in order. \
|
||||||
|
Filters combine freely with the search text.",
|
||||||
|
);
|
||||||
|
ui.add_space(6.0);
|
||||||
|
egui::Grid::new("query-syntax-table")
|
||||||
|
.num_columns(2)
|
||||||
|
.spacing([18.0, 5.0])
|
||||||
|
.striped(true)
|
||||||
|
.show(ui, |ui| {
|
||||||
|
let row = |ui: &mut egui::Ui, syntax: &str, meaning: &str| {
|
||||||
|
ui.monospace(syntax);
|
||||||
|
ui.label(meaning);
|
||||||
|
ui.end_row();
|
||||||
|
};
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"budget report",
|
||||||
|
"names, contents, and paths containing \"budget report\"",
|
||||||
|
);
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"\"exact phrase\"",
|
||||||
|
"quotes keep spaces, stars, and filter-like words literal",
|
||||||
|
);
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"bud*report",
|
||||||
|
"* matches any run of characters (within a line); \
|
||||||
|
also works in name: values",
|
||||||
|
);
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"regex:\"(foo|bar)\\d+\"",
|
||||||
|
"regular expression, matched against names, contents, \
|
||||||
|
and paths; case-insensitive — use (?-i:…) to override; \
|
||||||
|
quote patterns containing spaces",
|
||||||
|
);
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"type:Document",
|
||||||
|
"file class: Audio, Image, Video, Document, Text, \
|
||||||
|
Archive, Spreadsheet, Presentation, Folder",
|
||||||
|
);
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"modified:>=2024-01-01",
|
||||||
|
"modification date (yyyy-mm-dd); also <, <=, > and =",
|
||||||
|
);
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"path:/home/me/docs",
|
||||||
|
"only results in that folder and its subfolders; \
|
||||||
|
quote paths containing spaces",
|
||||||
|
);
|
||||||
|
row(ui, "mime:application/pdf", "exact MIME type");
|
||||||
|
row(
|
||||||
|
ui,
|
||||||
|
"name:report",
|
||||||
|
"filename contains, applied as an unranked filter",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.label("Example:");
|
||||||
|
ui.monospace("type:Document modified:>=2024-01-01 quarterly budget");
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"Ranking: exact filename matches, then filename substrings, then \
|
||||||
|
full-text matches (ordered by occurrences), then fuzzy matches \
|
||||||
|
when enabled, and finally matches on the rest of the file path.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
ui.label(
|
||||||
|
egui::RichText::new(
|
||||||
|
"The complete reference, including ranking details and the \
|
||||||
|
fuzzy edit budget, is the \"Query syntax\" section of \
|
||||||
|
README.md in the QuickSearch folder.",
|
||||||
|
)
|
||||||
|
.small()
|
||||||
|
.weak(),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
self.help_open = open;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct SnippetFormats {
|
||||||
|
normal: TextFormat,
|
||||||
|
highlight: TextFormat,
|
||||||
|
weak: TextFormat,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn snippet_formats(ui: &egui::Ui) -> SnippetFormats {
|
||||||
|
let font_id = egui::TextStyle::Body.resolve(ui.style());
|
||||||
|
SnippetFormats {
|
||||||
|
normal: TextFormat {
|
||||||
|
font_id: font_id.clone(),
|
||||||
|
color: ui.visuals().text_color(),
|
||||||
|
..Default::default()
|
||||||
|
},
|
||||||
|
highlight: TextFormat {
|
||||||
|
font_id: font_id.clone(),
|
||||||
|
color: ui.visuals().strong_text_color(),
|
||||||
|
background: ui.visuals().selection.bg_fill.gamma_multiply(0.4),
|
||||||
|
..Default::default()
|
||||||
|
},
|
||||||
|
weak: TextFormat {
|
||||||
|
font_id,
|
||||||
|
color: ui.visuals().weak_text_color(),
|
||||||
|
..Default::default()
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build a highlighted snippet LayoutJob from byte ranges, wrapped to at
|
||||||
|
/// most `max_rows`. Cheap enough to run per visible row per frame.
|
||||||
|
fn snippet_job(ui: &egui::Ui, snip: &Snippet, max_rows: usize) -> LayoutJob {
|
||||||
|
let fmt = snippet_formats(ui);
|
||||||
|
let mut job = LayoutJob::default();
|
||||||
|
job.wrap.max_rows = max_rows;
|
||||||
|
if max_rows == 1 {
|
||||||
|
job.wrap.break_anywhere = true;
|
||||||
|
}
|
||||||
|
if snip.truncated_start {
|
||||||
|
job.append("… ", 0.0, fmt.weak.clone());
|
||||||
|
}
|
||||||
|
let mut cursor = 0;
|
||||||
|
for &(start, end) in &snip.ranges {
|
||||||
|
if start > cursor {
|
||||||
|
job.append(&snip.window[cursor..start], 0.0, fmt.normal.clone());
|
||||||
|
}
|
||||||
|
job.append(&snip.window[start..end], 0.0, fmt.highlight.clone());
|
||||||
|
cursor = end;
|
||||||
|
}
|
||||||
|
if cursor < snip.window.len() {
|
||||||
|
job.append(&snip.window[cursor..], 0.0, fmt.normal.clone());
|
||||||
|
}
|
||||||
|
if snip.truncated_end {
|
||||||
|
job.append(" …", 0.0, fmt.weak);
|
||||||
|
}
|
||||||
|
job
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Match column cell: one line with the (first) matched span centered
|
||||||
|
/// and an equal amount of context on both sides, trimmed to what fits the
|
||||||
|
/// column width. Matches on a whole field — a filename or a path — are
|
||||||
|
/// wrapped in brackets: `[name]`.
|
||||||
|
fn centered_match_job(
|
||||||
|
ui: &egui::Ui,
|
||||||
|
snip: &Snippet,
|
||||||
|
width_px: f32,
|
||||||
|
whole_field: bool,
|
||||||
|
) -> LayoutJob {
|
||||||
|
let fmt = snippet_formats(ui);
|
||||||
|
let font_id = egui::TextStyle::Body.resolve(ui.style());
|
||||||
|
let char_width = ui.fonts(|f| f.glyph_width(&font_id, '0')).max(1.0);
|
||||||
|
let mut budget = ((width_px / char_width) as usize).saturating_sub(2).max(8);
|
||||||
|
if whole_field {
|
||||||
|
budget = budget.saturating_sub(2); // room for the brackets
|
||||||
|
}
|
||||||
|
|
||||||
|
// Newlines force line breaks even in a one-row LayoutJob, wrecking the
|
||||||
|
// centered single-line cell. Flatten them to spaces — a byte-for-byte
|
||||||
|
// ASCII replacement, so the match ranges stay valid. The mouseover
|
||||||
|
// renders the original window untouched.
|
||||||
|
let flattened = snip.window.replace(['\n', '\r', '\t'], " ");
|
||||||
|
let window = flattened.as_str();
|
||||||
|
let (start, end) = match snip.ranges.first().copied() {
|
||||||
|
Some((a, b)) => {
|
||||||
|
let match_chars = window[a..b].chars().count();
|
||||||
|
let side = budget.saturating_sub(match_chars) / 2;
|
||||||
|
let before = &window[..a];
|
||||||
|
let after = &window[b..];
|
||||||
|
let before_count = before.chars().count();
|
||||||
|
let after_count = after.chars().count();
|
||||||
|
// Equal context on both sides; leftover budget from a short
|
||||||
|
// side flows to the other.
|
||||||
|
let take_before = (side + side.saturating_sub(after_count)).min(before_count);
|
||||||
|
let take_after = (side + side.saturating_sub(before_count)).min(after_count);
|
||||||
|
let start = if take_before == 0 {
|
||||||
|
a
|
||||||
|
} else {
|
||||||
|
before
|
||||||
|
.char_indices()
|
||||||
|
.nth_back(take_before - 1)
|
||||||
|
.map(|(i, _)| i)
|
||||||
|
.unwrap_or(0)
|
||||||
|
};
|
||||||
|
let end = b + after
|
||||||
|
.char_indices()
|
||||||
|
.nth(take_after)
|
||||||
|
.map(|(i, _)| i)
|
||||||
|
.unwrap_or(after.len());
|
||||||
|
(start, end)
|
||||||
|
}
|
||||||
|
None => {
|
||||||
|
// No ranges (shouldn't happen for match cells) — head trim.
|
||||||
|
let end = window
|
||||||
|
.char_indices()
|
||||||
|
.nth(budget)
|
||||||
|
.map(|(i, _)| i)
|
||||||
|
.unwrap_or(window.len());
|
||||||
|
(0, end)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut job = LayoutJob::default();
|
||||||
|
job.wrap.max_rows = 1;
|
||||||
|
job.wrap.break_anywhere = true;
|
||||||
|
if whole_field {
|
||||||
|
job.append("[", 0.0, fmt.weak.clone());
|
||||||
|
}
|
||||||
|
if start > 0 || snip.truncated_start {
|
||||||
|
job.append("…", 0.0, fmt.weak.clone());
|
||||||
|
}
|
||||||
|
let mut cursor = start;
|
||||||
|
for &(a, b) in &snip.ranges {
|
||||||
|
let (a, b) = (a.max(start), b.min(end));
|
||||||
|
if a >= b || a >= end {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if a > cursor {
|
||||||
|
job.append(&window[cursor..a], 0.0, fmt.normal.clone());
|
||||||
|
}
|
||||||
|
job.append(&window[a..b], 0.0, fmt.highlight.clone());
|
||||||
|
cursor = b;
|
||||||
|
}
|
||||||
|
if cursor < end {
|
||||||
|
job.append(&window[cursor..end], 0.0, fmt.normal.clone());
|
||||||
|
}
|
||||||
|
if end < window.len() || snip.truncated_end {
|
||||||
|
job.append("…", 0.0, fmt.weak.clone());
|
||||||
|
}
|
||||||
|
if whole_field {
|
||||||
|
job.append("]", 0.0, fmt.weak);
|
||||||
|
}
|
||||||
|
job
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Tier-list chip color per cascade stage — lower rank, higher tier:
|
||||||
|
/// S-red for exact case-sensitive filename matches down through the
|
||||||
|
/// pastel ramp to purple for fuzzy full-text and on to the grey path
|
||||||
|
/// tiers. Dark text on these pastels stays readable in both themes.
|
||||||
|
fn rank_tier_color(stage: u8) -> egui::Color32 {
|
||||||
|
match stage {
|
||||||
|
1 => egui::Color32::from_rgb(255, 127, 127), // S
|
||||||
|
2 => egui::Color32::from_rgb(255, 191, 127), // A
|
||||||
|
3 => egui::Color32::from_rgb(255, 223, 127), // B
|
||||||
|
4 => egui::Color32::from_rgb(255, 255, 127), // C
|
||||||
|
5 => egui::Color32::from_rgb(191, 255, 127), // D
|
||||||
|
6 => egui::Color32::from_rgb(127, 255, 127), // E
|
||||||
|
7 => egui::Color32::from_rgb(127, 191, 255), // F
|
||||||
|
8 => egui::Color32::from_rgb(191, 127, 255), // G
|
||||||
|
9 => egui::Color32::from_rgb(223, 159, 255), // H — path, exact case
|
||||||
|
10 => egui::Color32::from_rgb(239, 191, 239), // I — path, any case
|
||||||
|
_ => egui::Color32::from_rgb(199, 199, 199), // J — fuzzy path
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Timestamp color: fresh files get a green tint that fades into the weak
|
||||||
|
/// text color over ~2 years on a log scale.
|
||||||
|
fn recency_color(ui: &egui::Ui, mtime: i64) -> egui::Color32 {
|
||||||
|
let now = std::time::SystemTime::now()
|
||||||
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
.map(|d| d.as_secs() as i64)
|
||||||
|
.unwrap_or(0);
|
||||||
|
let age_hours = ((now - mtime).max(0) as f32 / 3600.0).max(1.0);
|
||||||
|
const HORIZON_HOURS: f32 = 24.0 * 365.0 * 2.0;
|
||||||
|
let t = (age_hours.ln() / HORIZON_HOURS.ln()).clamp(0.0, 1.0);
|
||||||
|
let fresh = egui::Color32::from_rgb(87, 187, 122);
|
||||||
|
let old = ui.visuals().weak_text_color();
|
||||||
|
let lerp = |a: u8, b: u8| (a as f32 + (b as f32 - a as f32) * t).round() as u8;
|
||||||
|
egui::Color32::from_rgb(
|
||||||
|
lerp(fresh.r(), old.r()),
|
||||||
|
lerp(fresh.g(), old.g()),
|
||||||
|
lerp(fresh.b(), old.b()),
|
||||||
|
)
|
||||||
|
}
|
||||||
144
crates/quicksearch-gui/src/tracker.rs
Normal file
|
|
@ -0,0 +1,144 @@
|
||||||
|
//! Indexing-rate estimation for the status displays.
|
||||||
|
//!
|
||||||
|
//! The old tracker sampled the counter every poll tick but pruned to a
|
||||||
|
//! 1-second window, so anything slower than ~1 file/sec measured a
|
||||||
|
//! genuine zero and displayed "0.0 files/sec" despite progress. This one
|
||||||
|
//! records a point only when the counter *changes*, keeps up to 60 s of
|
||||||
|
//! history but never fewer than two points (so slow rates stay
|
||||||
|
//! computable), and measures against `now` so the estimate decays during
|
||||||
|
//! stalls instead of freezing at the last burst.
|
||||||
|
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
const HISTORY: Duration = Duration::from_secs(60);
|
||||||
|
|
||||||
|
pub struct SpeedTracker {
|
||||||
|
/// (when, counter value) — appended only on counter change.
|
||||||
|
points: Vec<(Instant, usize)>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SpeedTracker {
|
||||||
|
pub fn new() -> SpeedTracker {
|
||||||
|
SpeedTracker { points: Vec::new() }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reset between phases (each phase restarts its counter).
|
||||||
|
pub fn reset(&mut self) {
|
||||||
|
self.points.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn record(&mut self, files_processed: usize) {
|
||||||
|
self.record_at(Instant::now(), files_processed);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn record_at(&mut self, now: Instant, files_processed: usize) {
|
||||||
|
match self.points.last() {
|
||||||
|
Some(&(_, last)) if last == files_processed => return,
|
||||||
|
// Counter went backwards — a new phase started without an
|
||||||
|
// explicit reset.
|
||||||
|
Some(&(_, last)) if files_processed < last => self.points.clear(),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
self.points.push((now, files_processed));
|
||||||
|
// Prune old points, but always keep at least two so a slow but
|
||||||
|
// steady rate never becomes unmeasurable.
|
||||||
|
while self.points.len() > 2
|
||||||
|
&& now.duration_since(self.points[0].0) > HISTORY
|
||||||
|
{
|
||||||
|
self.points.remove(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Estimated files/sec, measured from the oldest retained progress
|
||||||
|
/// point to *now*. `None` until two data points exist.
|
||||||
|
pub fn files_per_sec(&self) -> Option<f64> {
|
||||||
|
self.files_per_sec_at(Instant::now())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn files_per_sec_at(&self, now: Instant) -> Option<f64> {
|
||||||
|
let (t0, c0) = *self.points.first()?;
|
||||||
|
let (_, c1) = *self.points.last()?;
|
||||||
|
if self.points.len() < 2 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let span = now.duration_since(t0).as_secs_f64();
|
||||||
|
if span <= 0.0 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some((c1 - c0) as f64 / span)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn needs_two_points() {
|
||||||
|
let mut t = SpeedTracker::new();
|
||||||
|
let base = Instant::now();
|
||||||
|
assert!(t.files_per_sec_at(base).is_none());
|
||||||
|
t.record_at(base, 10);
|
||||||
|
assert!(t.files_per_sec_at(base).is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn slow_rate_is_measurable_not_zero() {
|
||||||
|
// One file every 2.5 s — the old 1 s window reported 0.0 here.
|
||||||
|
let mut t = SpeedTracker::new();
|
||||||
|
let base = Instant::now();
|
||||||
|
for i in 0..4 {
|
||||||
|
t.record_at(base + Duration::from_millis(2500 * i), 10 + i as usize);
|
||||||
|
}
|
||||||
|
let rate = t.files_per_sec_at(base + Duration::from_millis(7500)).unwrap();
|
||||||
|
assert!((rate - 0.4).abs() < 0.01, "expected ~0.4/s, got {}", rate);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unchanged_counter_adds_no_points() {
|
||||||
|
let mut t = SpeedTracker::new();
|
||||||
|
let base = Instant::now();
|
||||||
|
for i in 0..100 {
|
||||||
|
t.record_at(base + Duration::from_millis(50 * i), 42);
|
||||||
|
}
|
||||||
|
assert_eq!(t.points.len(), 1, "only the first observation recorded");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn stall_decays_toward_zero() {
|
||||||
|
let mut t = SpeedTracker::new();
|
||||||
|
let base = Instant::now();
|
||||||
|
t.record_at(base, 0);
|
||||||
|
t.record_at(base + Duration::from_secs(1), 100); // 100/s burst
|
||||||
|
let just_after = t.files_per_sec_at(base + Duration::from_secs(1)).unwrap();
|
||||||
|
let stalled = t.files_per_sec_at(base + Duration::from_secs(20)).unwrap();
|
||||||
|
assert!(just_after > 90.0);
|
||||||
|
assert!(stalled < 6.0, "estimate must decay during a stall: {}", stalled);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pruning_keeps_at_least_two_points() {
|
||||||
|
let mut t = SpeedTracker::new();
|
||||||
|
let base = Instant::now();
|
||||||
|
t.record_at(base, 1);
|
||||||
|
t.record_at(base + Duration::from_secs(30), 2);
|
||||||
|
// Far beyond the history window; both points are older than 60 s
|
||||||
|
// relative to this record.
|
||||||
|
t.record_at(base + Duration::from_secs(300), 3);
|
||||||
|
assert!(t.points.len() >= 2);
|
||||||
|
assert!(t
|
||||||
|
.files_per_sec_at(base + Duration::from_secs(300))
|
||||||
|
.is_some());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn counter_regression_resets() {
|
||||||
|
let mut t = SpeedTracker::new();
|
||||||
|
let base = Instant::now();
|
||||||
|
t.record_at(base, 500);
|
||||||
|
t.record_at(base + Duration::from_secs(1), 600);
|
||||||
|
// New phase restarts from a small number.
|
||||||
|
t.record_at(base + Duration::from_secs(2), 3);
|
||||||
|
assert_eq!(t.points.len(), 1);
|
||||||
|
}
|
||||||
|
}
|
||||||
224
packaging/build-deb.sh
Normal file
|
|
@ -0,0 +1,224 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# Build a Debian package for QuickSearch.
|
||||||
|
#
|
||||||
|
# ./packaging/build-deb.sh build and package
|
||||||
|
# ./packaging/build-deb.sh --no-build package an existing release binary
|
||||||
|
# ./packaging/build-deb.sh --no-strip keep debug symbols (25 MB vs 20 MB)
|
||||||
|
# ./packaging/build-deb.sh -o /tmp/out write the .deb somewhere else
|
||||||
|
#
|
||||||
|
# Environment: DEB_REVISION (default 1), DEB_MAINTAINER, SOURCE_DATE_EPOCH.
|
||||||
|
#
|
||||||
|
# Deliberately does not use cargo-deb, debhelper, fakeroot or an SVG rasteriser:
|
||||||
|
# dpkg-deb and desktop-file-utils are the only tools required, and both are part
|
||||||
|
# of a standard Debian or Ubuntu install.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
# Directories created along the way must be 0755, not whatever the caller's
|
||||||
|
# umask happens to be, or the package ships group-writable directories.
|
||||||
|
umask 022
|
||||||
|
|
||||||
|
readonly PKG=quicksearch
|
||||||
|
# The GUI binary and the console-subsystem terminal binary. Both ship: on Unix
|
||||||
|
# `quicksearch` does both jobs, but the README and the shared man page name
|
||||||
|
# `quicksearch-cli` too, so it has to exist wherever the docs are installed.
|
||||||
|
readonly BINARIES=(quicksearch quicksearch-cli)
|
||||||
|
readonly REPO_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
readonly ICON_SRC="$REPO_ROOT/crates/quicksearch-gui/assets/icons"
|
||||||
|
readonly ICON_SVG="$ICON_SRC/quicksearch_icon.svg"
|
||||||
|
|
||||||
|
do_build=1
|
||||||
|
do_strip=1
|
||||||
|
out_dir="$REPO_ROOT/dist"
|
||||||
|
|
||||||
|
die() { printf 'build-deb: %s\n' "$*" >&2; exit 1; }
|
||||||
|
say() { printf '\033[1m==>\033[0m %s\n' "$*"; }
|
||||||
|
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--no-build) do_build=0 ;;
|
||||||
|
--no-strip) do_strip=0 ;;
|
||||||
|
-o|--output-dir) shift; [ $# -gt 0 ] || die "--output-dir needs a path"; out_dir="$1" ;;
|
||||||
|
# Print the header comment block, however long it grows.
|
||||||
|
-h|--help) awk 'NR > 1 { if ($0 !~ /^#/) exit; sub(/^# ?/, ""); print }' "${BASH_SOURCE[0]}"; exit 0 ;;
|
||||||
|
*) die "unknown option: $1 (try --help)" ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
|
||||||
|
for tool in dpkg-deb dpkg desktop-file-validate objdump gzip; do
|
||||||
|
command -v "$tool" >/dev/null 2>&1 || die "missing required tool: $tool"
|
||||||
|
done
|
||||||
|
[ "$do_strip" -eq 0 ] || command -v strip >/dev/null 2>&1 || die "missing strip (install binutils, or pass --no-strip)"
|
||||||
|
|
||||||
|
# Version comes from [workspace.package] so the package can never drift from the
|
||||||
|
# crate version.
|
||||||
|
version="$(sed -n '/^\[workspace\.package\]/,/^\[/{ s/^version[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p }' "$REPO_ROOT/Cargo.toml")"
|
||||||
|
[ -n "$version" ] || die "could not read version from Cargo.toml"
|
||||||
|
|
||||||
|
revision="${DEB_REVISION:-1}"
|
||||||
|
maintainer="${DEB_MAINTAINER:-Jeremy <jeremy@karsttech.com>}"
|
||||||
|
arch="$(dpkg --print-architecture)"
|
||||||
|
deb_version="${version}-${revision}"
|
||||||
|
stage="$out_dir/${PKG}_${deb_version}_${arch}"
|
||||||
|
deb="$out_dir/${PKG}_${deb_version}_${arch}.deb"
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- build ----
|
||||||
|
|
||||||
|
if [ "$do_build" -eq 1 ]; then
|
||||||
|
say "Building quicksearch $version (release)"
|
||||||
|
( cd "$REPO_ROOT" && cargo build --release -p quicksearch-gui )
|
||||||
|
fi
|
||||||
|
|
||||||
|
for bin in "${BINARIES[@]}"; do
|
||||||
|
[ -x "$REPO_ROOT/target/release/$bin" ] \
|
||||||
|
|| die "no release binary at target/release/$bin (drop --no-build?)"
|
||||||
|
done
|
||||||
|
# Both binaries link the same crates, so either gives the same glibc floor.
|
||||||
|
readonly primary_binary="$REPO_ROOT/target/release/$PKG"
|
||||||
|
[ -f "$ICON_SVG" ] || die "no icon at $ICON_SVG"
|
||||||
|
|
||||||
|
say "Validating desktop entry"
|
||||||
|
desktop-file-validate "$REPO_ROOT/packaging/$PKG.desktop"
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- stage -----
|
||||||
|
|
||||||
|
say "Staging $stage"
|
||||||
|
rm -rf "$stage"
|
||||||
|
mkdir -p "$stage"
|
||||||
|
|
||||||
|
for bin in "${BINARIES[@]}"; do
|
||||||
|
install -Dm755 "$REPO_ROOT/target/release/$bin" "$stage/usr/bin/$bin"
|
||||||
|
done
|
||||||
|
install -Dm644 "$REPO_ROOT/packaging/$PKG.desktop" "$stage/usr/share/applications/$PKG.desktop"
|
||||||
|
install -Dm644 "$ICON_SVG" "$stage/usr/share/icons/hicolor/scalable/apps/$PKG.svg"
|
||||||
|
install -Dm644 "$REPO_ROOT/packaging/copyright" "$stage/usr/share/doc/$PKG/copyright"
|
||||||
|
install -Dm644 "$REPO_ROOT/config_example.toml" "$stage/usr/share/doc/$PKG/config_example.toml"
|
||||||
|
install -Dm644 "$REPO_ROOT/README.md" "$stage/usr/share/doc/$PKG/README.md"
|
||||||
|
|
||||||
|
# The PNGs are committed next to the quicksearch_icon.svg they came from, so
|
||||||
|
# this script only copies them and an ordinary `cargo build` needs no image
|
||||||
|
# tooling. To re-render them after editing the SVG, build a throwaway crate
|
||||||
|
# (outside this workspace, to keep it out of Cargo.lock) depending on
|
||||||
|
# `resvg = { version = "0.45", default-features = false }` and, for each size N,
|
||||||
|
# parse with usvg::Tree::from_data, make a tiny_skia::Pixmap::new(N, N), call
|
||||||
|
# resvg::render with Transform::from_scale(N/240.0, N/240.0) and save_png. The
|
||||||
|
# SVG contains no <text>, so no font support is needed.
|
||||||
|
#
|
||||||
|
# Sizes follow whatever is committed, so adding one needs no script change.
|
||||||
|
shopt -s nullglob
|
||||||
|
icons=("$ICON_SRC"/$PKG-*.png)
|
||||||
|
shopt -u nullglob
|
||||||
|
[ "${#icons[@]}" -gt 0 ] || die "no icons in $ICON_SRC"
|
||||||
|
for png in "${icons[@]}"; do
|
||||||
|
n="$(basename "$png" .png)"; n="${n#$PKG-}"
|
||||||
|
# The glob would also catch a non-size name like quicksearch-cli.png.
|
||||||
|
case "$n" in ''|*[!0-9]*) die "unexpected icon name: $(basename "$png")" ;; esac
|
||||||
|
install -Dm644 "$png" "$stage/usr/share/icons/hicolor/${n}x${n}/apps/$PKG.png"
|
||||||
|
done
|
||||||
|
say "Installed ${#icons[@]} icon sizes plus the scalable SVG"
|
||||||
|
|
||||||
|
# Debian wants man pages and the changelog compressed, with no gzip timestamp so
|
||||||
|
# repeat builds are byte-identical. quicksearch-cli.1 is a one-line .so stub
|
||||||
|
# pointing at quicksearch.1, which documents both binaries.
|
||||||
|
install -dm755 "$stage/usr/share/man/man1"
|
||||||
|
for page in "${BINARIES[@]}"; do
|
||||||
|
gzip -9nc "$REPO_ROOT/packaging/$page.1" > "$stage/usr/share/man/man1/$page.1.gz"
|
||||||
|
chmod 644 "$stage/usr/share/man/man1/$page.1.gz"
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ -n "${SOURCE_DATE_EPOCH:-}" ]; then
|
||||||
|
changelog_date="$(date -R -u -d "@$SOURCE_DATE_EPOCH")"
|
||||||
|
else
|
||||||
|
changelog_date="$(date -R)"
|
||||||
|
fi
|
||||||
|
gzip -9nc <<EOF > "$stage/usr/share/doc/$PKG/changelog.Debian.gz"
|
||||||
|
$PKG ($deb_version) unstable; urgency=medium
|
||||||
|
|
||||||
|
* Package build of $PKG $version.
|
||||||
|
|
||||||
|
-- $maintainer $changelog_date
|
||||||
|
EOF
|
||||||
|
chmod 644 "$stage/usr/share/doc/$PKG/changelog.Debian.gz"
|
||||||
|
|
||||||
|
# No config.toml is installed anywhere. Config::config_path() treats a
|
||||||
|
# config.toml sitting next to the executable as portable mode and lets it
|
||||||
|
# override the per-user config outright, so one in /usr/bin would hijack every
|
||||||
|
# account on the machine. Config::load_from creates ~/.config/quicksearch/
|
||||||
|
# config.toml on first run instead.
|
||||||
|
|
||||||
|
if [ "$do_strip" -eq 1 ]; then
|
||||||
|
for bin in "${BINARIES[@]}"; do
|
||||||
|
before="$(du -h "$stage/usr/bin/$bin" | cut -f1)"
|
||||||
|
strip --strip-unneeded "$stage/usr/bin/$bin"
|
||||||
|
say "Stripped $bin: $before -> $(du -h "$stage/usr/bin/$bin" | cut -f1)"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
installed_size="$(du -ks "$stage" | cut -f1)"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- control -----
|
||||||
|
|
||||||
|
# The dynamic section only names libc, libgcc, libm and libbz2: winit and glutin
|
||||||
|
# dlopen the entire display stack, so dpkg-shlibdeps cannot see it and the list
|
||||||
|
# below is maintained by hand. Re-derive it with
|
||||||
|
# objdump -p target/release/quicksearch | grep NEEDED
|
||||||
|
# strings -a target/release/quicksearch | grep -oE 'lib[A-Za-z0-9_+-]+\.so(\.[0-9]+)*' | sort -u
|
||||||
|
# and map each soname to a package with `dpkg -S`.
|
||||||
|
glibc_min="$(objdump -T "$primary_binary" | sed -n 's/.*GLIBC_\([0-9][0-9.]*\).*/\1/p' | sort -V | tail -1)"
|
||||||
|
[ -n "$glibc_min" ] || die "could not determine the glibc version requirement"
|
||||||
|
|
||||||
|
depends="libc6 (>= ${glibc_min}), libgcc-s1 (>= 3.0), libbz2-1.0"
|
||||||
|
depends="$depends, libx11-6, libxcb1, libxcursor1, libxi6, libxrender1"
|
||||||
|
depends="$depends, libxkbcommon0, libxkbcommon-x11-0"
|
||||||
|
depends="$depends, libwayland-client0, libwayland-egl1"
|
||||||
|
depends="$depends, libegl1, libgl1"
|
||||||
|
# Required by policy for anything installing into the hicolor theme; it also
|
||||||
|
# provides the dpkg trigger that refreshes the icon cache on install.
|
||||||
|
depends="$depends, hicolor-icon-theme"
|
||||||
|
|
||||||
|
# desktop-file-utils owns the /usr/share/applications trigger; dbus-bin provides
|
||||||
|
# the dbus-send used by "reveal in folder"; xdg-utils provides the xdg-open
|
||||||
|
# fallback; xdg-desktop-portal backs the native folder picker. None are needed
|
||||||
|
# to search, so none are hard dependencies.
|
||||||
|
recommends="desktop-file-utils, xdg-utils, dbus-bin, xdg-desktop-portal"
|
||||||
|
|
||||||
|
install -dm755 "$stage/DEBIAN"
|
||||||
|
cat > "$stage/DEBIAN/control" <<EOF
|
||||||
|
Package: $PKG
|
||||||
|
Version: $deb_version
|
||||||
|
Section: utils
|
||||||
|
Priority: optional
|
||||||
|
Architecture: $arch
|
||||||
|
Maintainer: $maintainer
|
||||||
|
Installed-Size: $installed_size
|
||||||
|
Depends: $depends
|
||||||
|
Recommends: $recommends
|
||||||
|
Homepage: https://code.karsttech.com/jeremy/quick_search
|
||||||
|
Description: fast full-text search across your files
|
||||||
|
QuickSearch keeps a SQLite/FTS5 index of the directories you choose and
|
||||||
|
searches them by both filename and file content. It extracts text from
|
||||||
|
documents, PDFs, archives and office files, watches the indexed paths for
|
||||||
|
changes and reindexes in the background while the application is open.
|
||||||
|
.
|
||||||
|
The same binary doubles as a terminal search tool: "quicksearch <terms>"
|
||||||
|
prints ranked results and exits without starting the indexer, the file
|
||||||
|
watcher or any background thread.
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# No postinst/postrm: hicolor-icon-theme and desktop-file-utils declare
|
||||||
|
# interest-noawait on /usr/share/icons/hicolor and /usr/share/applications, so
|
||||||
|
# dpkg refreshes both caches on install and removal by itself.
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- build -----
|
||||||
|
|
||||||
|
say "Building $deb"
|
||||||
|
dpkg-deb --root-owner-group --build "$stage" "$deb" >/dev/null
|
||||||
|
rm -rf "$stage"
|
||||||
|
|
||||||
|
echo
|
||||||
|
dpkg-deb --info "$deb"
|
||||||
|
echo
|
||||||
|
say "Done: $deb"
|
||||||
|
echo " install with: sudo apt install $deb"
|
||||||
|
echo " inspect with: dpkg-deb --contents $deb"
|
||||||
61
packaging/copyright
Normal file
|
|
@ -0,0 +1,61 @@
|
||||||
|
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
|
||||||
|
Upstream-Name: quicksearch
|
||||||
|
Upstream-Contact: Jeremy <jeremy@karsttech.com>
|
||||||
|
Source: https://code.karsttech.com/jeremy/quick_search.git
|
||||||
|
|
||||||
|
Files: *
|
||||||
|
Copyright: 2025-2026 Jeremy <jeremy@karsttech.com>
|
||||||
|
License: GPL-3.0-or-later
|
||||||
|
|
||||||
|
Files: crates/quicksearch-gui/assets/icons/*
|
||||||
|
Copyright: 2026 Jeremy <jeremy@karsttech.com>
|
||||||
|
License: GPL-3.0-or-later
|
||||||
|
|
||||||
|
License: GPL-3.0-or-later
|
||||||
|
This program is free software: you can redistribute it and/or modify
|
||||||
|
it under the terms of the GNU General Public License as published by
|
||||||
|
the Free Software Foundation, either version 3 of the License, or
|
||||||
|
(at your option) any later version.
|
||||||
|
.
|
||||||
|
This program is distributed in the hope that it will be useful,
|
||||||
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
GNU General Public License for more details.
|
||||||
|
.
|
||||||
|
You should have received a copy of the GNU General Public License
|
||||||
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
|
.
|
||||||
|
On Debian systems the full text of the GNU General Public License
|
||||||
|
version 3 can be found in /usr/share/common-licenses/GPL-3.
|
||||||
|
|
||||||
|
Comment:
|
||||||
|
/usr/bin/quicksearch and /usr/bin/quicksearch-cli are statically linked Rust
|
||||||
|
binaries built from the same crates. Their third-party components are not
|
||||||
|
shipped as separate files, so they are summarised here
|
||||||
|
rather than given individual stanzas. Every one of the 512 crates in
|
||||||
|
Cargo.lock is under a permissive licence, all of which are compatible with
|
||||||
|
GPL-3.0-or-later:
|
||||||
|
.
|
||||||
|
* MIT and/or Apache-2.0 (the large majority)
|
||||||
|
* Unicode-3.0, BSD-2-Clause, BSD-3-Clause, ISC, Zlib, 0BSD,
|
||||||
|
BSL-1.0, CC0-1.0, Unlicense
|
||||||
|
.
|
||||||
|
There are no GPL-only, LGPL-only, MPL or proprietary dependencies. Note
|
||||||
|
that a dozen crates (winit, glutin, ab_glyph, owned_ttf_parser and
|
||||||
|
related) are Apache-2.0 only; Apache-2.0 is compatible with GPL version 3
|
||||||
|
but not with GPL version 2, which is why this package is
|
||||||
|
GPL-3.0-or-later rather than GPL-2.0-or-later.
|
||||||
|
.
|
||||||
|
Bundled C sources compiled into the binary: SQLite (public domain, via
|
||||||
|
rusqlite), Zstandard (BSD-3-Clause arm of its BSD/GPL-2.0 dual licence,
|
||||||
|
via zstd-sys) and bzip2 (BSD-style, via bzip2-sys).
|
||||||
|
.
|
||||||
|
The binary also embeds egui's default fonts (crate epaint_default_fonts,
|
||||||
|
"(MIT OR Apache-2.0) AND OFL-1.1 AND Ubuntu-font-1.0"). The FSF regards
|
||||||
|
OFL-1.1 and the Ubuntu Font Licence as free but GPL-incompatible; they are
|
||||||
|
included here as font data rather than as linked program code, which is
|
||||||
|
the same basis on which Debian ships these fonts and other egui-based
|
||||||
|
applications.
|
||||||
|
.
|
||||||
|
Run `cargo metadata --all-features` against the source tree to reproduce
|
||||||
|
the per-crate licence list.
|
||||||
3
packaging/quicksearch-cli.1
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
.\" quicksearch-cli is documented alongside quicksearch; this stub makes
|
||||||
|
.\" `man quicksearch-cli` resolve to that page.
|
||||||
|
.so man1/quicksearch.1
|
||||||
136
packaging/quicksearch.1
Normal file
|
|
@ -0,0 +1,136 @@
|
||||||
|
.TH QUICKSEARCH 1 "2026-08-02" "quicksearch 0.1.0" "User Commands"
|
||||||
|
.SH NAME
|
||||||
|
quicksearch, quicksearch\-cli \- fast full\-text search across your files
|
||||||
|
.SH SYNOPSIS
|
||||||
|
.B quicksearch
|
||||||
|
.br
|
||||||
|
.B quicksearch
|
||||||
|
.RI [ FLAGS ]
|
||||||
|
.IR "query terms" ...
|
||||||
|
.br
|
||||||
|
.B quicksearch\-cli
|
||||||
|
.RI [ FLAGS ]
|
||||||
|
.IR "query terms" ...
|
||||||
|
.SH DESCRIPTION
|
||||||
|
.B quicksearch
|
||||||
|
maintains a SQLite/FTS5 index of the paths listed in its configuration and
|
||||||
|
searches it by filename and file content.
|
||||||
|
.PP
|
||||||
|
Invoked with no positional arguments it opens the desktop application, which
|
||||||
|
owns the indexer: it walks the configured roots, watches them for changes and
|
||||||
|
periodically reindexes.
|
||||||
|
.PP
|
||||||
|
Invoked with one or more query terms it runs the same ranked search cascade the
|
||||||
|
application uses, prints the results and exits. Terminal mode never starts the
|
||||||
|
indexer, the filesystem watcher or any background thread, so a one\-shot query
|
||||||
|
neither leaves anything running nor competes for inotify watches with a running
|
||||||
|
application. It requires an index that already exists; run the application once
|
||||||
|
to build one.
|
||||||
|
.PP
|
||||||
|
.B quicksearch\-cli
|
||||||
|
is terminal search and nothing else, and accepts the same flags and query
|
||||||
|
syntax. On this platform it is equivalent to giving
|
||||||
|
.B quicksearch
|
||||||
|
a query, and exists so that scripts and documentation have one name that never
|
||||||
|
opens a window. It matters on Windows, where the two cannot be one executable:
|
||||||
|
a GUI built as a console application flashes a console window at every launch,
|
||||||
|
and a console tool built as a GUI application cannot write back to the shell
|
||||||
|
that invoked it. Given no query it prints usage and exits 2 rather than falling
|
||||||
|
back to the application.
|
||||||
|
.SH OPTIONS
|
||||||
|
.TP
|
||||||
|
.B \-\-fuzzy
|
||||||
|
Also run the fuzzy filename and full\-text passes, which tolerate spelling
|
||||||
|
differences at the cost of speed. The edit distance comes from
|
||||||
|
.I [search].fuzzy_max_edits
|
||||||
|
in the configuration.
|
||||||
|
.TP
|
||||||
|
.BI \-\-limit " N"
|
||||||
|
Return at most
|
||||||
|
.I N
|
||||||
|
results. Defaults to
|
||||||
|
.I [search].display_limit
|
||||||
|
in the configuration.
|
||||||
|
.B \-\-limit=N
|
||||||
|
is also accepted.
|
||||||
|
.TP
|
||||||
|
.B \-\-long
|
||||||
|
Print rank, size, modification time and a matching snippet for each hit instead
|
||||||
|
of bare paths. Highlights the match in bold when stdout is a terminal.
|
||||||
|
.TP
|
||||||
|
.BR \-h ", " \-\-help
|
||||||
|
Print usage and exit.
|
||||||
|
.PP
|
||||||
|
An unrecognised option given without any query terms is passed through and the
|
||||||
|
application is opened, since it may be an option for the windowing backend.
|
||||||
|
.SH QUERY SYNTAX
|
||||||
|
Plain words form a single phrase. Filters may be combined with it:
|
||||||
|
.TP
|
||||||
|
.B type:\fIName\fR
|
||||||
|
Match a file class, for example
|
||||||
|
.IR type:Document ", " type:Image ", " type:Audio .
|
||||||
|
.TP
|
||||||
|
.B modified:\fIexpr\fR
|
||||||
|
Compare against the modification date, for example
|
||||||
|
.IR modified:>=2024-01-01 .
|
||||||
|
.TP
|
||||||
|
.B path:\fI/dir\fR
|
||||||
|
Restrict results to a directory.
|
||||||
|
.TP
|
||||||
|
.B mime:\fItype\fR
|
||||||
|
Match a MIME type, for example
|
||||||
|
.IR mime:application/pdf .
|
||||||
|
.TP
|
||||||
|
.B name:\fIfragment\fR
|
||||||
|
Match a fragment of the filename.
|
||||||
|
.SH FILES
|
||||||
|
.TP
|
||||||
|
.I ~/.config/quicksearch/config.toml
|
||||||
|
Configuration. Created with defaults on first run. See
|
||||||
|
.I /usr/share/doc/quicksearch/config_example.toml
|
||||||
|
for the annotated reference.
|
||||||
|
.TP
|
||||||
|
.I ~/.local/share/quicksearch/index.sqlite
|
||||||
|
The index. The location is set by
|
||||||
|
.I [paths].database_path
|
||||||
|
in the configuration.
|
||||||
|
.TP
|
||||||
|
.I ./config.toml
|
||||||
|
A configuration file placed next to the
|
||||||
|
.B quicksearch
|
||||||
|
executable selects portable mode and overrides the per\-user configuration
|
||||||
|
entirely. The packaged build deliberately installs no such file.
|
||||||
|
.PP
|
||||||
|
.I XDG_CONFIG_HOME
|
||||||
|
and
|
||||||
|
.I XDG_DATA_HOME
|
||||||
|
are honoured when set to absolute paths.
|
||||||
|
.SH EXIT STATUS
|
||||||
|
.TP
|
||||||
|
.B 0
|
||||||
|
The search completed, or
|
||||||
|
.B \-\-help
|
||||||
|
was requested.
|
||||||
|
.TP
|
||||||
|
.B 1
|
||||||
|
The application failed to start.
|
||||||
|
.TP
|
||||||
|
.B 2
|
||||||
|
The configuration could not be read, the index could not be opened, the query
|
||||||
|
was rejected, or
|
||||||
|
.B quicksearch\-cli
|
||||||
|
was given no query.
|
||||||
|
.SH EXAMPLES
|
||||||
|
.TP
|
||||||
|
Search for a phrase:
|
||||||
|
.B quicksearch quarterly revenue report
|
||||||
|
.TP
|
||||||
|
Recent PDFs under a directory, with details:
|
||||||
|
.B quicksearch \-\-long type:Document mime:application/pdf path:/home/me/docs modified:>=2026-01-01
|
||||||
|
.TP
|
||||||
|
Tolerate misspellings and cap the output:
|
||||||
|
.B quicksearch \-\-fuzzy \-\-limit 10 recieve
|
||||||
|
.SH SEE ALSO
|
||||||
|
.I /usr/share/doc/quicksearch/README.md
|
||||||
|
.SH AUTHOR
|
||||||
|
Jeremy <jeremy@karsttech.com>
|
||||||
19
packaging/quicksearch.desktop
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
[Desktop Entry]
|
||||||
|
Type=Application
|
||||||
|
Version=1.0
|
||||||
|
Name=QuickSearch
|
||||||
|
GenericName=File Search
|
||||||
|
Comment=Fast full-text search across your files
|
||||||
|
Exec=quicksearch
|
||||||
|
TryExec=quicksearch
|
||||||
|
Icon=quicksearch
|
||||||
|
Terminal=false
|
||||||
|
# One main category only (Utility), so the app shows up exactly once in the
|
||||||
|
# menu; Filesystem is the additional category that describes what it does.
|
||||||
|
Categories=Utility;Filesystem;
|
||||||
|
Keywords=search;find;index;full-text;files;content;
|
||||||
|
StartupNotify=true
|
||||||
|
# Matches ViewportBuilder::with_app_id("quicksearch") in the GUI, which becomes
|
||||||
|
# the Wayland app id and the X11 WM_CLASS. Without the match the desktop shows a
|
||||||
|
# generic window icon.
|
||||||
|
StartupWMClass=quicksearch
|
||||||
12
run.bat
|
|
@ -1 +1,11 @@
|
||||||
cargo run -p quicksearch-gui
|
@echo off
|
||||||
|
rem Build and launch the QuickSearch GUI, mirroring run.sh.
|
||||||
|
rem Terminal search is a separate binary on Windows, because the GUI is built
|
||||||
|
rem as a window-subsystem app and cannot write to the calling shell:
|
||||||
|
rem target\release\quicksearch-cli.exe --help
|
||||||
|
setlocal
|
||||||
|
cargo build --release -p quicksearch-gui
|
||||||
|
if errorlevel 1 exit /b 1
|
||||||
|
rem %~dp0 is this script's own directory (with a trailing backslash), so the
|
||||||
|
rem launch does not depend on the current working directory.
|
||||||
|
"%~dp0target\release\quicksearch.exe" %*
|
||||||
|
|
|
||||||
7
run.sh
|
|
@ -1,2 +1,7 @@
|
||||||
|
#!/usr/bin/env sh
|
||||||
|
# Build and launch the QuickSearch GUI. On Unix the same binary doubles as the
|
||||||
|
# terminal search tool: ./target/release/quicksearch --help
|
||||||
|
# (On Windows that role belongs to quicksearch-cli.exe — see run.bat.)
|
||||||
|
set -e
|
||||||
cargo build --release -p quicksearch-gui
|
cargo build --release -p quicksearch-gui
|
||||||
./target/release/quicksearch
|
exec ./target/release/quicksearch "$@"
|
||||||
|
|
|
||||||
23
setup.sh
|
|
@ -1,23 +0,0 @@
|
||||||
#!/usr/bin/env bash
|
|
||||||
# One-time system setup for building and running the Dioxus-based
|
|
||||||
# `quicksearch-gui` binary on Debian / Ubuntu. The core library
|
|
||||||
# (`quicksearch-core`) has no native system deps and doesn't need this.
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# ./setup.sh # installs everything
|
|
||||||
#
|
|
||||||
# Adds `pkg-config` and `build-essential` explicitly because the Dioxus
|
|
||||||
# build scripts (glib-sys, gdk-sys, gio-sys, javascriptcoregtk-sys,
|
|
||||||
# webkit2gtk-sys) shell out to `pkg-config` and will refuse to build
|
|
||||||
# without it. Some minimal Ubuntu flavors and container images don't
|
|
||||||
# install pkg-config transitively.
|
|
||||||
set -e
|
|
||||||
|
|
||||||
sudo apt update
|
|
||||||
sudo apt install -y \
|
|
||||||
pkg-config \
|
|
||||||
build-essential \
|
|
||||||
libsoup-3.0-dev \
|
|
||||||
libjavascriptcoregtk-4.1-dev \
|
|
||||||
libwebkit2gtk-4.1-dev \
|
|
||||||
libxdo-dev
|
|
||||||