I've been using a system for searching in my CF powered apps for some years
now, and it works pretty well., but now I've got a bit of a problem with a new
app.

I need to take the results of the various cfsearch statements in the app's
search results page, and do a query to get other data, because I do not want to
go directly to the found "thing", I want to load a metadata display page about
the thing. The app is essentially a document library. I have one verity
collection built from a database query of the metadata, so users can type a
word or phrase in a simple search box and have all the various fields be
searched (label, author, date, brief and full description, etc.). I've given
them an option to do a full-text search, which will go against a collection
indexed by type=path.

The problem is that in order to cfindex the documents, I have to provide the
full path to the documents as the argument to the KEY parameter. But CF
prepends that value to each document's filename, which it returns when searched
with cfsearch. I need to use that KEY to do my metadata query, which means that
I have to loop over the cfsearch results and chop off the path info. That lets
me create a list of those results which I can use in an IN statement of my
metadata query. If more than a few hundred results are found, it's too slow. So
I've added a check on the full-text cfsearch's recordcount, and if it's more
than 100 I tell them to narrow their search or something.

Another issue I've had is degrading performance, in that as collections are
updated, they got slower and slower. I resolved that years ago by having my
scheduled indexing page delete the collections and recreate them. Things may
have changed, I certainly hope so, but I couldn't get PURGE or REFRESH to
suffice. Now I have a new app, with some 1500+ PDF files, some of them 15 or 20
megs large. I cannot index this collection in one action, it times out after
some hours. I might be able to make the timeout something ridiculous, but
that's not going to work. So I am just using UPDATE to index only new additions
as they come in. It still takes a long time, though.

Any suggestions?