Changes

Jump to: navigation, search

DXR Storages

44 bytes added, 22:11, 6 March 2014
m
Tentative Roadmap
#* We could index the pathnames into each line, denormalizing, and always search on lines. That would make those easy to AND together. We don’t even need to mget the files afterward as with parent-child relationships, since every line contains the full path (but not icon or encoding—important? Probably not, if highlighting works, which it should without parent-child.). We could even support search-by-color: just index all the green stuff into an array stored as a separate property of the line. (Highlighting would probably have to be done app-side.)
# Build routine to extract trigrams from regexes. (There is no existing work apparent in Python. We could require re2 and call through to its <code>Prefilter::Info::TakeMatch</code> etc., but it doesn't look too hard to implement or too CPU-intense (when you start from the sre_parse.parse() in stdlib); I'd have to do some work in any case to bridge Python to that C routine; and fewer build steps, git submodules, and build-time checkouts make for a lower contributor support load.) Add trigram indices for lines and switch to a filtered query for regexes. Deploy.
# Get rid of the rest of the on-disk instance, embed necessary region and ref offsets and payloads into the ES index (out of band with the source code), and build pages at request time. Add caching if needed. Something like config.py might still hang around so we don't have to fetch trivial things like WWW_ROOT over a socketand so we know which ES hosts to connect to.
== PostgreSQL ==
Confirm
574
edits

Navigation menu