User:Rkentjames:SearchAndFilter
Contents
Search, Filter, and Virtual Folders
This page provides my thoughts and references concerning Mailnews (Thunderbird, Seamonkey) issues with filters, including related issues in search and virtual folders.
The Scope Problem
Different search and filter actions work in different scopes, such as offline email, offline email filter, etc. In theory, we tell what scope you are in, and present the usable menu of search terms for your current scope. In practice, there are problems.
IMAP offline versus online
Apparently a year or so ago, IMAP was changed from online to offline filtering. That enabled some features, and killed others. Do we need finer control of this?
Unread filters versus after-the-fact filters
The junk bugs show the issue. We have one scope for offline mail filter - but many properties (such as all of the junk properties) are not valid during the initial filter process, but are valid later for after-the-fact search. If we allow these, then we get invalid results during normal filtering. If we don't allow them, then perfectly reasonable after-the-fact filtering (such as junk status, percent, or origin) are not allowed.
Per account versus per folder
Mail filters before-the-fact are per account, but after-the-fact are per folder.
Virtual Folders
As a long-term goal, I would like for mailnews to allow a mode of operating where all mail viewing is done from virtual folders, and real folders are used for storage management. To allow this requires making virtual folders work just like regular folders. Some things we need are:
- Fix all of the count bugs
- Give virtual folders the same context menus as real folders
- Allow virtual folders to have subfolders
- Allow virtual folders to search other virtual folders
Change visibility in open folders
What do you do in an open virtual folder when something changes in the messages causing messages to be added or removed from the folder?
Currently for junk processing, when junk messages are deleted they are removed immediately from the view.
In virtual folders, typically when message metadata changes affecting the view, nothing happens until the view is reopened. This is different from the handling of junk, and prevents normal search and filters from being used in the same way as junk filters. There is also a whole infrastructure of junk notifications to support this, which can't be scaled to generic filtering. Some people also don't like the way junk filtering messes with the selection (bug 200138).
I think it should work this way. Whenever a user is manually marking messages in an open folder (be it junk, stars, tags, whatever) that would result in the removal of that message from the view, then a style change occurs on the message to allow graying it out, but it remains visible in the view. But if automatic processing happens (for example a message is processed by junk analysis that results in deleting the message from the open view) then it should be deleted immediately unless is is selected. If selected, gray it out. This would allow a search view to be used similar to the existing handling of junk processing.
"gray it out" could use the same styling currently used for the MSG_FLAG_IMAP_DELETED status, which I think is a line through the message. Could we use the same flag? Probably a discussion for the bug (as yet unfiled)
Big Things we need to fix
allow search to operate on multiple scopes
(for example, online IMAP combined with data from the local database). Related to this is the need to expand news search to work with local database items, such as tags and starred.
support threaded views in cross-folder virtual searches
bug 379806: – threaded/grouped-by-sort views unavailable in saved searches across multiple folders
allow reliable metadata storage outside of the mork folder databases
Buglist
bug 66425: [META] Improve mail message filters
Features in existing design
bug 200252: filter action "print message"
bug 229142: Allow search of complete message (ALL headers, as well as body)
bug 128944: [RFE] Message filters should include standard rule "to+received"
bug 256461: Add mail filters to apply arbitrarily style
bug 74430: RFE: Filter Actions: Leave Message On Server, Download Message From Server
bug 242550: Filter criterion "doesn't contain" not found in user defined header fields (IMAP)
bug 135201: Add filter based on signed/encrypted message status
bug 253268: add message read , junk status to search results window
bug 121214: Filter mail on existence (presence OR absence) of any header
bug 397009: A filter will let me tag, but not untag
bug 379806: threaded/grouped-by-sort views unavailable in saved searches across multiple folders
bug 378080: Ability to filter messages on spamassassin score (This is implicitly a request to parse numbers in headers)
bug 188551: Implement Save search as mailview from Advanced Search
bug 297852: Search/Filter and Boolean expressions
bug 116075: Quicksearch should not hide *threads* which contain search results (in threaded view)
bug 212619: Build-in support for filter on reply-to: (as address)
Bugs in existing design
bug 193325: multiple mail check/filtering instances cause mail multiplication
bug 398498: Message filter of "copy and move" generates phantom mail in copy target folder
bug 353036: "Label is None" custom view lists no messages instead of unlabeled/untagged ['No Tag' filter]
bug 222068: Reading filtered mails does not clear "New Mail" icon
bug 404489: filters using as custom mail header "Sender" do not work (matches From instead)
bug 428427: Unread count in virtual folders not updated
bug 372372: Fix search counts for NEW attribute
bug 389098: Automatic junk (spam) detection on messages moved by a filter from IMAP - should move to junk folder directly after move and detecting junk
bug 373967: Custom column added by extension is not displayed properly while in "grouped by sort"
bug 286523: Unable to create saved search folder(virtual folder) properly when "/"(slash) in name. (X.sbd and Y & Y.msf for 'X/Y')
bug 172104: multiple subject headers causes different subject in thread pane than preview pane, defeats subject filtering, should use first subject
Scope-affected features
bug 414179: Add junkpercent support to nsIMsgSearchTerm
bug 217034: Restore capability: "Label"/"Tag" as filter criterion
bug 310359: search All Addresses capability
bug 187768: allow filter of "To or CC" to use "is in Address Book
bug 196036: Add junk mail status to search/filter criteria
bug 11040: Allow filters to control biff UI
bug 118952: new action "custom sound" for message filter
bug 198100: Option to change precedences (order) of junk control and normal filters
bug 223591: Junk classification should execute on each message as it's received, not after all have been fetched
bug 202605: junk mail filter capabilities should be like other filters
bug 389146: "Mark as Junk" using message filters do not train junk-filter
bug 189970: Ability to block certain folders from junk filters
bug 183929: Add "manual" option to filter
bug 11039: Filter outgoing messages (perhaps to use a different Sent/FCC folder)
bug 294632: Filters should be per-Folder, not per-Account
bug 184490: After-the-fact Filters on custom header (IMAP)
bug 129883: Filters Should be Shareable Across Multiple E-mail Accounts
bug 105169: Filter for Attachments
bug 59365: Filters should have triggers
bug 67421: Missing possibility to filter on text in body for IMAP
bug 166842: [RFE] Ability to import and export message filters
bug 79525: Support Sieve
bug 34973: global message filter option
