Autonag, our bugzilla automatic triage bot and alerting system
This page lists all the actions done by the Autonag bot.
Contents
- 1 Introduction
- 2 Rules
- 2.1 With autofix
- 2.1.1 Bug assigned but marked as UNCONFIRMED
- 2.1.2 Regression keyword is missing
- 2.1.3 Remove leave-open keyword on closed bug
- 2.1.4 [meta] in title but not in keywords
- 2.1.5 Update Firefox Status flags for bugs reopened during Nightly cycle
- 2.1.6 Bug with no assignee but a patch landed
- 2.1.7 Close crash bugs with no crashes over 12 weeks
- 2.1.8 Remove stalled keyword on closed bugs
- 2.1.9 Has meta keyword but not [meta] in the bug title
- 2.1.10 Set a Firefox status flag for beta if there is one for nightly and release
- 2.1.11 Increase priority of bugs tracked y Release Managers
- 2.1.12 Add regression keywords to bugs (uses Machine Learning)
- 2.1.13 Move untriaged bug into the correct component (uses machine learning)
- 2.1.14 Copy crash signature from duplicate bugs to main bugs
- 2.1.15 Increase severity of bugs tracked by Release Managers
- 2.1.16 Nag triage owner to set priority on bugs
- 2.2 Without autofix
- 2.2.1 Feature vs regression
- 2.2.2 Inactive bugs with the leave-open keyword set
- 2.2.3 Inactive Meta bugs
- 2.2.4 Low-severity bug but tracked by a release manager
- 2.2.5 Alert for lack of feedback in a bug
- 2.2.6 Bugs with no priority and no activity
- 2.2.7 Bugs with only one or two words in the summary
- 2.2.8 Remind developers about tracked bugs
- 2.2.9 Reporter not answering to Needinfo request
- 2.2.10 Top crashers vs normal severity
- 2.2.11 Identify tracked bugs with a NeedInfo request
- 2.2.12 Tracked bugs untouched for a week
- 2.2.13 Regressed in upcoming release
- 2.2.14 Identify unlanded uplifts to Beta and ESR
- 2.2.15 Bugs with high severity in Firefox::Untriaged
- 2.2.16 Version set but not status_firefox
- 2.2.17 Check release dates on the Wiki
- 2.2.18 Information about bugs landed during Soft Code Freeze
- 2.2.19 Hightlight Fixed bugs in nightly we may want to uplift
- 2.2.20 New user with a Needinfo request
- 2.2.21 Patch not landed
- 2.1 With autofix
Introduction
Every day hundreds of tickets are opened in Bugzilla which track the tasks, defects and enhancements needed for the development of Firefox and other Mozilla projects. Triaging and priotirizing these bugs are an essential part of our development process and we automate part of this process so as to increase our development turn around and improve the quality of our bugs metadata. The set of scripts we use to improve the quality of our bugs, decrease our triaging time and help Release Management ship better quality software is called Autonag.
Initially created as an alerting system (by email) to our triagers, Autonag rules now also make changes to our bugs metadata on Bugzilla so as to fix inconsistencies.
Rules that change automatically some data in Bugzilla (change a priority, needinfo somebody, close a bug…) are called "Rules with Autofix" and those rules are applied once per hour. For now, security bugs aren't touched.
Rules that do not change data but have other results such as generate a report or send an email are called "Rules without Autofix". Rules without autofix are processed once a day (at 2PM CET).
Rules
With autofix
Bug assigned but marked as UNCONFIRMED
Purpose | Mitigate an issue in Bugzilla (bugs reported by new users are not tagged as NEW |
---|---|
Action | Change the status from UNCONFIRMED to ASSIGNED if there is an assignee |
Example | Bug 1495908 |
Code | assignee_but_unconfirmed.py |
Regression keyword is missing
Purpose | Regression keyword is important to differentiate an actual defect |
---|---|
Action | Sets the regression keyword on bugs with the regression-range-wanted keyword is set |
Example | Bug 1461034 |
Code | has_regression_range_no_keyword.py |
Remove leave-open
keyword on closed bug
Purpose | Clean up a mismatch in metadata |
---|---|
Action | If a bug is closed but the leave-open keyword is still set, remove it |
Example | Bug 1382185 |
Code | leave_open.py |
[meta] in title but not in keywords
Purpose | Improve metedata quality |
---|---|
Action | Adds the meta keyword if the bug title starts by [META] |
Example | Bug 1435799 |
Code | meta_summary_missing.py |
Update Firefox Status flags for bugs reopened during Nightly cycle
Purpose | Avoids potential issues for sheriffs and release managers |
---|---|
Action | Set firefox-status flag back from fixed to affected |
Example | Bug 1495962 |
Code | nightly_reopened.py |
Bug with no assignee but a patch landed
Purpose | Attribute unassigned bug to the developer that fixed it |
---|---|
Action | The ASSIGNEE field on the bug is changed from nobody@mozilla.org to the author of the patch that landed in mozilla-central |
Example | Bug 1514338 |
Code | no_assignee.py |
Close crash bugs with no crashes over 12 weeks
Purpose | Reduce the backlog of bugs to check for Release Managers |
---|---|
Action | Crash bugs without any crash reports for more than 12 weeks |
Example | Bug 1470864 |
Code | no_crashes.py |
Remove stalled
keyword on closed bugs
Purpose | Fix inconsistency between metadata and bug status |
---|---|
Action | If a bug is marked as FIXED and also has a stalled keyword set, the keyword is removed |
Example | Bug 1491624 |
Code | stalled.py |
Has meta keyword but not [meta] in the bug title
Purpose | Having [meta] in the bug title helps with quick search results |
---|---|
Action | If a bug has the meta keyword set, [meta] is added to the bug title |
Example | Bug 1257692 |
Code | summary_meta_missing.py |
Set a Firefox status flag for beta if there is one for nightly and release
Purpose | Fix inconsistency in Firefox status flags that can lead to a bug going undetected by Release Managers during the beta cycle |
---|---|
Action | If a status exist for Firefox N-1 and for Firefox N+1, guess a value for Firefox N |
Example | Bug 1500273 |
Code | missing_beta_status.py |
Increase priority of bugs tracked y Release Managers
Purpose | Priotitizes bugs needing an action for shipping quality software |
---|---|
Action | If a bug is tracked by a Release Manager, update the priority (P2 for nightly, P1 for the rest) |
Example | Bug 1515946,1508277,1512493 |
Code | mismatch_priority_tracking_release.py |
Note | There are multiple files mismatch-priority-tracking-*.py, one per channel |
Add regression
keywords to bugs (uses Machine Learning)
Purpose | Surface regressions not filed as such |
---|---|
Action | If a bug is tracked by a Release Manager, update the priority (P2 for nightly, P1 for the rest) |
Example | Bug 1529139 |
Code | regression.py |
Move untriaged bug into the correct component (uses machine learning)
Purpose | Decrease manual triagin time |
---|---|
Action | Uses machine learning to mass move bugs in Firefox::Untriaged into a better component |
Example | Bug 1530316 |
Code | component.py |
Copy crash signature from duplicate bugs to main bugs
Purpose | Crash bugs marked as duplicate of another one may have a different crash signature. We need to consolidate all signatures in the bug where a patch to fix them is being worked on, other wise we may not fix them all |
---|---|
Action | If a crash bug is marked as a duplicate, the signatures it referenced are added to the new bug |
Example | Bug 1517205 |
Code | copy_duplicate_info.py |
Increase severity of bugs tracked by Release Managers
Purpose | Fix inconsistency in metadata |
---|---|
Action | If a bug is tracked for an upcoming release but it's severity field is low, the severity is increased |
Example | Bug 1538966 |
Code | tracked_bad_severity.py |
Nag triage owner to set priority on bugs
Purpose | Bugs with no priority set are harder to prioritize for Release Managers with regards to the release |
---|---|
Action | The triage owner is needinfoed in Bugzilla or nagged via email to set a priority on bugs |
Example | Bug 1527818 |
Code | workflow/no_priority.py |
Without autofix
Feature vs regression
Purpose | Fix inconsistency in bugs with both the regression and feature keywords set |
---|---|
Action | No action yet |
Code | feature_regression.py |
Inactive bugs with the leave-open
keyword set
Purpose | Help identify dead/inactive bugs |
---|---|
Action | Needinfo the triage owner if a bug is inactive but has the leave-open keyword set |
Example | Bug 1367072 |
Code | leave_open_no_activity.py |
Inactive Meta bugs
Purpose | Help identify dead/inactive bugs |
---|---|
Action | Needinfo the triage owner if a meta bug has no activity and no dependencies set |
Code | meta_no_deps_no_activity.py |
Low-severity bug but tracked by a release manager
Purpose | Idenitfy mismatch in metadata, a tracked bug should not have a priotity lower than normal |
---|---|
Action | No action yet |
Code | mismatch-priority-tracking.py |
Alert for lack of feedback in a bug
Purpose | Increase reaction time on bugs prioritized by upper management or release managers |
---|---|
Action | If a bug as an unanswered NeedInfo request from a release manager or a director, send a warning email |
Code | ni_from_manager.py |
Bugs with no priority and no activity
Purpose | Triage owners need to process the backlog |
---|---|
Action | Needinfo the triage owner (only :Overholt for now) |
Example | Bug 1503461 |
Code | ni_triage_owner.py |
Bugs with only one or two words in the summary
Purpose | Make sure the bug is actually useful, a two-words summary often indicates a poor quality bug report |
---|---|
Action | No action yet |
Example | Bug 1512823 |
Code | one_two_word_summary.py |
Remind developers about tracked bugs
Purpose | Make sure that bugs tracked for the next release are addressed |
---|---|
Action | No action yet |
Code | query_creator.py |
Reporter not answering to Needinfo request
Purpose | identify bugs stalled because we need more information fron the reporter to reproduce it |
---|---|
Action | No action yet |
Code | reporter_with_ni.py |
Top crashers vs normal severity
Purpose | Consistency and help getting traction on bugs |
---|---|
Action | No action yet |
Example | Bug 1471692 |
Code | topcrash_bad_severity.py |
Identify tracked bugs with a NeedInfo request
Purpose | Make sure that bugs tracked for a release or nominated for tracking which also have a NeedInfo request do not get stuck |
---|---|
Action | Send an email |
Code | tracked_needinfo.py |
Tracked bugs untouched for a week
Purpose | Identify important bugs for a release which are not being acted upon |
---|---|
Action | Send an email |
Code | tracking.py |
Regressed in upcoming release
Purpose | Identify potential regression |
---|---|
Action | Send an email with the list of bugs marked as non affecting a release but affecting the next one |
Code | unaffected_affected_no_reg.py |
Identify unlanded uplifts to Beta and ESR
Purpose | Make sure that we ship with the bugs that we need |
---|---|
Action | Send an email wih the list of bugs with uplift requests not landed on the affected branch |
Example | Bug 1509394 |
Code | unlanded.py |
Bugs with high severity in Firefox::Untriaged
Purpose | Identify potentially important issues left untriaged |
---|---|
Action | No action yet |
Code | untriage_important_sev.py |
Version
set but not status_firefox
Purpose | The Version value is set automatically by Bugzilla on landing a patch but not the status_firefoxXX |
---|---|
Action | Send an email |
Code | version_affected.py |
Check release dates on the Wiki
Purpose | Verify the consistency of our release dates on wiki.mozilla.org as we use them in automation |
---|---|
Action | Send an email |
Code | ../next_release.py |
Information about bugs landed during Soft Code Freeze
Purpose | List fixed bug with patches which landed in mozilla-central during the soft freeze week. The listing includes patch statistics. |
---|---|
Action | Send an email |
Code | code_freeze_week.py |
Hightlight Fixed bugs in nightly we may want to uplift
Purpose | Identify potentially upliftable bugs. |
---|---|
Action | Send an email with the recent fixes, the affected branches and metadata for prioritization |
Code | missed_uplifts.py |
New user with a Needinfo request
Purpose | Help identify potentially unactionnable bugs per lack of information |
---|---|
Action | Send an email |
Code | newbie_with_ni.py |
Patch not landed
Purpose | Identify bugs with an unlanded r+ patch. Maybe the patch is now obsolete or the bug needs to have a sheriff land the code with the checkin-needed keyword set. |
---|---|
Action | Send an email |
Code | not_landed.py |