AllHands/BugzillaMetrics
From MozillaWiki
< AllHands
Last year, dmose asked for various metrics with respect to outstanding patches in Bugzilla; here are the results obtained in mid-December 2005.
How many [patches] are there with what distributions of age / patch size / module?
- How many outstanding patches?
SELECT count(attach_id) as count FROM attachments INNER JOIN bugs ON attachments.bug_id = bugs.bug_id WHERE ispatch = 1 AND isobsolete = 0 AND resolution = "";
7713
- Distribution of age?
SELECT (((YEAR(attachments.creation_ts) * 12) + MONTH(attachments.creation_ts)) / 12) AS period, count(bugs.bug_id) AS count FROM attachments INNER JOIN bugs ON attachments.bug_id = bugs.bug_id WHERE ispatch = 1 AND isobsolete = 0 AND resolution = "" GROUP BY period;
period | count |
---|---|
1999.75 | 7 |
1999.83 | 2 |
1999.92 | 2 |
2000.00 | 3 |
2000.08 | 2 |
2000.17 | 9 |
2000.25 | 14 |
2000.33 | 4 |
2000.42 | 2 |
2000.50 | 24 |
2000.58 | 19 |
2000.67 | 26 |
2000.75 | 51 |
2000.83 | 53 |
2000.92 | 33 |
2001.00 | 48 |
2001.08 | 66 |
2001.17 | 65 |
2001.25 | 76 |
2001.33 | 89 |
2001.42 | 129 |
2001.50 | 95 |
2001.58 | 93 |
2001.67 | 91 |
2001.75 | 71 |
2001.83 | 120 |
2001.92 | 71 |
2002.00 | 91 |
2002.08 | 112 |
2002.17 | 122 |
2002.25 | 100 |
2002.33 | 98 |
2002.42 | 70 |
2002.50 | 79 |
2002.58 | 108 |
2002.67 | 111 |
2002.75 | 81 |
2002.83 | 100 |
2002.92 | 88 |
2003.00 | 83 |
2003.08 | 77 |
2003.17 | 57 |
2003.25 | 86 |
2003.33 | 103 |
2003.42 | 102 |
2003.50 | 105 |
2003.58 | 94 |
2003.67 | 74 |
2003.75 | 73 |
2003.83 | 77 |
2003.92 | 69 |
2004.00 | 71 |
2004.08 | 77 |
2004.17 | 111 |
2004.25 | 128 |
2004.33 | 117 |
2004.42 | 120 |
2004.50 | 93 |
2004.58 | 112 |
2004.67 | 123 |
2004.75 | 131 |
2004.83 | 156 |
2004.92 | 128 |
2005.00 | 123 |
2005.08 | 168 |
2005.17 | 141 |
2005.25 | 227 |
2005.33 | 161 |
2005.42 | 153 |
2005.50 | 153 |
2005.58 | 226 |
2005.67 | 260 |
2005.75 | 286 |
2005.83 | 349 |
2005.92 | 502 |
2006.00 | 172 |
- Distribution of size?
SELECT TRUNCATE(LENGTH(thedata) / 1000, 0) AS sizek, count(attach_id) AS count FROM attachments INNER JOIN bugs ON attachments.bug_id = bugs.bug_id WHERE ispatch = 1 AND isobsolete = 0 AND resolution = "" GROUP BY sizek; (b.m.o. version; it would be different on a tip install)
1k divisions:
sizek | count |
---|---|
0 | 1159 |
1 | 1433 |
2 | 866 |
3 | 588 |
4 | 420 |
5 | 319 |
6 | 257 |
7 | 228 |
8 | 196 |
9 | 162 |
10 | 135 |
11 | 127 |
12 | 120 |
13 | 99 |
14 | 94 |
15 | 79 |
16 | 63 |
17 | 73 |
18 | 55 |
19 | 51 |
20 | 39 |
21 | 47 |
22 | 54 |
23 | 46 |
24 | 46 |
25 | 40 |
26 | 39 |
27 | 27 |
28 | 27 |
29 | 35 |
30 | 21 |
31 | 24 |
32 | 19 |
33 | 38 |
34 | 20 |
35 | 15 |
36 | 23 |
37 | 8 |
38 | 16 |
39 | 9 |
40 | 13 |
41 | 13 |
42 | 18 |
43 | 20 |
44 | 15 |
45 | 16 |
46 | 10 |
47 | 17 |
48 | 12 |
49 | 8 |
50 | 6 |
51 | 10 |
52 | 7 |
53 | 8 |
54 | 6 |
55 | 10 |
56 | 10 |
57 | 8 |
58 | 6 |
59 | 13 |
60 | 3 |
61 | 8 |
62 | 8 |
63 | 12 |
64 | 6 |
65 | 8 |
66 | 4 |
67 | 4 |
68 | 5 |
69 | 6 |
70 | 2 |
71 | 7 |
72 | 6 |
73 | 2 |
74 | 3 |
75 | 4 |
76 | 2 |
77 | 2 |
78 | 2 |
79 | 6 |
80 | 5 |
81 | 5 |
82 | 5 |
83 | 5 |
84 | 7 |
85 | 10 |
86 | 5 |
87 | 3 |
88 | 6 |
89 | 4 |
90 | 2 |
91 | 4 |
92 | 1 |
93 | 3 |
94 | 1 |
95 | 2 |
96 | 2 |
98 | 5 |
100 | 1 |
101 | 4 |
102 | 2 |
103 | 1 |
104 | 4 |
105 | 4 |
106 | 1 |
107 | 4 |
108 | 3 |
110 | 5 |
111 | 3 |
112 | 1 |
113 | 1 |
114 | 2 |
115 | 1 |
116 | 5 |
117 | 2 |
118 | 2 |
119 | 1 |
120 | 3 |
122 | 1 |
123 | 1 |
124 | 4 |
125 | 1 |
127 | 2 |
130 | 2 |
131 | 2 |
132 | 4 |
133 | 3 |
134 | 2 |
138 | 1 |
139 | 2 |
140 | 1 |
141 | 1 |
142 | 1 |
143 | 1 |
144 | 2 |
149 | 3 |
150 | 2 |
152 | 3 |
153 | 1 |
157 | 2 |
160 | 1 |
162 | 2 |
163 | 4 |
164 | 3 |
165 | 2 |
166 | 1 |
168 | 3 |
170 | 2 |
171 | 3 |
175 | 2 |
176 | 1 |
177 | 1 |
180 | 1 |
184 | 1 |
185 | 2 |
187 | 1 |
188 | 1 |
189 | 1 |
190 | 1 |
195 | 1 |
199 | 2 |
200 | 1 |
202 | 1 |
203 | 2 |
204 | 2 |
209 | 1 |
212 | 2 |
213 | 1 |
214 | 1 |
218 | 1 |
219 | 3 |
222 | 1 |
224 | 1 |
227 | 1 |
233 | 1 |
234 | 1 |
235 | 1 |
236 | 2 |
237 | 1 |
241 | 1 |
256 | 1 |
259 | 1 |
263 | 2 |
267 | 1 |
271 | 1 |
274 | 1 |
276 | 1 |
282 | 2 |
283 | 1 |
285 | 1 |
289 | 1 |
299 | 1 |
312 | 1 |
316 | 1 |
336 | 1 |
340 | 1 |
344 | 1 |
348 | 1 |
349 | 1 |
353 | 2 |
364 | 1 |
386 | 1 |
389 | 1 |
417 | 1 |
430 | 1 |
434 | 1 |
436 | 1 |
440 | 1 |
449 | 1 |
457 | 1 |
462 | 1 |
466 | 1 |
510 | 1 |
630 | 1 |
665 | 1 |
672 | 1 |
700 | 1 |
707 | 1 |
817 | 1 |
916 | 1 |
5k divisions:
sizek | count |
---|---|
0 | 4466 |
1 | 1162 |
2 | 575 |
3 | 321 |
4 | 232 |
5 | 168 |
6 | 122 |
7 | 71 |
8 | 79 |
9 | 63 |
10 | 37 |
11 | 47 |
12 | 37 |
13 | 27 |
14 | 20 |
15 | 16 |
16 | 27 |
17 | 28 |
18 | 11 |
19 | 9 |
20 | 12 |
21 | 12 |
22 | 12 |
23 | 11 |
24 | 9 |
25 | 3 |
26 | 13 |
27 | 3 |
28 | 6 |
29 | 3 |
30 | 6 |
31 | 2 |
32 | 10 |
33 | 6 |
34 | 5 |
35 | 4 |
36 | 2 |
37 | 5 |
38 | 1 |
39 | 3 |
40 | 6 |
41 | 1 |
42 | 4 |
43 | 4 |
44 | 2 |
45 | 1 |
46 | 2 |
47 | 4 |
48 | 1 |
51 | 2 |
52 | 2 |
53 | 1 |
54 | 2 |
55 | 1 |
56 | 3 |
57 | 2 |
59 | 1 |
62 | 1 |
63 | 1 |
67 | 1 |
68 | 2 |
69 | 2 |
70 | 2 |
72 | 1 |
77 | 2 |
83 | 1 |
86 | 2 |
87 | 1 |
88 | 1 |
89 | 1 |
91 | 1 |
92 | 1 |
93 | 1 |
102 | 1 |
126 | 1 |
133 | 1 |
134 | 1 |
140 | 1 |
141 | 1 |
163 | 1 |
183 | 1 |
- Distribution of module?
SELECT components.name, count(attach_id) AS count FROM attachments INNER JOIN bugs ON attachments.bug_id = bugs.bug_id INNER JOIN components ON bugs.component_id = components.id WHERE ispatch = 1 AND isobsolete = 0 AND resolution = "" GROUP BY component_id ORDER BY count DESC;
name | count |
---|---|
XPCOM | 266 |
XP Toolkit/Widgets | 233 |
MailNews: Main Mail Window | 216 |
Build Config | 192 |
Internationalization | 174 |
General | 141 |
Libraries | 135 |
XP Toolkit/Widgets: XUL | 125 |
JavaScript Engine | 122 |
XP Apps: GUI Features | 121 |
Bugzilla-General | 121 |
Layout | 111 |
NSPR | 110 |
MailNews: Backend | 103 |
Plug-ins | 96 |
Networking | 93 |
General | 90 |
XPConnect | 89 |
Editor | 88 |
MailNews: Composition | 86 |
String | 81 |
XP Apps | 79 |
DOM | 79 |
Creating/Changing Bugs | 77 |
Mail Window Front End | 77 |
ImageLib | 71 |
Sunbird and Calendar-Extension Front End | 70 |
Query/Bug List | 69 |
Build Config | 65 |
Preferences | 62 |
Themes | 60 |
Style System (CSS) | 58 |
User Interface | 57 |
Printing | 57 |
XForms | 56 |
Embedding: APIs | 55 |
Composer | 53 |
Security: UI | 52 |
SVG | 52 |
Administration | 51 |
Bookmarks | 51 |
XP Toolkit/Widgets: Trees | 50 |
File Handling | 50 |
Layout: Misc Code | 49 |
General | 49 |
Layout: Form Controls | 48 |
Bookmarks | 47 |
MailNews: Address Book | 46 |
Other | 45 |
Toolbars | 45 |
GFX | 42 |
Tabbed Browser | 42 |
GFX: Gtk | 41 |
Layout: Tables | 41 |
webmaster@mozilla.org | 40 |
Web Developer | 40 |
HTML: Parser | 40 |
Networking: HTTP | 39 |
MailNews: Account Manager | 39 |
Embedding: Docshell | 38 |
XP Toolkit/Widgets: Menus | 37 |
MailNews: MIME | 37 |
Keyboard: Navigation | 35 |
Event Handling | 35 |
Networking: IMAP | 34 |
Tracking | 34 |
Installer | 34 |
Layout: BiDi Hebrew & Arabic | 33 |
Selection | 33 |
General | 33 |
Embedding: GTK Widget | 33 |
DOM: Level 0 | 32 |
DOM: Events | 29 |
User Accounts | 29 |
XML | 29 |
Email Notifications | 29 |
DOM Inspector | 28 |
Preferences | 28 |
Networking: Cookies | 27 |
MailNews: Filters | 27 |
XUL Widgets | 26 |
Location Bar and Autocomplete | 26 |
LXR | 25 |
Build | 25 |
Layout: View Rendering | 25 |
Build Config | 24 |
Preferences: Backend | 24 |
RDF | 24 |
Places | 24 |
Developers | 24 |
Search | 24 |
Layout: Fonts and Text | 23 |
Tabbed Browser | 23 |
DOM: Core | 23 |
de-AT / German-Austria | 23 |
Web Site | 23 |
Cmd-line Features | 22 |
Sidebar | 22 |
Tools | 22 |
XBL | 22 |
JavaScript Debugger | 22 |
MailNews: Internationalization | 22 |
MailNews: Database | 22 |
XSLT | 21 |
Bonsai | 21 |
Java to XPCOM Bridge | 21 |
GFX: Win32 | 21 |
Networking: News | 21 |
Help | 20 |
Embedding: ActiveX Wrapper | 20 |
Layout: Block and Inline | 20 |
Base | 20 |
Extension/Theme Manager | 19 |
MailNews: Offline | 19 |
xpidl | 19 |
Java APIs to WebShell | 19 |
Address Book | 19 |
Tinderbox | 19 |
Installer: XPInstall Engine | 18 |
ChatZilla | 18 |
Message Compose Window | 18 |
DOM: HTML | 18 |
Networking: Cache | 17 |
Java: Live Connect | 17 |
Profile: BackEnd | 17 |
Help Viewer | 17 |
DOM: Mozilla Extensions | 17 |
XRE Startup | 17 |
Mozbot | 17 |
Image: GFX | 16 |
Location Bar | 16 |
Drag and Drop | 15 |
Page Info | 15 |
Download Manager | 15 |
RSS | 15 |
Security | 15 |
Download Manager | 15 |
Disability Access APIs | 15 |
Lightning | 15 |
Widget: Gtk | 14 |
Attachments & Requests | 14 |
Documentation | 14 |
History: Global | 14 |
Menus | 13 |
Installer | 13 |
Mozilla Developer | 13 |
Web Services | 13 |
Find Toolbar / FastFind | 13 |
Security: CAPS | 13 |
History: Session | 12 |
Software Update | 12 |
Java: OJI | 12 |
Spelling checker | 12 |
X-remote | 12 |
Storage | 12 |
Installation & Upgrading | 12 |
GFX: Mac | 12 |
MailNews: LDAP Integration | 12 |
LDAP C SDK | 12 |
Reporting/Charting | 11 |
Layout: CTL | 11 |
Widget: Mac | 11 |
General | 11 |
Test | 11 |
Embedding: Mac | 10 |
MailNews: Networking | 10 |
Networking: File | 10 |
JavaScript Console | 10 |
MailNews: Security | 10 |
Print Preview | 10 |
Security: PSM | 10 |
DOM to Text Conversion | 10 |
MailNews: Attachments | 10 |
OS Integration | 10 |
DOM: CSSOM | 10 |
Widget: Win32 | 9 |
MailNews: Profile Migration | 9 |
Profile: Manager | 9 |
Networking: FTP | 9 |
Ports: Qt | 9 |
Password Manager | 9 |
Networking: POP | 9 |
Miscellaneous | 8 |
HTML: Form Submission | 8 |
MailNews: Search | 8 |
Bugzilla: Other b.m.o Issues | 8 |
Profile: Roaming | 8 |
Autocomplete | 8 |
Java APIs for DOM | 8 |
Profile: Migration | 7 |
Address Book | 7 |
XP Miscellany | 7 |
MathML | 7 |
Layout: HTML Frames | 7 |
Database | 7 |
Preferences | 7 |
Image Blocking | 7 |
Installer: XPI Packages | 7 |
DOM: Traversal-Range | 7 |
ViewSource | 7 |
MailNews: Simple MAPI | 7 |
Testing Suite | 7 |
Tinderbox Configuration | 7 |
Build Config | 7 |
Networking: SMTP | 6 |
nl-NL / Dutch | 6 |
Core | 6 |
Plugin Finder Service | 6 |
History | 6 |
Reporter | 6 |
Preferences | 6 |
Account Manager | 6 |
LDAP XPCOM SDK | 6 |
Search | 6 |
MailNews: Import | 6 |
Bookmarks | 6 |
Skinability | 6 |
GFX: BeOS | 6 |
Migration | 6 |
XPCOM Registry | 5 |
QuickLaunch (AKA turbo mode) | 5 |
English US | 5 |
MailNews: Palm Sync | 5 |
View Source | 5 |
Page Layout | 5 |
Layout: Floats | 5 |
HTML Form Controls | 5 |
Preferences | 5 |
Widget: Cocoa | 5 |
es-ES / Spanish | 5 |
Accessibility | 5 |
Localization | 5 |
Widget: BeOS | 5 |
Password Manager | 5 |
Printing: Xprint | 5 |
File Handling | 4 |
Disability Access | 4 |
Library | 4 |
Migration | 4 |
he-IL / Hebrew | 4 |
Embedding: Packaging | 4 |
XTF | 4 |
Layout: Canvas | 4 |
libical | 4 |
Bug Import/Export & Moving | 4 |
LDAP Java SDK | 4 |
MailNews: Subscribe | 4 |
MailNews: Notification | 4 |
XP Apps: Autocomplete | 4 |
Widget | 4 |
Trademark Permissions | 4 |
developer.mozilla.org | 4 |
Startup and Profile System | 4 |
History | 3 |
it-IT / Italian | 3 |
Despot | 3 |
MailNews: Localization | 3 |
WebDAV | 3 |
Administration | 3 |
IPC | 3 |
GFX: Thebes | 3 |
Layout: R & A Pos | 3 |
XULRunner | 3 |
Server Operations | 3 |
Talkback Client | 3 |
GFX: OS/2 | 3 |
Sample Code | 3 |
Java-Implemented Plugins | 3 |
JavaScript Console | 3 |
Tinderbox2 | 3 |
Whining | 3 |
RSS Discovery and Preview | 2 |
Toolbars and Toolbar Customization | 2 |
Security: S/MIME | 2 |
Keyboard: Find as you Type | 2 |
Listings | 2 |
libxpical | 2 |
en-GB / English, United Kingdom | 2 |
Build Config | 2 |
Tinderbox3 | 2 |
Build Config | 2 |
CA Certificates | 2 |
tr-TR / Turkish | 2 |
Doctor | 2 |
Help Documentation | 2 |
pl-PL / Polish | 2 |
Embedding: GRE Core | 2 |
DOM: Load and Save | 2 |
cs-CZ / Czech | 2 |
Keyboard Navigation | 2 |
OS Integration | 2 |
Installer: GRE | 2 |
MailNews: Movemail | 2 |
Layout: Images | 2 |
Security | 2 |
Location Bar & Autocomplete | 2 |
Korean | 1 |
General | 1 |
MailNews: Return Receipts | 1 |
Widget: OS/2 | 1 |
Printing | 1 |
MailNews: Search | 1 |
GFX: Xlib | 1 |
Downloading | 1 |
Form Manager | 1 |
Security | 1 |
Viewer App | 1 |
Installer | 1 |
Registration & Management | 1 |
Form Manager | 1 |
Composer CSS Editor | 1 |
CCK | 1 |
Security | 1 |
Installer | 1 |
User Interface | 1 |
MailNews: Printing | 1 |
Compiler | 1 |
CVS Account Request | 1 |
Business Development | 1 |
Drag & Drop | 1 |
nb-NO / Norwegian bokmal | 1 |
Plugin Listings | 1 |
Tabbed Browsing | 1 |
Page Info | 1 |
Tests | 1 |
Product Site | 1 |
Toolbars & Menus | 1 |
SQL | 1 |
- The review page shows the date that the patch was posted to bugzilla, not the date that review was requested. For how many patches do these dates significantly differ (eg by more than two weeks)?
SELECT count(*) from attachments INNER JOIN bugs ON attachments.bug_id = bugs.bug_id INNER JOIN flags on attachments.attach_id = flags.attach_id INNER JOIN flagtypes ON flags.type_id = flagtypes.id WHERE flagtypes.name = 'review' AND ispatch = 1 AND isobsolete = 0 AND resolution = "" AND (TO_DAYS(flags.creation_date) - TO_DAYS(attachments.creation_ts) > 14);
362 (4.7%)
- How man/what percentage of patches have r? set without an email target?
SELECT bugs.bug_id from attachments INNER JOIN bugs ON attachments.bug_id = bugs.bug_id INNER JOIN flags on attachments.attach_id = flags.attach_id INNER JOIN flagtypes ON flags.type_id = flagtypes.id WHERE flagtypes.name = 'review' AND flags.status = "?" AND requestee_id IS NULL AND ispatch = 1 AND isobsolete = 0 AND resolution = "";
71 (0.9%)
279181 |
161018 |
191752 |
232720 |
148726 |
182076 |
175175 |
248448 |
248450 |
182076 |
105960 |
239499 |
211808 |
284279 |
284755 |
284755 |
279035 |
268448 |
289864 |
289864 |
289864 |
214551 |
300421 |
303647 |
306978 |
236721 |
307228 |
171529 |
301945 |
301945 |
310299 |
310299 |
276232 |
315616 |
218746 |
276075 |
282631 |
65493 |
288038 |
314871 |
312537 |
324291 |
304546 |
327124 |
267108 |
328406 |
328407 |
329205 |
45715 |
323783 |
181446 |
314871 |
125995 |
125995 |
330255 |
329205 |
182082 |
282687 |
331102 |
320751 |
327269 |
331366 |
330600 |
330600 |
331198 |
332066 |
332068 |
332069 |
332128 |
332190 |
328628 |
- How many/what percentage of patches have no apparent communication from the reviewer that the review will not be immediately forthcoming?
Can't tell automatically.
- Of patches that do have such communication, how many/what percentage is incorrect (ie the reviewer said "I'll review next week", but that didn't happen)?
Can't tell automatically.
- What percentage have already gone through an iteration of the review process? Of those that have, how many / what percentage have already been reviewed by the same person on whom review is currently waiting?
Can't tell automatically.
- How many/what percentage have review requests set against people who are no longer active in the project?
We have to define "no longer active" as "no logincookies entry" (i.e. not logged in in the past 30 days), because Bugzilla keeps no other information on this question.
SELECT profiles.login_name AS name, count(flags.requestee_id) AS reviews FROM profiles LEFT JOIN logincookies ON profiles.userid = logincookies.userid INNER JOIN flags ON flags.requestee_id = profiles.userid INNER JOIN flagtypes ON flags.type_id = flagtypes.id INNER JOIN attachments ON attachments.attach_id = flags.attach_id INNER JOIN bugs ON attachments.bug_id = bugs.bug_id WHERE logincookies.userid IS NULL AND flagtypes.name = 'review' AND attachments.ispatch = 1 AND attachments.isobsolete = 0 AND bugs.resolution = "" GROUP BY name ORDER BY reviews DESC;
name | reviews |
adamlock@eircom.net | 7 |
beng@google.com | 4 |
hyatt@mozilla.org | 4 |
roland.mainz@nrubsig.org | 4 |
timeless@myrealbox.com | 3 |
ccarlen@mac.com | 3 |
hewitt@formerly-netscape.com.tld | 3 |
leaf@mozilla.org | 3 |
p_ch@verizon.net | 3 |
documentation@bugzilla.bugs | 3 |
beng.bugs@gmail.com | 2 |
john@johnkeiser.com | 2 |
rmegginson0224@aol.com | 2 |
kinmoz@netscape.net | 2 |
jband_mozilla@rattlebrain.com | 2 |
mpgritti@gmail.com | 2 |
imajes@php.net | 1 |
ftang@formerly-netscape.com.tld | 1 |
law@formerly-netscape.com.tld | 1 |
bernd.mielke@gmx.de | 1 |
tara@tequilarista.org | 1 |
namachi@gmail.com | 1 |
samir_bugzilla@yahoo.com | 1 |
jdunn@maine.rr.com | 1 |
janidarshan@yahoo.com | 1 |
mozilla@rjcdb.com | 1 |
braddr@puremagic.com | 1 |
lordpixel@mac.com | 1 |
jrgmorrison@aol.com | 1 |
scc@mozilla.org | 1 |
zack@kde.org | 1 |
erhyuan@pacbell.net | 1 |
john.schneider@agiledelta.com | 1 |
vladimir+bm@vlad1.com | 1 |
waldemar@acm.org | 1 |
limval@yahoo.com | 1 |
me@mollyandgeoff.com | 1 |
browser-china-ab@sun.com | 1 |
harabai@hotmail.com | 1 |
composer@editor.bugs | 1 |
bryner@gmail.com | 1 |
julien.pierre.track@sun.com | 1 |
kevdig@hypersurf.com | 1 |
mconners@mconners.com | 1 |
- How many such inactive review requestees are there?
44
- How many/what percentage are large enough and old enough that they're likely to have bitrotted and no longer be applicable to the current trunk?
See above: the total count is 77, which is not really significant compared to the total of 7713.
- How many/what percentage include complaints about the lack of review?
Can't tell automatically.