CA/Bug Triage

CA Program Bugzilla Dashboards

• CA Inclusion/Update Requests: https://wiki.mozilla.org/CA/Dashboard
• CA Compliance Bugs: https://wiki.mozilla.org/CA/Incident_Dashboard
• Common CA Database (CCADB) Bugs: https://wiki.mozilla.org/CA/CCADB_Dashboard

Bug Triage in Mozilla's CA Certificate Program

Mozilla’s CA Certificate Program governs inclusion of root certificates in Network Security Services (NSS), a set of open source libraries designed to support cross-platform development of security-enabled client and server applications. The NSS root certificate store is not only used in Mozilla products such as the Firefox browser, but is also used by other companies in a variety of products.

The Bugzilla products/components related to the CA Certificate Program are:

• CA Program :: CA Certificate Compliance - Problems found in certificates issued by Certificate Authorities included in the default certificate store.
  • Concerns that are raised about certificates being issued by CAs, and the resulting action items for the CAs.
• CA Program :: CA Certificate Root Program - For Certificate Authorities to file requests asking for their certificates to be included in the default certificate store.
  • Root inclusion/change requests. When approved, the actual code changes are requested via a new Bugzilla Bug in NSS :: CA Certificates Code.
  • EV treatment enablement requests. When approved, the actual code changes are requested via a new Bugzilla Bug for PSM.
  • CA Program-related concerns or action items.
  • Requests to add certs to OneCRL.
• CA Program :: CA Documents - For CA audit statements, when they are not on the auditor's website.
• CA Program :: Common CA Database - For requesting updates to the Common CA Database (CCADB).
• NSS :: CA Certificates Code - For actual code changes to NSS. Kathleen should be the only person filing these bugs on behalf of the CA Program.

The CA Certificate Program deviates from Mozilla's standardized Bugzilla Bug Triage process for bug priorities (P1, P2, P3, P4, P5). Priorities are not used for CA compliance bugs because they do not directly include code changes to Mozilla's release trains or iterations. Priorities are used, however, for tracking CCADB enhancements and for prioritizing root inclusion requests. See https://wiki.mozilla.org/CA/Prioritization.

Compliance Problems and Incidents

To report a concern about certificates being issued by a CA in Mozilla's Program, or their audit statements:

• https://bugzilla.mozilla.org/enter_bug.cgi?product=CA%20Program&component=CA%20Certificate%20Compliance

If the bug concerns CA certificate issuance, then the bug summary should begin with the CA name (followed by a colon and then a space), so that sorting the bugs by Summary will sort the bugs by CA.

Open CA Compliance bugs: https://wiki.mozilla.org/CA/Incident_Dashboard

If the bug concerns audit statements not containing expected information, then the bug summary should begin with auditor's name, so that sorting the bugs by Summary will sort the bugs by auditor name.

Open Auditor Compliance bugs: https://wiki.mozilla.org/CA/Auditor_Compliance

Whiteboard Tags

The whiteboard tags for CA Program :: CA Certificate Compliance include:

• [ca-infosharing] -- For non-incident "lessons learned" and other descriptions of comprehensive steps a CA might take when addressing compliance, or cascading incidents, or to share its compliance-related experiences for the benefit of the ecosystem.
• [ca-compliance] -- For concerns about a CA's certificates failing to comply with Mozilla's CA Certificate Policy and/or the CA/Browser Forum's Baseline Requirements, and it is not considered to be an imminent security concern.
• [auditor-compliance] -- For concerns about an auditor failing to properly detect and report on CA compliance issues that occurred during one or more periods when the CA was audited.
• [ca-revocation-delay] or [leaf-revocation-delay] -- appended after [ca-compliance] whenever a CA fails to abide by the Baseline Requirements' requirement to revoke certificates in a timely fashion.
• [audit-delay] -- appended after [ca-compliance] when a CA is unable to provide audit statements within one year and 3 months of the previous audit period end date.
• [covid-19] -- appended after [ca-compliance], [audit-delay], or [ca-revocation-delay] when delays are due to mandated restrictions regarding COVID-19.

New Whiteboard Tags appended to [ca-compliance] include the following:

• [ca-misissuance] mis-issuance of a CA certificate
• [dv-misissuance] mis-issuance of a DV TLS end-entity certificate
• [ov-misissuance] mis-issuance of an OV TLS end-entity certificate
• [ev-misissuance] mis-issuance of an EV TLS end-entity certificate
• [smime-misissuance] mis-issuance of an end-entity email (S/MIME) certificate
• [crl-failure] failure to provide certificate status via CRL; malformed, expired CRL
• [ocsp-failure] failure to provide certificate status via OCSP; malformed, expired OCSP
• [policy-failure] failure to update CP/CPS annually, failure to comply with practice in CP/CPS, misunderstanding requirements, failed implementation
• [disclosure-failure] failure to disclose an ICA, failure to report revocation of an ICA, non-disclosure-of-EV-sources, miscommunication, poor communication, etc.
• [audit-failure] failure to perform an audit, failure to upload audits, etc.
• [audit-finding] see https://www.ccadb.org/cas/incident-report#audit-incident-reports

Vulnerability and Security Incident Reporting

To report a vulnerability or security incident pertaining to a CA in Mozilla's Program:

• https://bugzilla.mozilla.org/enter_bug.cgi?bug_type=task&component=CA%20Security%20Vulnerability&groups=ca-program-security&product=CA%20Program

Additionally, and not in lieu of the requirement to publicly report incidents as outlined in section 2.4 of Mozilla's Root Store Policy, a CA Operator MUST disclose a serious vulnerability or security incident in Bugzilla as a secure bug in accordance with guidance found on the Vulnerability Disclosure wiki page.

Root Inclusion/Change requests and EV Treatment Enablement Requests

A representative of a CA may begin the process of root inclusion, change, or ev-enablement by filing a Bugzilla Bug as described here:

• https://wiki.mozilla.org/CA/Application_Process

Root Inclusion Requests are prioritized as described here:

• https://wiki.mozilla.org/CA/Prioritization

The whiteboard tags for CA Program :: CA Certificate Root Program are:

• [ca-initial] -- not enough information to begin the Information Verification phase, or not yet assigned to someone to do the Information Verification
• [ca-verifying] -- in Information Verification phase. This is a high-level review to ensure that all required data has been provided in Bugzilla and the CCADB and that the appropriate tests have been run.
• [ca-ready-for-discussion yyyy-mm-dd] -- Information Verification phase is complete. Ready for public discussion and detailed CP/CPS review.
• [ca-in-discussion] -- in discussion in the CCADB public mailing list.
• [ca-cps-review] -- Detailed Review, which requires that the relevant CP/CPS and audit documents be thoroughly reviewed. As a result of such review(s), the CA may be required to update their CP/CPS to become fully aligned with Mozilla's Root Store Policy.
• [ca-discussion-hold] -- discussion on hold, pending CA actions.
• [ca-hold] -- CA's request is on hold, typically because the CA is a super-CA, so all of their subCAs have to achieve inclusion first.
• [ca-pending-approval] -- final notice of intent to approve the CA's request
• [ca-approved] -- request is approved, pending code changes in NSS, also including certs which are in NSS and pending code changes in PSM
• [ca-denied] -- request was denied. Under normal circumstances the CA may submit a new root inclusion request for a new root certificate that fully complies with Mozilla's Root Store policy.

CCADB: To maintain consistency and improve searchability across Bugzilla bugs associated with CCADB cases, please include a standardized CCADB case number tag in the whiteboard field using the following format:

[ccadb-case-NNNNNNN], where NNNNNNN is the CCADB root inclusion case number.

Examples:

Whiteboard: [ca-verifying] [ccadb-case-1943001]

Whiteboard: [ca-cps-review] [ccadb-case-2000456]

(Place the tag following the existing [ca-*] workflow label.)

CA Audit Statement Bugs

• [ca-audits] -- One bug may be created per CA to store audit statements or CP/CPS documents.
  • Link to create ca-audit bug
  • Make sure the bug has the correct product/component for the CA Certificate Program, which is CA Program :: CA Documents
  • Add [ca-audits] to the Whiteboard

CA Program Process or Policy Related Bugs

• [ca-program] -- bugs related to CA Program process, wiki pages, or policy. Note that most CA Program Policy issues are tracked on Github.

Certificate Revocation Related Bugs

• [ca-onecrl] -- bugs related to updating entries in OneCRL. Under normal circumstances a Bugzilla Bug is not needed for this. Rather, the CA should report the revocation via the Common CA Database.
• OneCRL Entries Generated -- bugs for verifying OneCRL entries before they are pushed to production. These bugs are automatically generated from CCADB for standard revocations of intermediate certificates that are reported by CAs. Otherwise these bugs are generated by manually running the tools for specially requested additions to OneCRL.

Common CA Database (CCADB)

• [ccadb-api] -- for requesting API access to the CCADB as per https://github.com/mozilla/CCADB-Tools/tree/master/API_AddUpdateIntermediateCert
• [ccadb-bug] -- for issues or problems using the CCADB.
• All "CA Program : Common CA Database" Bugzilla Bugs that do not have a whiteboard tag are considered to be enhancement requests that will be prioritized and schedule by the CCADB Steering Committee.

The Priority field is used for CCADB Enhancement Requests and bugs as follows:

• P1 - Development in progress
• P2 - Design complete
• P3 - Prioritized
• P4 or not set - To be prioritized and scheduled later

Bugs that unintentionally remove pre-existing functionality or negatively impact CCADB users should have priority over Enhancements, and should be set to P1. Low impact bugs will start at P4 and be considered with ERs. If it's a low LOE bug (less than 4 hours of work) it can go from P4 to P1 without prioritization/design.