Changes

own shared shareable database implementation, and configured NSS to use it.

NSS will try to find a shared database library named librdb.so (rdb.dll on Windows) in its path and load it. This shared library is expected to implement a superset of the old dbm interface. The main entry point is rdbopen, which will be passed the appName, database name, and open flags. The rdb shared library will pick a location or method to store the database (it may not necessarily be a file), then handle the raw db records from NSS. The records passed to and from this library use exactly the same schema and record formats as the records in the DBM library.

does not have integrity checks and pkcs12 has not no definition for storing purely mac MAC data,the shared DB database integrity checks uses use pkcs5 v2 to store that pbe PBE and mac MAC data.

In our previous database, we had issues with database corruption resulting in hard to diagnose issues. In order to mitigate that, This this section analyses analyzes how various forms of corruption can affect the new database design, and possible ways of repairing that corruption in the field.

CKO_NSS_SMIME object holds the email address, subject of the S/MIME certificate, and the s/mime profile. In the legacy database, multiple email addresses could hold the same profile data and certificate subject, but only one s/mime profile data and subject would be allowed for each email address. In the new database, multiple independent email records can exist for the same email address. While S/MIME will function without CKO_NSS_SMIME objects, Certificates that verify multiple email address can not be found by the email addresses (other then the 'primary' address) without CKO_NSS_SMIME objects. If S/MIME records are corrupted, the Cerificates Certificates will not be findable for other email addresses. Unlike the legacy database, these records are destroyed by S/MIME records for other certificates with the same email address. Now if you have multiple certificates with the same email address, all those certificates can be found. This corruption can be repaired with a PKCS #11 level tool.

==== Accessing the shared shareable Database ====

'''dbm:'''''directory'' opens an existing non-shared shareable libdbm version 8 database.<br/>'''sql:'''''directory1''[''':'''''directory2''] opens a shared shareable database,

This design assumes that new NSS init functions will be defined for applications wanting to do 'standard user initialization', rather than building special knowledge into softoken or the database model. Note: This is different from the 2001 design, or and earlier prototype shared shareable database, where the database code knew the location of the shared shareable database.

In the shared shareable database design, upgrade is no longer mandatory. Applications may choose to continue to use the old DBM database, update to use the new shared shareable database from old DBM databases, or update and merge old DBM database into a new location shared shareable by multiple apps. There is still a desire for this update to be automatic, at least as far as the application user is concerned. The following describe how NSS deals with update in different applications, and what the different applications must do to get the correct update behavior.

in the shared shareable database. In this Mode, the nssdbm3 shared library must be

Mode 2A Applications can also continue to call traditional NSS_Initialize() functions. The should, however, prepend the string "sql:" to the directory path passed to NSS in the configdir parameter. If the sql databases do not exist, NSS will automatically update any old DBM databases in the config directory to shared shareable databases. Like the upgrade from cert7 to cert8, if the update does not work, the app will open and use the old DBM database. Upgrade will not happen if

# shutdown NSS and initialize it only with the new shared shareable database.

Exception B. Applications needs to decide what happens if the new shared shareable DB

# force NSS to reset the shared shareable database password, throwing away private keys and saved passwords, and trust information from the shared shareable DB.

in that legacy database, and so the new shared shareable database password

it's in the same directory that the shared shareable database lives in. NSS opens both

password to update the new shared shareable database with the records from the old.

shared shareable database. If the user never supplies a password, NSS will continue totreat the new shared shareable database as uninitialized and will attempt to update from

open shared shareable DB

< is open shared DB shareable > yes < DB initialized? >-------> done

update (and use) shared shareable DB

update (and use) shared shareable DB

start, the new shared shareable database will already be initialized with it's own

since it's a an application private directory compared the the multiple

# The shared database does not have a master password set and The the legacy database for Mozilla app does a master password set.

the database to a shared shareable database? In order to complete this we will need

the data that was protected by it). We need to acquire the shared shareable db's password so we can encrypt and mac MAC the data properly. If the shared shareable db doesn't

user. If the shared shareable db has the same password as the legacy db, then we can

If both of these fail, we prompt for the password for the shared shareable database. This

If the user has a master password set on his shared shareable database, but does not

2) reset the password on the shared shareable database (loosing losing all private and secret keys, possibly loosing losing some trust).

4) run with the new shared shareable database and allow the user to update later.

It seems pretty unlikely that the user truly does not know the shared shareable database

Shared databases, in general, means mean that some of the current semantics of user

While not necessarily a feature of shared databaseshareable databases, it is an important tool for successful shared shareable database deployments.

Characteristic 3 allows database merge to work on arbitrary database types. You can merge a shared shareable db into a shared shareable db as well as an old datase database into a shared shareable db (in fact, to a point, on arbitrary tokens - you can merge a hardware token into a shared shareable db as long as the keys are extractable).

* private is a pointer to opaque private data specific to the Shared Shareable DB implementation.

NSS will automaticall load the legacy database support under the following conditions: # The application requests that the old databases be loaded (either implicitly or explicitly).# The application request that new databases are loaded, but the new databases do not exist and the old databases do. [[Category:NSS]]   ==== legacy DB support ==== The old dbm code can be supported with the above SDB structure with the following exceptions: # The old db code cannot be extensible (can't dynamically handle new types).# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format. This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys. NSS will automaticall load the legacy database support under the following conditions: # The application requests that the old databases be loaded (either implicitly or explicitly).# The application request that new databases are loaded, but the new databases do not exist and the old databases do. [[Category:NSS]] ==== Layering ==== In order to keep clean separation between the data and database operations, we will continue to maintain an layer between the actual data handling and interpretation and the database itself. The database code will not need to understand: #What objects are actually stored in it.#The types of the attributes.#The meaning of the stored data. Softoken (not the database adapter layer) will manage canonicalizing any CK_ULONGs, encrypting or decrypting private data blobs, checking integrity and deciding what attributes an object should have and setting the appropriate defaults if necessary. Since softoken deals with PKCS #11 templates internally, its interface to the database will be in terms of those templates. The database layer must be multi-thread safe. If the underlying database is not thread safe, sdb_ layer must implement the appropriate locking. ===== s_open ===== The database API consists of an initialization call, which returns an SDB data structure (defined below).  CK_RV s_open(const char *directory, const char *certPrefix, const char *keyPrefix, int cert_version, int key_version, int flags, SDB **certdb, SDB **keydb, int *newInit) The sdb_init function takes: * directory full path to where the database lives.* certPrefix a prefix string to add in front of the key and cert db (if keyPrefix is null), null means add no prefix.* keyPrefix a prefix string to add in front of the key db. Null means use the same prefix as the cert db.* cert_version current version is the current database version* key_version is the current key database version* flags are:** FORCE** READONLY** READ/WRITE/CREATE* certdb is the returned cert SDB structure* keydb is the returned key SDB structure* newInit returns 1 of s_open created new instances of cert and key (used for update). The returned SDB structure has the following format:  typedef struct SDBStr SDB;  struct SDBStr { void *private; void *sdb_app_private; int sdb_type; int sdb_flags; int sdb_version; CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template, int count, SDBFind **find); CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids, int arraySize, int *count); CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find); CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, CK_ATTRIBUTE *template, int count); CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, const CK_ATTRIBUTE *template, int count); CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object, const CK_ATTRIBUTE *template, int count); CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object); CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry); CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry); CK_RV (*sdb_Begin)(SDB *sdb); CK_RV (*sdb_Commit)(SDB *sdb); CK_RV (*sdb_Abort)(SDB *sdb); CK_RV (*sdb_Reset)(SDB *sdb); CK_RV (*sdb_Close)(SDB *sdb); }; where: * private is a pointer to opaque private data specific to the Shared DB implementation.* sdb_type is the type of database (key [aka private] or cert [aka public]).* sdb_flags specifies how the database was opened (ReadOnly, Create, etc).* sdb_version specifies the version of the underlying sdb structure. This allows us to handle future expansion of the sdb data structure safely.* The rest are function pointers to database primitives described next. ===== sdb_FindObjectsInit =====  CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template, int count, SDBFind **find); This function is the equivalent of PKCS #11 C_FindObjectsInit(). It returns a SDBFind context with is opaque to the caller. The caller must call sdb_FindObjectsFinal with this context if sdb_FindobjectsInit succeeds. ===== sdb_FindObjects =====  CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids, int arraySize, int *count); This function is the equivalent of PKCS #11 C_FindObjects(). It takes a SDBFindcontext returned by sdb_FindObjectsInit. This function has the same semantics as C_FindObjects with respect to handling how many objects are returned in a single call. ===== sdb_FindObjectsFinal =====  CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find); This function is the equivalent of PKCS #11 C_FindObjectsFinal(). It frees any resources associated with SDBFIND. ===== sdb_GetAttributeValue =====  CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_GetAttributeValue(). It has the same memory allocation and error code semantics of the PKCS #11 call.The attributes passed to sdb_GetAttributeValues are already transformed fromtheir native representations in the following ways: # CKU_LONG values are stored as 32-bit values in network byte order.# Private attributes will be encrypted. ===== sdb_SetAttributeValue =====  CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, const CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_SetAttributeValue().The attributes returned to sdb_SetAttributeValues are transformed fromtheir native representations in the following ways: # CKU_LONG values returned 32-bit values in network byte order.# Private attributes returned encrypted. ===== sdb_CreateObject =====  CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object, const CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_CreateObject(). The value of 'object' is chosen by the implementer of sdb_CreateObject. This value must be unique for this sdb instance. It should be no more than 30 bits long. ===== sdb_DestroyObject =====  CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object); This function is the equivalent of PKCS #11 C_Destroy object(). It removed the object from the database. ===== sdb_GetPWEntry =====  CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry); Get the password entry. This only applies to the private database. ===== sdb_PutPWEntry =====  CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry); Write the password entry. This only applies to the private database.Writing a password entry will overwrite the old entry. ===== sdb_Begin =====  CK_RV (*sdb_Begin)(SDB *sdb); Begin a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction. ===== sdb_Commit =====  CK_RV (*sdb_Commit)(SDB *sdb); Commit a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. ===== sdb_Abort =====  CK_RV (*sdb_Abort)(SDB *sdb); Abort a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction. ===== sdb_Close =====  CK_RV (*sdb_Close)(SDB *sdb); Close the SDB and free up any resources associated with it. ===== sdb_Reset =====  CK_RV (*sdb_Reset)(SDB *sdb); Reset zeros out the key database and resets the password. ==== legacy DB support ==== The old dbm code can be supported with the above SDB structure with the following exceptions: # The old db code cannot be extensible (can't dynamically handle new types).# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format. This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys. NSS will automaticall load the legacy database support under the following conditions: # The application requests that the old databases be loaded (either implicitly or explicitly).# The application request that new databases are loaded, but the new databases do not exist and the old databases do. [[Category:NSS]]   ==== legacy DB support ==== The old dbm code can be supported with the above SDB structure with the following exceptions: # The old db code cannot be extensible (can't dynamically handle new types).# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format. This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys. NSS will automaticall automatically load the legacy database support under the following conditions: