Changes

</pre> ==== Layering ==== In order to keep clean separation between the data and database operations, we will continue to maintain an layer between the actual data handling and interpretation and the database itself. The database code will not need to understand: #What objects are actually stored in it.#The types of the attributes.#The meaning of the stored data. Softoken (not the database adapter layer) will manage canonicalizing any CK_ULONGs, encrypting or decrypting private data blobs, checking integrity and deciding what attributes an object should have and setting the appropriate defaults if necessary. Since softoken deals with PKCS #11 templates internally, its interface to the database will be in terms of those templates. The database layer must be multi-thread safe. If the underlying database is not thread safe, sdb_ layer must implement the appropriate locking. ===== s_open ===== The database API consists of an initialization call, which returns an SDB data structure (defined below).  CK_RV s_open(const char *directory, const char *certPrefix, const char *keyPrefix, int cert_version, int key_version, int flags, SDB **certdb, SDB **keydb, int *newInit) The sdb_init function takes: * directory full path to where the database lives.* certPrefix a prefix string to add in front of the key and cert db (if keyPrefix is null), null means add no prefix.* keyPrefix a prefix string to add in front of the key db. Null means use the same prefix as the cert db.* cert_version current version is the current database version* key_version is the current key database version* flags are:** FORCE** READONLY** READ/WRITE/CREATE* certdb is the returned cert SDB structure* keydb is the returned key SDB structure* newInit returns 1 of s_open created new instances of cert and key (used for update). The returned SDB structure has the following format:  typedef struct SDBStr SDB;  struct SDBStr { void *private; void *sdb_app_private; int sdb_type; int sdb_flags; int sdb_version; CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template, int count, SDBFind **find); CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids, int arraySize, int *count); CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find); CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, CK_ATTRIBUTE *template, int count); CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, const CK_ATTRIBUTE *template, int count); CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object, const CK_ATTRIBUTE *template, int count); CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object); CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry); CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry); CK_RV (*sdb_Begin)(SDB *sdb); CK_RV (*sdb_Commit)(SDB *sdb); CK_RV (*sdb_Abort)(SDB *sdb); CK_RV (*sdb_Reset)(SDB *sdb); CK_RV (*sdb_Close)(SDB *sdb); }; where: * private is a pointer to opaque private data specific to the Shared DB implementation.* sdb_type is the type of database (key [aka private] or cert [aka public]).* sdb_flags specifies how the database was opened (ReadOnly, Create, etc).* sdb_version specifies the version of the underlying sdb structure. This allows us to handle future expansion of the sdb data structure safely.* The rest are function pointers to database primitives described next. ===== sdb_FindObjectsInit =====  CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template, int count, SDBFind **find); This function is the equivalent of PKCS #11 C_FindObjectsInit(). It returns a SDBFind context with is opaque to the caller. The caller must call sdb_FindObjectsFinal with this context if sdb_FindobjectsInit succeeds. ===== sdb_FindObjects =====  CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids, int arraySize, int *count); This function is the equivalent of PKCS #11 C_FindObjects(). It takes a SDBFindcontext returned by sdb_FindObjectsInit. This function has the same semantics as C_FindObjects with respect to handling how many objects are returned in a single call. ===== sdb_FindObjectsFinal =====  CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find); This function is the equivalent of PKCS #11 C_FindObjectsFinal(). It frees any resources associated with SDBFIND. ===== sdb_GetAttributeValue =====  CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_GetAttributeValue(). It has the same memory allocation and error code semantics of the PKCS #11 call.The attributes passed to sdb_GetAttributeValues are already transformed fromtheir native representations in the following ways: # CKU_LONG values are stored as 32-bit values in network byte order.# Private attributes will be encrypted. ===== sdb_SetAttributeValue =====  CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, const CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_SetAttributeValue().The attributes returned to sdb_SetAttributeValues are transformed fromtheir native representations in the following ways: # CKU_LONG values returned 32-bit values in network byte order.# Private attributes returned encrypted. ===== sdb_CreateObject =====  CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object, const CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_CreateObject(). The value of 'object' is chosen by the implementer of sdb_CreateObject. This value must be unique for this sdb instance. It should be no more than 30 bits long. ===== sdb_DestroyObject =====  CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object); This function is the equivalent of PKCS #11 C_Destroy object(). It removed the object from the database. ===== sdb_GetPWEntry =====  CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry); Get the password entry. This only applies to the private database. ===== sdb_PutPWEntry =====  CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry); Write the password entry. This only applies to the private database.Writing a password entry will overwrite the old entry. ===== sdb_Begin =====  CK_RV (*sdb_Begin)(SDB *sdb); Begin a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction. ===== sdb_Commit =====  CK_RV (*sdb_Commit)(SDB *sdb); Commit a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. ===== sdb_Abort =====  CK_RV (*sdb_Abort)(SDB *sdb); Abort a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction. ===== sdb_Close =====  CK_RV (*sdb_Close)(SDB *sdb); Close the SDB and free up any resources associated with it. ===== sdb_Reset =====  CK_RV (*sdb_Reset)(SDB *sdb); Reset zeros out the key database and resets the password. ==== legacy DB support ==== The old dbm code can be supported with the above SDB structure with the following exceptions: # The old db code cannot be extensible (can't dynamically handle new types).# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format. This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys. NSS will automaticall load the legacy database support under the following conditions: # The application requests that the old databases be loaded (either implicitly or explicitly).# The application request that new databases are loaded, but the new databases do not exist and the old databases do. [[Category:NSS]] ==== Layering ==== In order to keep clean separation between the data and database operations, we will continue to maintain an layer between the actual data handling and interpretation and the database itself. The database code will not need to understand: #What objects are actually stored in it.#The types of the attributes.#The meaning of the stored data. Softoken (not the database adapter layer) will manage canonicalizing any CK_ULONGs, encrypting or decrypting private data blobs, checking integrity and deciding what attributes an object should have and setting the appropriate defaults if necessary. Since softoken deals with PKCS #11 templates internally, its interface to the database will be in terms of those templates. The database layer must be multi-thread safe. If the underlying database is not thread safe, sdb_ layer must implement the appropriate locking. ===== s_open ===== The database API consists of an initialization call, which returns an SDB data structure (defined below).  CK_RV s_open(const char *directory, const char *certPrefix, const char *keyPrefix, int cert_version, int key_version, int flags, SDB **certdb, SDB **keydb, int *newInit) The sdb_init function takes: * directory full path to where the database lives.* certPrefix a prefix string to add in front of the key and cert db (if keyPrefix is null), null means add no prefix.* keyPrefix a prefix string to add in front of the key db. Null means use the same prefix as the cert db.* cert_version current version is the current database version* key_version is the current key database version* flags are:** FORCE** READONLY** READ/WRITE/CREATE* certdb is the returned cert SDB structure* keydb is the returned key SDB structure* newInit returns 1 of s_open created new instances of cert and key (used for update). The returned SDB structure has the following format:  typedef struct SDBStr SDB;  struct SDBStr { void *private; void *sdb_app_private; int sdb_type; int sdb_flags; int sdb_version; CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template, int count, SDBFind **find); CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids, int arraySize, int *count); CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find); CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, CK_ATTRIBUTE *template, int count); CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, const CK_ATTRIBUTE *template, int count); CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object, const CK_ATTRIBUTE *template, int count); CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object); CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry); CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry); CK_RV (*sdb_Begin)(SDB *sdb); CK_RV (*sdb_Commit)(SDB *sdb); CK_RV (*sdb_Abort)(SDB *sdb); CK_RV (*sdb_Reset)(SDB *sdb); CK_RV (*sdb_Close)(SDB *sdb); }; where: * private is a pointer to opaque private data specific to the Shared DB implementation.* sdb_type is the type of database (key [aka private] or cert [aka public]).* sdb_flags specifies how the database was opened (ReadOnly, Create, etc).* sdb_version specifies the version of the underlying sdb structure. This allows us to handle future expansion of the sdb data structure safely.* The rest are function pointers to database primitives described next. ===== sdb_FindObjectsInit =====  CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template, int count, SDBFind **find); This function is the equivalent of PKCS #11 C_FindObjectsInit(). It returns a SDBFind context with is opaque to the caller. The caller must call sdb_FindObjectsFinal with this context if sdb_FindobjectsInit succeeds. ===== sdb_FindObjects =====  CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids, int arraySize, int *count); This function is the equivalent of PKCS #11 C_FindObjects(). It takes a SDBFindcontext returned by sdb_FindObjectsInit. This function has the same semantics as C_FindObjects with respect to handling how many objects are returned in a single call. ===== sdb_FindObjectsFinal =====  CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find); This function is the equivalent of PKCS #11 C_FindObjectsFinal(). It frees any resources associated with SDBFIND. ===== sdb_GetAttributeValue =====  CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_GetAttributeValue(). It has the same memory allocation and error code semantics of the PKCS #11 call.The attributes passed to sdb_GetAttributeValues are already transformed fromtheir native representations in the following ways: # CKU_LONG values are stored as 32-bit values in network byte order.# Private attributes will be encrypted. ===== sdb_SetAttributeValue =====  CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object, const CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_SetAttributeValue().The attributes returned to sdb_SetAttributeValues are transformed fromtheir native representations in the following ways: # CKU_LONG values returned 32-bit values in network byte order.# Private attributes returned encrypted. ===== sdb_CreateObject =====  CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object, const CK_ATTRIBUTE *template, int count); This function is the equivalent of PKCS #11 C_CreateObject(). The value of 'object' is chosen by the implementer of sdb_CreateObject. This value must be unique for this sdb instance. It should be no more than 30 bits long. ===== sdb_DestroyObject =====  CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object); This function is the equivalent of PKCS #11 C_Destroy object(). It removed the object from the database. ===== sdb_GetPWEntry =====  CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry); Get the password entry. This only applies to the private database. ===== sdb_PutPWEntry =====  CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry); Write the password entry. This only applies to the private database.Writing a password entry will overwrite the old entry. ===== sdb_Begin =====  CK_RV (*sdb_Begin)(SDB *sdb); Begin a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction. ===== sdb_Commit =====  CK_RV (*sdb_Commit)(SDB *sdb); Commit a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. ===== sdb_Abort =====  CK_RV (*sdb_Abort)(SDB *sdb); Abort a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holdinga transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction. ===== sdb_Close =====  CK_RV (*sdb_Close)(SDB *sdb); Close the SDB and free up any resources associated with it. ===== sdb_Reset =====  CK_RV (*sdb_Reset)(SDB *sdb); Reset zeros out the key database and resets the password. ==== legacy DB support ==== The old dbm code can be supported with the above SDB structure with the following exceptions: # The old db code cannot be extensible (can't dynamically handle new types).# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format. This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys. NSS will automaticall load the legacy database support under the following conditions: # The application requests that the old databases be loaded (either implicitly or explicitly).# The application request that new databases are loaded, but the new databases do not exist and the old databases do. [[Category:NSS]]