SummerOfCode/2013/EnsembleBackendConnector

From MozillaWiki
Jump to: navigation, search

Summary of Google Summer of Code Project Proposal

I’m applying for the ‘Backend Connectors for Ensemble’ Thunderbird address book extension project for Mozilla.

After discussing the task with the project’s mentor, I plan on researching, then implementing possible options to connect the Thunderbird address book extension to external contact storages, specifically LDAP and/or CardDAV.

The overall project goal would be to implement both types of connections; although after doing some initial research as well as from gathering the mentor’s opinion, it has been found that a LDAP connection implementation might be a sizeable task in its own right, as its advanced nature alongside the lack of previous LDAP implementations in the JavaScript ecosystem might produce a more difficult task.

As referenced by the initial project conversations, this project will involve quite a lot of research and exploration, as much of the work will be done on new areas of interest within the programming landscape, which means a sizeable amount of time will be dedicated to the research and planning portions of development.

Therefore, with the project goal being set as an initial implementation of LDAP and/or CardDav connection support, and the potential for such connections hanging on a set of extensive research findings, an initial goal would be to pursue the most effective path forward to produce the most developmental progress, may it be for LDAP support, CardDav support, or both, if the findings prove effective and implementation are within the scope of the Google Summer of Code timeline.

In regards to work on the project I’ve already done, I’ve done some initial research on the LDAP protocol, to better understand the context to which such an implementation might involve. Furthermore, I’ve taken a look at the current codebase found on the Github repo, and from it I’ve forked and contributed a commit that allows the build shell script to be run on a OSX based system (my preferred development platform).

Planned Schedule of Deliverables

June 17th: Coding period begins.
June 17th-(exact date TBD) June 21th: Initial Research (time split between GSoC & finishing other summer course commitment overlap).
June 22nd-July 6th: Continue initial research and begin project planning, develop specific project structures and tasks.
July 6th-20th: Begin implementation tasks, initial code snippets and groundwork.
July 20th-August 10th: Solidify code implementations and begin to see end quality code content.
August 10th-August 30th: Project tasks should be finalizing content, testing, serious project completion submissions.
August 30th-September 16th: Buffer period before ‘pencils down’ timeline for any scheduling extensions.
September 16th-September 27th: ‘Post-pencils down’ period. I would assume this time would be filled with submissions and other non-programming requirements.

Status Reports

Status 1: 01/06/13

After a quick meeting between mentor and the Google Summer of Code student, it was determined that a good first goal would be to set up the project development environment. This would consist of downloading the project dependencies, including the tools, source, and tests that would allow for the building and deployment of the project contents. Upon completion of this task, it would then be beneficial to consider the design of the assorted Ensemble tests, as it is the goal for Ensemble to be test driven during development.

At this time, the source and toolsets required to build and deploy the Ensemble project have been prepared on a clean Ubuntu installation. However, a few tweaks in the setup still need to be adjusted, as there are a few tests that still do not pass. Improvements are being made to address the remaining failed test attempts. The test design has been considered very briefly, as the time available has been prioritized to address the finalization of the development environment setup.

Status 2: 08/06/13

During the most recent meeting, the final project development environment issues were resolved, and the system was able to pass all required Ensemble tests. At this meeting, a list of goals was produced for completion over the next few weeks. These goals were: reading up on CardDAV, Task.js, and Promise, getting an empty test to run in a new test file, set up a local CardDAV server on the machine, and then finally, to develop a basic implementation of a CardDAV connection. As of this status report, CardDAV, Task.js, and Promise were read up, and an empty test has been made and passed within the Ensemble project.

Status 3: 14/06/13

The meeting prior, it was determined that the main goal over the next week(s) would be to build a function that produces a simple, non-secure connection between a CardDAV server and Ensemble. This task would include a function that initiated such a connection, as well as a test to accompany it. Specifically, this would be done using the Radicale local CardDAV server software. This task was accomplished by developing a connection function that produced an OPTIONS HTTP Request towards a parametered server URL, which would return a positive result if the request status was successful (200). However, the function is not fully completed, as it seems to not address a failed connection response in the expected manner (i.e. A case in which it should not return a resolved promise, does). The recent push request needs to be looked over to see if any solutions can be found for these unexpected results, before the next steps can be taken.

Status 4: 23/06/13

This week was a breather week to address external commitments. After the external commitments were dealt with, work was done on improving the previously discussed connection function, attempting to determine the cause of the odd test behaviour. The latest iteration of the connection code was placed onto the repo for further discussion. A calendar was made to assist the scheduling of future meetings. See https://github.com/mikeconley/thunderbird-ensemble/pull/7 for the pull requests current state of discussion, specifically the commit https://github.com/Demelode/thunderbird-ensemble/commit/5a0cee631b58aa1a17b4971133051cb44adcce71 describing the latest change.

Status 5: 04/07/13

Recent developments include redesigning the structure of the connector test suite and a progressive expansion of tests and the features these tests address in the connector. It was determined that the development pattern for the project would be test-driven, and thus any new development would first begin with tests on new features, then working on those features until it passes the expected test. Furthermore, each feature will be developed using a localized Radicale server package, and then generalized into a JS server test suite included in Mozmill. The current feature in development is the ability to create a new Address Book on a designated location on the working CardDAV server, which upon completion would lead to the deletion of Address Books, and then in turn the ability to add new CardDAV objects to an Address Book.

Status 6: 09/07/13

The Address Book creation function is approaching completion, although currently the process has been halted due to an unknown error that has effected all tests. We are going through documentation to understand the reason for this vague error (as it has appeared without any context or apparent cause), and will move forward from there. While this is being dealt with, the groundwork for other functions such as the deletion of address books, and the creation and deletion of contacts have been laid, and are being prepped for after the overarching mystery error has been resolved.

Status 7: 20/07/13

Significant rewrites and project goal realignments have produced many changes since the last report. The testConnection function has received better error protection, and the testing suite has received major upgrades. readRecords has been realigned to new goals, and in turn has been aligned closer to the project goals. Overall the development process has been curated to allow for more efficient production and development, and progress is happening at an always gathering rate. The pull request acceptance has occurred during this period, and a second has been proposed for review.

Status 8: 07/08/13

First off, apologies for a gap of time between this update cycle, a lot has been done and my concentration on the tasks distracted me from posting. Quite honestly, the work done during this period has dwarfed all previous work done on this project, as while before I was working on understanding the software setup and learning our overall connector goals, I've now come to not only understand, but also efficiently and effectively produce the desired project goals at a rapid pace. Overall, I've become quite comfortable with everything involved, and thus the production and development has increased dramatically. Specifically, not only have I almost completely overhauled the previously developed functions such as read and testConnection, but I have (at this point) also worked on the majority (if not all) of the major features and functions within the connector, as well as many external entities such as the RecordCache and the test suite. I've been quite happy with the progress in this period, as when this section of time began, the project was at a point where a function or two were falling into place for initial completion. Yet now, at the closing of this period, we are seeing the connector come together as a fully featured code base. We are not yet done, but the project has developed to a point where the deadlines are very achievable, and if progress continues at the desired pace, potentially beatable.

Status 9: 15/08/13

During the meeting, team members recapped current progress and the state of the code. The next week's goals were proposed; namely to test the connector in its current state into a real world example, and to begin research on pursuing authentication features into the connector.

Status 10: 22/08/13

Authentication was researched, and from the results of this research some initial code was dropped that provided the ability to react to https Basic Authentication -type servers. Working on setting up a local cardDAV emulator Radicale on new machine which would allow for final testing of the connector for real world situations.

Status 11: 30/08/13

Based on the previous week's goal, a new installation of the Radicale CardDAV server was sought, which caused an unexpected handful of issues along the way. After these issues were addressed, we were able to test the CardDAV Connector and its features using Radicale in a real world live setting, which produced a few adjustments in the connector's handling, but otherwise confirmed the quality of the current code-base. Code was then finalized and a pull request was made to be merged into the project master branch.

Status 12: 08/09/13

Code was reviewed, and a few small adjustments were proposed (i.e. trailing whitespace, not defaulting a combobox, etc.) and the changes were addressed. A followup discussion was proposed for the next meeting on the return value of the authorize function. Final pull request was created.

Google Summer of Code Wrap-Up

As stated in the original proposal, the initial goal of the Google Summer of Code project was to develop a backend connector that would allow access to selected protocol-based contact storage servers for the Ensemble Thunderbird Address Book Add-on. Initially, the top two desired protocols were CardDAV and LDAP, and upon further research and discussion, it was chosen that a CardDAV backend connector would be the selected development goal for the summer term.

Upon reaching the 'pencils-down' date, the goal of developing the CardDAV backend connector was met through the use of a rapid, iterative, and test driven design using the JavaScript language alongside the Mozilla comm-central source code. Through the use of the Git revision control system, the project was developed seperate from the main Ensemble code base at https://github.com/Demelode/thunderbird-ensemble/tree/new_master which was in turn merged into the final master branch located at https://github.com/mikeconley/thunderbird-ensemble .

Specifically, the CardDAVConnector file can be found at https://github.com/Demelode/thunderbird-ensemble/blob/new_master/modules/connectors/CardDAVConnector.jsm , and its test suite is located at https://github.com/Demelode/thunderbird-ensemble/blob/new_master/tests/test-carddav-connector.js .

Alongside other additions such as Memory Cache objects https://github.com/Demelode/thunderbird-ensemble/blob/new_master/modules/connectors/MemoryRecordCache.jsm and the modularization of an open sourced VCard Parser https://github.com/Demelode/thunderbird-ensemble/blob/new_master/modules/lib/VCardParser.jsm , a patch was developed to add promptUsernameAndPassword capabilities to test-prompt-helpers.js in the Mozilla central code base, located at https://bugzilla.mozilla.org/show_bug.cgi?id=906520 . This was added because Ensemble will require its use when prompting for CardDAV server authentication during fetching and polling requests.

Now that CardDAV Connector development is finished, the future goals outside of the scope of the Google of Summer Code project will be maintaining the CardDAV Connector for any new changes or requirements as they appear, and the target of developing a LDAP-based version for the overall Ensemble project. Research would suggest that to complete a LDAP-based version of the backend connector, the project would use https://developer.mozilla.org/en-US/docs/Mozilla/js-ctypes to allow the usage of the LDAP C code located in comm-central at the location of http://mxr.mozilla.org/comm-central/source/ldap/sdks/c-sdk/ldap/ .