Auto-tools/Projects/WebUXPlatform

Overview

The A-Team works on a variety of web applications, either as independent tools or as UIs to existing automation frameworks. Over time, we have experimented with a variety of web frameworks and systems. It now behooves us to take this experience, evaluate the frameworks and other building blocks, and converge on a set that will suffice for the majority of our new projects. Developers should be able to save time by using these blocks as starting points and thus concentrate on the "business logic" of their projects.

In short, the goal here is to exploit past experience to shorten development time of new web applications. However, we need to be flexible and thus shouldn't be enforcing strict policy, as there will be exceptions, and, since technology will continue to progress, we don't want to be locked into anything if at all possible.

Deadline/Deliverables

No deadline, but we would like a usable framework by the end of Q2. It will almost certainly be an on-going project after this, as we find other ways to improve our applications.

ATeam

• mcote

Dependencies

A new brasstacks, with a more recent OS, is strongly desirable, for the simple fact that nginx + WSGI (the most common Python-web server protocol) resists all attempts to be activated.

Major Tasks

• Examine the current web apps on brasstacks, backends and frontends (8-16h)
• Compare/contrast purpose of each tool with its backend and frontend and look for similarities--are some frameworks better/worse for particular types of applications? Create document with findings. (8h)
• Decide on a set of frameworks/modules/libraries for the most popular type of tool. Note that these do not have to work for *all* our future tools; rather, they should be sufficient for the majority. There will always be reasons to use other systems. Write this up with justifications. (8h)
• Document installation and usage of the UX system components, and templates/sample code where applicable (24-40h). The latter should include
  • Simple server code.
  • Examples of db connections from server
  • init.d script
  • nginx config

Notes

Server Architecture

For most apps, a growing consensus is that web.py provides most required functionality for relatively simple web tools. More complex applications might benefit from a heavier framework like Django, which will not be covered here (yet).

The standard described here will be client-side apps, that is, applications that have static HTML, CSS, and JS which use server-side REST APIs to update content. See [1] for a short summary.

The proposed layout is that the index file and other basic content resides at the root, with REST APIs being in a api/ directory. For example, the app 'toolfoo' might be found at http://server/toolfoo/ and its APIs at http://server/toolfoo/api/. Keeping a consistent approach will allow other applications to easily find and hook into our REST APIs should the need arise.

To keep scalability in mind, static content should be hosted by the main web server (nginx) and all dynamic calls passed through WSGI, FastCGI, or another gateway to the server-side app. Example nginx config, assuming static files are in the project's html/ directory and the application is being served on port 7890 (note that the order is important, to ensure the /api/ patch is matched first):

     location /toolfoo/api/ {
         rewrite /toolfoo/api/(.*) /$1 break;
         proxy_pass http://127.0.0.1:7890;
         proxy_redirect off;
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     }

     location ~ ^/toolfoo/(.*)$ {
       alias /path/to/toolfoo/html/$1;
     }

For rapid development, we can use web.py itself with its built-in development web server to serve both the REST API and the static files. However, out of the box, web.py wants all static files in a static/ subdirectory, so we will need to hack around that. The advantage, however, of having the APIs be in a subdirectory off the application root means that everything is agnostic as to the absolute path on the server. All AJAX calls should be to the relative path "api/", so it doesn't matter if the full path is http://localhost:7890/ during development and http://server/toolfoo/ when deployed.