Content Management System Criteria – Draft 2

What makes a good CMS? The following criteria are being collected from developers, users, and potential users of CMSs. They are being collected in hopes of fostering a common sense of direction among the hundreds of CMSs that are currently in use or under development.

These criteria not only guide the development of new systems but motivate the evolution of existing systems; for example, when adding support for multiple databases, the SQLite support may be more satisfactory if it leverages SQLite's inherent ease of installation and backup. These criteria also help identify useful CMS extensions; for example, Joomla extended with SEO plug-ins may be more satisfactory than Joomla out of the box. Desirable criteria need not be fully consistent; for example, a single-author blog may need a different login and access control mechanism than a community bulletin board.

  1. Easy installation. Just upload the CMS and perhaps run an installation script. This criterion rules out CMSs built on MySQL, as MySQL requires a separate, site-dependent manual installation.
  2. Easy backup and restore. The dynamic portion of the site is stored in a single file or directory that can be downloaded or uploaded via a single command (either an f.t.p. command or a supplied backup/restore script).
  3. Installation in an arbitrary subdirectory. This facilitates testing by allowing several different websites to be installed and tested in parallel. 
  4. Flexible content editing. The CMS provides a general online editing interface for designated content-editable regions. CMSs that incorporate CKEditor probably qualify. Editors that cannot reliably move between visual editing and source code editing probably don't.
  5. SEO-friendly URLs. The CMS provides readable key-word oriented URLs. URLs whose semantics leans heavily on the question mark, for example, index.php?id=534, don't qualify.
  6. SEO meta tag support. Either the CMS's editor supports editing of page headers or the templates include parameters that instantiate description and key-word meta tags.
  7. Well-formatted, readable source code. A customer who looks at the HTML source code should get the impression of a quality product.
  8. Developer support. At a minimum there should be a support forum and a working demo.
  9. Flexible, readable templates/themes. Are the themes PHP scripts? Alternatively, is there an easy-to-learn, well-explained custom templating language? This criterion is not intended as a feature description. It covers the ability to instantiate templates with HTML snippets but does not specify a specific method of instantiation and does not exclude the possibility of more complex mechanisms such as those found in SMARTY.
  10. Flexible content. Different pages can have different themes/templates, different CSS style sheets, and different JavaScript includes.
  11. Support for particular functions. Supported functions may include image galleries, e-commerce shopping carts, an online user community, news items, blogs, unlimited static pages. This requirement is completely open ended.

 

Posted in Uncategorized | 3 Comments

A Login Plug-in

The login plug-in is a login script written as a drop-in plug-in — something that can simply be dropped into a project. It is intended as a building block for configuration management systems. This overview begins with a brief presentation of design goals, then gives recipes for using the login plug-in, and finishes with points of interest regarding the implementation.

Before reading further, you may wish to view the login demo. And after reading the documentation, you may wish to download the complete zip archive.

Design Goals

Intended use. There may be multiple users of the login mechanism. Such users do not register themselves but are introduced to the system by a user with administrative privileges. This distinction implies the existence of user roles.

Convenience versus security. A user whose computer is in a physically secure area has the option of having his browser remember the password. If a user forgets his password, the system mails it to him.

Self containment. The fact that this is a drop-in plug-in implies that the plug-in itself creates and manages the user database. Of course, there must still be page-specific PHP code for pages that respond to whether a user is logged in.

Simplicity. There is no configuration file. The initial user account has user id "admin" and password "easy". The name of the landing page is assumed to be index.php. And the name of the login page is is login/index.php.

How to use the Login Mechanism

From the landing page, make your way to the login page by adding /login to the URL. Log in, returning yourself to the landing page; it will now have additional information. This additional information allows you to get to privileged pages that are available only to logged in users. The whole process is as pictured in the following diagram:

Login Process

The first use of the login mechanism is to set up user accounts and initial passwords via the initial admin user account and the identity management page.

How to implement the Login Plug-in

Place the plug-in's login folder in the same folder as your landing page.

There are two recipes for responding to whether a user is logged in, one for pages that are viewable only by logged-in users, and one for pages that differ for logged in users. On each page that is to be accessible only to logged in users, place the following at the beginning of the file, where ../index.php is a URL for the landing page:

<?php
   session_start();
   if (empty($_SESSION['userId'])) {
      header('Location: ../index.php');
   }
?>

On pages that are to differ for logged in users, bracket portions that are only for logged in users as follows:

<?php
   session_start(); // Only say this once
   if (isset($_SESSION['userId']))
      echo <<<EOT
         // material for the logged in user
EOT
?>

In this design, user roles are numbered from 0 to 15, with 15 being the admin role, and 0 being the disabled role for users that are no longer allowed to log in. The use of roles 1 – 14 is optional and is independent of the login mechanism. The actual mechanism for distinguishing among user roles is similar to that for distinguishing among users:

<?php
   session_start(); // Only say this once
   if (isset($_SESSION['role'] &&
      $_SESSION['role'] >= n))
      echo <<<EOT
         // material for logged in users in roles  >= n
EOT
?>

Implementation Points of Interest

This plug-in cannot be constructed atop a MySQL database, as MySQL requires separate site-dependent, manual installation steps. Two kinds of databases that could be used for this application are SQLite and the NoSQL database, SDB. Both of these include functions for creating the initial database. This implementation uses an ad hoc database that provided the inspiration for SDB. If having a well-accepted underlying database is important, I suggest Chad Smith's mechanism based on SQLite3 with PDO.

A useful feature of MySQL is that the database is in a place that is hard to get to. At first thought this feature seems to be absent in a drop-in plug-in with an automatically constructed database. However, we have borrowed a technique from Brian R. McBride: We include an .htaccess file that prevents viewers from browsing sensitive files, even if they are in the same directory as other files intended for viewing.

Most of the forms used rely on AJAX rather than the HTML form element, as the former allows a more straightforward implementation. For more on this point, see "Formless" HTML Forms.

Posted in Uncategorized | 4 Comments

“Formless” HTML Forms

Although forms are traditionally created using the HTML form element, there is a new approach that uses AJAX in place of the form element. Long story short: The new approach supports a more logical organization of code, but the auto-fill capabilities of browsers only work with form elements, so there are cases where form element is still needed.

I'll start with the new formless forms, then show how their behavior can be imitated using the HTML form element in place of AJAX, then finally compare the relative merits of the two approaches and consider the possibility of an HTML future without the venerated HTML form element.

Formless forms based on AJAX

Here is the basic layout of a formless form:

<!– start of form –>
   <label class='input-label' id='elem1Label'>text1
      <input type="text" class="input"
         id="elem1"></label><br>
   <!– .  .  . additional form elements .  .  . –>
   <button type="button" class="button"
      onClick="doForm()">SUBMIT</button>
<!– end of form –>

The form is missing its HTML form element, as form submission is handled by the JavaScript doForm function instead.  What the form does have are familiar content-editable elements such as input, select, textarea, and so forth.

The doForm function has three parts: preliminary checks before form submission, the actual submission via AJAX, and post processing after the submission. In this illustration, the preliminary checks just see that the user has provided some sort of input. The actual form submission relies on an AJAX call accomplished via the really simple rsAjax interface.

<script type=text/javascript>
   function doForm() {
      var input1 = getAndCheckValue('elem1')
      // . . . additional element checks . . .
      if (input1 && input2 && . . .)
         rsAjax({
            url: 'formScript.ajax.php',
            data: {
              input1: input1,
              input2: input2,
              .  .  .  .  .
            },
            success: processAjaxResponse
         })
         // formScript.ajax.php processes the form
         // parameters and provides a response.
 
      function processAjaxResponse(response){
         if (!response.error) {
            window.location.assign('someNewPage.php');
            return
         } else {
            // diagnostics upon failure …
         }
      }
   }
</script>

In the interests of full disclosure, there is one additional detail. After the user enters the last input in a form, hitting the return key triggers form submission. To make this happen in a formless form, it is necessary to add to the last input element something like onKeyPress="checkForReturnKey(event)", where the requisite event handler is:

function checkForReturnKey(evt) {
   if (evt.keyCode && evt.keyCode == 13) doForm()
}

Using the HTML form element

We next consider using an HTML form element. When the form is submitted, the server response is an arbitrary HTML page, but if there are user errors, it is necessary to return to the same page in order to provide corrective feedback to the user. So the same page may be displayed in two different ways. We will use the POST method for submitting the form. Since the page is initially displayed using the GET method, this gives us a convenient way of distinguishing the two ways of displaying the page.

The first step after the form is submitted is to invoke server side processing by including the script formScript.form.php, which is basically just the formScript.Ajax.php without the AJAX interface.

In the event that a new page is to be opened, this is done up front in PHP, in order to avoid pointlessly repainting the current page before opening the new one. This is why the response to form submission has to come first, before the form itself.

Diagnostics in case of failure are now handled in PHP rather than JavaScript, and they are typically interleaved with the HTML code.

<?php
    global $_RESPONSE;
    $_RESPONSE['error'] = 'false';
    if ($_SERVER['REQUEST_METHOD'] == 'POST') {
       require('formScript.form.php');
    }
    if (!$_RESPONSE['error'])
       header('Location: ' . someNewPage.php);
?>
   .  .  .  .  .  .
<?php
   if ($_RESPONSE['error'])
     // echo diagnostics upon failure

?>

Preliminary checking of user inputs is now handled by the checkForm() function. The form's action is the default, namely to reload the page via the POST method. The form's input elements now include PHP snippets whose purpose is to remember user input values in the event that there has been an error. And of course the submit button is now of type submit.

<form method="post"
   onsubmit="return checkForm()" class="form">
   <label class='input-label' id='elem1Label'>text1
      <input type="text" class="input" id="elem1"
         name="input1"
         value="<?php if (isset($_POST['input1']))
            echo $_POST['input'];?>">
</label><br>
   <!– .  .  . additional form elements .  .  . –>
   <input value="SUBMIT" type="submit">
</form>

Relative Merits

Client-server separation. The main advantage of the formless form is that there is a clean separation between client-side JavaScript processing and server-side PHP processing. By contrast, the use of the traditional form element requires an intermingling of PHP with HTML and JavaScript.

Temporal order. With formless forms, processing can be laid out in temporal order: (1) the user fills out the form, (2) the "doForm" function checks to see that the form has been filled out and (3) ships the user's inputs to the server, (4) the server responds to user input, and (5) the client responds to the results of the server's processing via the processAjaxResponse function.

By contrast, the approach using the HTML form element begins with Step (4) in which the server responds to the user inputs. Next, we encounter Step (1), where the user fills in the form, followed by Step (2), checking the user's inputs via the checkForm function. Then comes a new step of intermingling user inputs with form elements, followed by Step (3), submitting the form inputs to the server. Step (3) is a little simpler than before, as the input elements to be passed to the server are explicitly marked by the presence of the form element. Finally, Step (5), which provides corrective feedback, is intermingled with the HTML; it's precise location depends on how feedback is delivered.

Format Neutrality. The form element contains in its definition certain formatting characteristics that may need to be neutralized in standards-based browsers. In one application, I was obliged to declare the following special class for use with forms:

<style type="text/css">
   .form { position: relative; left:-32px;
           margin-top: -32px }
</style>
<!–[if IE]>
<style type="text/css">
   .form { position:relative; left: 0px;
           margin-top: -32px }
</style>
<![endif]–>

Password Auto-fill in HTML forms. In the case of a login form, the browser will ask the user's preference for remembering passwords, paint the password field and the previous field with a yellow background, and fill them in from remembered information. This behavior is lost in the case of formless forms. Of course, one can program the formless form to remember passwords, but that's a fair amount of work. Usually, it's easier to just use the HTML form element. An exception would be where the login form has white letters on a dark background — switching to a light yellow background destroys both esthetics and readability. In this exceptional case, formless forms can provide a straightforward end run around the browser's effort to be helpful.

Spurious refresh messages. If you use an HTML form, and the user hits the submit button and then the refresh key (F5 on Windows), the user will see the following wordy message, independently of what page the submit action has displayed:

Confirm Form Resubmission

The page that you're looking for used information that you
entered. Returning to that page might cause any action you took
to be repeated. Do you want to continue?

This cryptic message appears even if the "page that you're looking for" is a completely static HTML page! Happily, formless forms don't carry this burden of unnecessary guilt.

A future without form elements?

The existing conventions for browser auto-fill are jerry-rigged. What's needed is an explicit convention for marking form elements that should be remembered, e.g., value=remember, in the case of address fields, or value=remember-for-page, in the case of user id and password. The question about whether the user wants to remember would come whenever the page is unloaded. Since these suggested conventions work at the element level rather than the form level, they make sense for both "formless" forms and for traditional form elements.

An illustrative example

The login plug-in I wrote uses a traditional HTML form for the actual login, but uses formless forms for a half dozen other forms. The interested reader is invited to download the zip file and have a look at the files to get a first hand view of the relative complexity of the two approaches.

Posted in javaScript Programming, Web Design | 2 Comments

SDB: A Small-Database Language

Abstract

SDB is designed for small online databases, has a JSON/PHP interface, and is implemented in PHP. This article presents the rationale for SDB, provides a link to the SDB files (SDB.zip), and a link to a simple demo. Also presented are the SDB design and an introduction to the SDB implementation.

Overview

Small-business websites typically have very simple databases that are used to perform authentication, maintain webmaster-supplied content, and tailor user experiences. A very simple database language is desirable for such sites in order to support easy creation and maintenance.  A simple design may also allow more efficient implementations. Yet another advantage is that a simple database language can provide an easy introduction to databases for the beginner. The SDB language is designed with these possibilities in mind.

Compared to traditional SQL-like languages, SDB can be much simpler as a result of increased generality and the elimination of unnecessary requirements. Specifically,

  • The SDB design is general in that database objects may be arbitrary JSON values, not just SQL-like tables. Consequently, in a given database application, the chosen structure of database objects can grow organically out of the programming problem at hand. But in order to facilitate comparison with SQL-like languages, SDB does include a design for table‑like objects.
  • There is no requirement to handle objects that are so large they can only be manipulated piecewise. This allows for simpler object-handling methods than is the case with SQL.
  • Unlike SQL, there is no command line interface. The databases live on a web server and interact exclusively with the server's web-integration language (so far, just PHP). Consequently, there is no need for the programming interface to prepare and pass command-line instructions. Hence, there is nothing analogous to SQL injection attacks.
  • As with SQLite, there is no explicit security interface. Placement of database files is at the discretion of the programmer. As a result, site-dependent details of how to access databases can be eliminated.

The above simplifications not withstanding, SDB does draw inspiration from SQL-like languages via an analogy with JSON objects: An SQL-like database is analogous a JSON object that maps identifiers to tables, where each table is characterized by its table rows. For our purposes:

A database is a JSON object implementation residing in permanent storage.

Relationship to Other Languages

There are by now more than a hundred noSQL database systems, and like SDB, several are inspired by JSON. The MongoDB database language replaces JSON objects with the more general and more efficient binary JSON (BSON) objects; this adaptation suggests an evolutionary path for SDB. MongoDB successfully gets beyond the one-size-fits-all mentality of SQL, but is far more ambitious than SDB. Tokyo Cabinet also enjoys some simplicity by not dealing with the internal structure of database objects. CouchDB handles key-value pairs where the values are required to be JSON objects; this is less general than SDB where arbitrary JSON values are allowed.

The Really Simple Database language places a high priority on simplicity. But unlike SDB, its primary interface is with humans via a command line user interface, its data objects are SQL-like tables, and as with SQL, there is not a strong separation between databases and the objects they store.

In contrast to SQL-like languages, SDB has no database schemas, at least not yet. The unmet challenge here is to provide useful schemas without sacrificing simplicity. Currently, there is a proposed notation and semantics for JSON schemas, but it is far from simple.

The SDB Design (PHP version)

$db = new SDatabase($fileName);

If the designated file contains JSON text, it is decoded to create the associative array $db->data, and $fileName is assigned to $db->$fileName. If the database file doesn't already exist, $db->data will be an empty array. For a database that exists only in memory, specify $db = new SDatabase("").

$db->save();

Writes $db->data to the file $db->fileName.

$db->savepoint($filename);

Writes $db->data to the named file. Savepoint files are structurally identical to SDatabase files and may be used to open a database via the new SDatabase method.

$db->release($filename);

Deletes or unlinks the named savepoint file. This operation may also be used to delete the main database file, so it is advisable to use a naming convention that avoids confusion of the main database with its savepoint files.

$db->rollback($filename);

Overwrites the current database by reading in the named savepoint file. Does not change the file name associated with the database. The savepoint and rollback methods may be used to back up and restore memory-only databases. A rollback to an empty file name is equivalent to starting over with a new memory-only database.

unset($db);

Closes the database, does not automatically write out the database. This is needed only in large scripts, as all variables are automatically unset when a PHP script terminates. This statement has a different structure, as it cannot be implemented as a database method.

unset($db->data[$name]);

Analogous to the SQL DROP TABLE statement. The statement succeeds whether $db->data[$name] existed or not.

$db->data[$name] = $value;

Generalizes the SQL CREATE TABLE coupled with INSERT OR REPLACE statements.

$db->data[$k]

Extracts the database object with key $k. 

$fn($db->data), where $fn is a function that maps objects to objects.

Generalizes CREATE VIEW, except that the view is not explicitly part of the database. As a consequence, there is no corresponding notion of DROP VIEW. Also generalizes CREATE VIRTUAL TABLE.

There are a few SQL database constructs that we have deliberately omitted for various reasons. The ANALYZE and PRAGMA statements aren't needed because they address efficiency concerns that are only relevant to large databases.  The ATTACH DATABASE statement isn't needed, as multiple databases may easily be opened and worked on in parallel without conflict. The BEGIN TRANSACTION and END TRANSACTION statements aren't supported due to the lack of a safe and effective file locking mechanism for PHP. The CREATE TRIGGER and DROP TRIGGER statements aren't supported, as these statements rely on a tight integration of table and database constructs. Finally, as already mentioned, there is no provision for database schemas due to the lack of a simple database schema language.

This concludes the presentation of the actual database language. But in order to get a valid comparison with SQL, we will explore the case where $db->data[$k] is an SQL-like table. The challenge is to find a concise table abstraction that is easy to describe and implement, and still allows us to do the kinds of things SQL does with tables. 

SQL-Like Tables

Virtually all SQL table manipulation is row-at-a-time. So we begin by defining a row as an associative array that maps identifiers (i.e., column names) to values. Rows include the kinds of things that might be created in MySQL as follows:

$table = mysqli_query($link, $query);
$row = mysqli_fetch_array($table, MYSQL_ASSOC);

A table is largely defined by its table rows, so our first table operations are:

rows($tbl), the rows of the table $tbl.
table($rows), the table created from a nonempty $rows array.
emptyTable($keys), the empty table with column names given by the $keys array.
columnNames($tbl), column names of the supplied table.

The functions rows and table are inverses of each other, except for the fact that an empty table will still have column names, unlike an empty array of table rows. There is no enforced data typing on columns.

append($tbl, $rows)

The $rows are appended to the rows of $tbl. This mimics the effect of repeated SQL INSERT OR REPLACE INTO statements, one for each row. The SQL UNION construct can also be mimicked as append($tbl1, rows(tbl2)). Row ordering isn't preserved and may need to be restored using the orderBy function. Duplicate rows may need to be removed using the removeDuplicates function. These functions are presented below.

select($tbl, $filter), where $filter is a Boolean function on $tbl rows.

Rows that don't satisfy the filter are removed from $tbl. Mimics the SQL statement, SELECT * from $tbl WHERE $filter. 

sanitize($a, $filter), where $filter is a Boolean function.

Rows that do satisfy the filter are removed from $tbl. Mimics the SQL statement DELETE FROM $tbl WHERE $filter.

orderBy($tbl, $clns) – where $tbl is a table and $clns is either a column name or an array of column names.

The rows in $tbl are sorted into lexicographic order using the column names of $clns. Mimics SQL features such as ORDER BY and COLLATE.

removeDuplicates($tbl);

Eliminates duplicate rows. Mimics the effect of the UNIQUE keyword in SELECT statements. 

selectColumns($tbl, $clns), where $clns is a column name or an array of column names from $tbl.

Removes table columns whose column names are not in the array $clns, and reorders the columns according to the order in $clns. Mimics the effect of SELECT $clns FROM $tbl. 

columnMap($tbl, $re), where $re is an associative array that maps names to expressions.

The columnMap function produces a new table, $nt, whose column names are the keys in $re.  For each row, $row, of $tbl, a corresponding row of $nt is obtained by evaluating the expression values in $re in an expanded context that treats column names in $row as variables with values as specified in $row. This function mimics the functionality of the SQL AS construct.

The columnMap function may be used in several ways:

  • Reorder selected columns by including pairs of the form 'id' => '$id' in $re, where id names the existing column to be included. In this use, columnMap mimics the simpler selectColumns function.
  • Rename selected columns by including pairs of the form 'id1' => '$id2' in $re.
  • Add a new blank column by including a pair of the form $id => NULL in $re.
  • Combine columns. For example, if $tbl contains numeric columns named 'min' and 'max', one can construct a difference column by including the pair 'diff' => '$max - $min' in $re.

Aggregate functions, e.g., sum($col), count.

sum($cln) is the function that, when applied to a table $tbl with a numeric column named $cln, returns the sum of all numbers in that column. In other words, sum($cln)($tbl) is the sum of all numbers in the $cln column of $tbl.

In PHP, 'count' is a reserved word, so we define sCount() to be the function such that sCount()($tbl) is the number of rows in $tbl.

regroup($tbl, $cln, $p), where $cln is a column name or array of column names, and $p is an associative array that maps identifiers to aggregate functions such as sum() or sCount().

First, $tbl is sorted and split into an array of sub-tables: $t1, …, $tn, where within each $ti, $cln has the same value in every row. For each $ti, a new table row, $ri, is constructed: The elements of $ri are pairs of the form $id => $val, where $id is a key in $p, and $val = $p[$id]($ti). Mimics the effect of the SQL GROUP BY construct.

naturalJoin($tbl1, $tbl2)

The result is a table whose rows are built from pairs of rows in $tbl1 and $tbl2. For each row $row1 of $tbl1 and each $row2 of $tbl2, if there are duplicate column names for which $row2 provides a different value than $row one, the pair is discarded; otherwise duplicate values are deleted from $row2 and the remaining values are appended to $row1. Mimics the SQL NATURAL JOIN construct. 

The above list intentionally omits a few table operations. Specifically, no effort has been made to support the ALTER TABLE construct. In addition, we only looked at a couple of aggregate functions and totally omitted any discussion of date-time functions, the latter being readily available in the underlying PHP programming language. For sake of simplicity, there are no methods for maintaining multiple orderings of the same table, and thus nothing directly mimics the SQL REINDEX, INDEX BY, or CREATE INDEX.

Introduction to the Implementation

The SDB database language is implemented as a library consisting of two PHP classes, an SDatabase class for manipulating databases, and an STable class of static functions for manipulating tables. The PHP classes are available in files SDatabase.inc.php and STable.inc.php. These and corresponding test files maybe downloaded in SDB.zip.

SDB errors are communicated by generating exceptions.  For example, the construct new SDatabase can generate the following exceptions:

new SDatabase: JSON error – $json_error in file '$fileName'. // 5 cases
new SDatabase: Corrupted database in file '$fileName'.
new SDatabase: Cannot write to file '$fileName'.

The SDB design takes some advantage of PHP 5.4's support for atomic file operations. It is not possible for two scripts to write a database file at the same time, so database file integrity is preserved. However, file reads are not fully atomic, with the result that it is possible for an in-memory database to be corrupted by a write while in the process of being read in, but such an error is immediately detected due to the fact that most strings are not JSON texts.

Finally, a word about JSON objects: Unlike JavaScript objects, PHP objects serve mainly as instances of PHP classes, and the available support for manipulating PHP objects is much less than what is available for manipulating PHP associative arrays. In recognition of this, the PHP json_decode and json_encode functions allow JSON objects to be implemented as associative arrays rather than as PHP objects. The only sticky point in doing this is that an empty array is then both a JSON object and a JSON array.

A Simple Example

Consider the problem of saving a grocery list onto the web for future use at a grocery store. First, try the grocery list demo, then take a look at how it's implemented. Here is how the saved grocery list is displayed in a web page:

Displaying data from the database

The above PHP code opens the database, extracts the grocery list if present, and dumps it into the web page. The doAjax function isn't shown, but its purpose is simply to send a revised grocery list to the server script, saveGroceries.ajax.php, whose content is as follows:

Saving data to a database

The first two lines bring in support for SDB and for really simple AJAX, an easy AJAX interface based on post requests. The grocery list from the client is examined for errors. If all is well, the database containing the grocery list is opened, a backup is made in case of unexpected errors, the new list is placed into the database, the database is saved, and a success message is returned to the client.

 

Posted in javaScript Programming, Web Design | 4 Comments

Really Simple AJAX

Although one can easily find voluminous tomes on the intricacies of the AJAX protocol, the primary task addressed by AJAX is laughably simple:

Send a message to a web server and deal with the server's response.

This blog post identifies compatible client and server routines, rsAjax and rsInitAjax, whose simplicity of use approximates that of the task at hand. 

Client-Side Interface

A total of four parameters are required on the client side:

data – the message to the server,
url – address of the server script that interprets the message,
response – the server's response to the message,
success – a javaScript function that interprets the server response, if AJAX succeeds.

 There are a few unavoidable details on how these are structured:

  1. Without sacrificing anything important, we may assume that the data and response parameters are JSON objects that will be flattened to character strings during transmission to and from the server.
  2. Execution of the rsAjax routine is suspended while awaiting a response from the server. An optional Boolean async parameter allows execution to continue while the AJAX request is being processed.
  3. The response field is implemented as a parameter to a success callback function. This suits the structure of javaScript and facilitates asynchronous programming.
  4. If a protocol error occurs, a detailed error message is displayed via an alert. An optional error parameter for a second callback function can be used to override this default.
  5. If an error is generated while executing the script, an error message is returned. In a PHP script, the message content is controlled via the PHP set_error_handler function.
  6. If the server script itself detects an error, it is not willy-nilly mixed in with normal return data for the client to sort through, but instead is returned in a response object, which, in this case, may contain the following fields:

data – data generated by the server script,
error – true, if an error was generated,
errorMessage – an optional error message indicating what went wrong.

Server-Side Interface

Due to an implementation decision explained below, the components of the client-side data object will arrive at the server script as components of the PHP $_POST variable.

The server-side response is stored in a global PHP variable, $_RESPONSE. As is customary in PHP implementations of JSON, we use associative arrays to implement JSON objects, not PHP objects. On the server, $_RESPONSE has the following form:

$_RESPONSE = array(
   'data' => HTML or other response data output by the script,
   'error' => error flag,
   'errorMessage' => error message if needed,
   additional user defined fields if needed
)

Implementation Overview

The client-side code for rsAjax is essentially found already in jQuery, it's just a matter of sifting through the available 49 parameters and methods for jQuery.ajax. (I tried implementing rsAjax from first principles as spelled out in Thomas Powell's voluminous Ajax: The Complete Reference but got an implementation that worked only on toy test examples.)

The implementation relies exclusively on AJAX post requests, as these are uniformly superior to AJAX get requests from the standpoint of simplicity. Unlike get requests, post requests avoid issues of URL encodings, and they aren't limited in size. People sometimes claim that get requests are good because they rely on URLs which are bookmarkable, but no, URLs are only bookmarkable if placed in the browser's address bar, which is not where AJAX puts them!

On the server, we have a problem. PHP scripts normally mix PHP statements with HTML code, with the result being implicitly delivered to the client after the PHP statements have been executed, and without being explicitly saved first in a variable such as $_RESPONSE. If we are to directly accommodate such PHP-HTML mixtures, we will be returning HTML rather than JSON to the client, and error data will be mixed with normal message data. Happily, PHP offers a way of collecting accumulated script output via its ob_start function. This function even offers a callback mechanism by which we may specify the processing that will happen after the script is run, allowing us to collect all AJAX related code into one place at the beginning of the script.

The rsAjax implementation and related demo files may be downloaded as rsAjax.zip.

A Simple Demo

Here is a demo of rsAjax that uses the rsAjax interface to send a message to the server and get a return message back. After trying the demo, take a look at the code that makes it happen.  On the client side, the rsAjax interface is called with three of the necessary parameters, and the fourth parameter comes along when the callback function, showResponse, is invoked. This callback looks to see if there was an error and updates the webpage accordingly.

Client-side AJAX interface

On the server side, the code is simpler. The message arrives as $_POST['msg'] and is inspected for error. If all is well, a return message is sent back via template instantiation.

Response to an AJAX request

 

Posted in javaScript Programming | Comments Off

Creating Web Images

Several limitations on web images combine to produce unappealing results. The most stringent is pixel density.  Images displayed on a traditional monitor have 96 pixels per inch, no matter how many mega-pixels were delivered by the camera they came from. This constraint is very different from print photography, where pixel densities are usually at least 300 DPI.  (Loosely put, dots per inch equals pixels per inch.) By the time a picture is rendered in a web browser, there has been a reduction in pixel density, which normally involves pixel averaging, resulting in a slightly blurred, hazy image. Typically, the web version of the image is saved using a default jpg compression of 85%, resulting in major loss of information and the introduction of jpg artifacts; these are most noticeable at the edges of flat surfaces, as around a window molding. A camera with a high megapixel rating may deliver good jpeg images despite information loss, but this defeats jpeg’s mission of delivering small file sizes. Websites that upload images often perform automated resizing, with significant loss of quality.  So it pays avoid such processing by uploading a pre‑cropped, pre-sized photo.  In some cases, the expected size is well advertized, as in the case of a standard 512 px X 400 px real estate photo.  In others, the best way to find out is to take a screen capture of an already posted image. Beyond web constraints, there is the fact that cameras and people see differently, and most of what goes up on the web isn’t coming from professional photographers. People know that buildings have vertical, parallel walls.  Cameras don’t; inside shots taken at eye-level have walls that jut outward towards the ceiling. The eye automatically adjusts light and shadow to achieve an easily understood image.  Cameras don’t; outside shots on a sunny day may have deep unreadable shadows. People automatically de-emphasize distractions of little interest. Cameras don’t, accentuating power lines and other clutter. Indoor shots are commonly taken with macro lenses that tend to scatter light, leaving a thin gray film on the image.  They also tend to cause “fish eye” barrel distortion. Happily, most of the above problems are readily addressed using modern image editors.  The exception is jpeg artifacts whose removal is difficult at best. For this and other reasons, it is much better to start with originals images that have high pixel densities. And when saving images in jpeg format, use the highest available quality setting. Better yet, use .png encodings.

My work on a photo of the raspberry field at Raspberry Knoll Farm illustrates many of the above ideas. Mouse over the numbers below the photos to see the progression from initial shot to final image. All photos are presented at a standard height of 153 px regardless of actual photo size.

 
1. Original photo taken with an 8 megapixel Cannon Power Shot.

Raspberry Knoll Farm

2. Sharpen. Sharpening the chaotic textures of raspberry vines causes some horizontal and vertical "linelets" that are apparent in the next few steps.

Initial Sharpening

3. Crop to de-emphasize utility poles, eliminate farm house, and emphasize the red row signs that identify the various raspberry varieties.

Cropping

4. Apply a semitransparent dark green bucket fill to background trees in order to de‑emphasize a road sign and further deemphasize electric polls and lines.

Semitransparent Bucket Fill

 
5. Adjust color values to improve readability.

Adjusting Color Values

 
6. Selectively sharpen grass and interesting foreground objects.

Selective Sharpening

 
7. De-emphasize Hurricane Irene's effect by making damaged leaves greener.

Deemphasizing Hurricane Irene

8. Apply a general-purpose readability filter, Gimp's "National Geographic" plugin. Also apply selective half-pixel blurring to remove linelets.

Applying National Geographic Filter

1 2 3 4 5 6 7 8

 
Another example is provided by a photo taken at the Coriander Cafe & Country Store in Eastford, Connecticut. All images are presented at a height of 423 px regardless of actual size.

1. Original photo taken with an 8 megapixel Cannon Power Shot.  The photo was taken from above, so vertical lines jut outward at the top.

Coriander Cafe & General Store

 
2. Rotate so that the man leans more towards the woman and so that the umbrella poll is more vertical.

Definition of 'vertical'

 
3. Crop to emphasize umbrella, people, food, and table.

Cropped image

 
4. Extrapolate upper left corner to avoid cropping the umbrella.

Top left corner filled

 
5. Blur background and wash with a warm gray bucket fill; sharpen foreground.

Foreground emphasized

 
6. Adjust color values to improve readability.

Values Adjusted

 
7. Lighten skin color to correct for back lighting of subjects.

Lightened Skin Tones

8. Clarify gold umbrella trim, apply sheer transformation to house columns, recede car and post caps, smooth left table edge, sharpen subjects' hair and foreground.

Final touches

1 2 3 4 5 6 7 8

I'm using the above finished images in my real estate web business, selling my home neighborhood of Chaplin Connecticut in the article Chaplin's Neighborhood.

Posted in Uncategorized, Web Design | 2 Comments

Essential Custom Property Websites

A website whose title and subject is a property is a property website.  This is to be contrasted with an add for a broker, brokerage, or franchise that features a particular property, let’s call these latter sites featured-property websites. There are perhaps a dozen companies selling templates that successfully transform featured-property sites into property sites, simply by putting the home information first.  Such property sites do legitimately support the new paradigm of houses introducing buyers to realtors, as described in my previous post, but they aren’t custom property sites. Natchaug Rivercustom property site goes further, attempting to tell the story of a home and its neighborhood through words and pictures.  Some outstanding examples of custom property sites may be seen at Designs for Homes, whose specialty is British and European rental properties. When a home is sufficiently unusual that finding its appropriate future occupants requires special measures, a custom property site is essential. Typical examples would include a home with its own vineyard and winery, or its own 50-acre horse farm. Unique luxury estates also qualify. As with other providers of custom property sites, J. G. Williams Design welcomes the challenge of working with unusual properties, luxurious or otherwise.  For example, a short walk from the scene pictured on the right, there’s a home that hasn’t been painted in 50 years.  It’s grounds are filled with junk. It’s listing says “Needs work.” And it has been on the market for some time.  A custom property site could help the next owner assess how restoration might proceed, beginning with an estimated cost for removing the junk.  Including the neighborhood in the home’s photo shoot could transform the listing from unsightly to beautiful without misrepresentation.  
Posted in Property Websites, Uncategorized | Tagged , , , | 6 Comments

Property Websites as New Paradigm

Property websites can do a much better job of presenting a home than a traditional real estate listing. But their effectiveness requires attention to content and the use of specialized search engines. Obvious benefits. The main reasons for the increasing popularity of single-property websites are fairly obvious. Owners love them and push their realtors to provide them, no surprise there. Buyers have the opportunity to meaningfully scan new homes before investing the time to physically visit homes they’re serious about. For realtors, this translates to showing about half as many homes before getting a sale. In some cases, the savings can be much greater. A $99,000 short sale in my neighborhood went through more than 135 showings, a lot of work for not much profit. The home’s online listing consisted of a single blurred photo.  A single-property website could have done much to identify buyers and eliminate non-buyers. Deeper reasons. Traditionally, realtors bring buyers to homes.  But just as often these days, homes on the Internet bring buyers to realtors. Websites based on the former paradigm give top billing to an agent or the agent’s brokerage, whereas property websites based on the latter paradigm normally give top billing to the home. This new paradigm provides an opportunity to tell a home’s story, a story that the home’s future owners will buy into and become part of, a story that traditional online real estate listings can’t begin to tell. Telling this story begins with asking the right questions. A couple of examples should make the point. A realtor in my area has a listing for an oceanfront cottage that is half off a tall cliff and held up by studs, I don’t mean pilings. The studs appear to have been pushed askew by the ocean and repeatedly patched. How long before the home falls into the ocean? Has a structural engineer been consulted? Would a sane banker consider this home a sound investment? The owner wants $1.2 million, is he for real? How much of the lot is still available to build on? Obviously, somebody has to answer these questions before the home can sell. What better place to collect answers than on a single-property website? The listing for a woodland lot near my home presumes the new owner will build. Its four roadside pictures were taken in winter. What does the lot look like during the other three seasons? Will the new owners be able to see the neighbors from the back yard? Who will move the stone fence to accommodate the driveway? And what better place to collect answers than a single-property website for the future owners to study at their leisure? Skeptical concerns. All of the major search engines favor well-established sites with many visitors and many incoming links. Google actually “sandboxes” new sites for three-to-six months, by which time a home may well have been sold.  The sandboxing can be evaded by using sub-sites of the realtor’s own site as property sites, but there is still the popularity issue, as property sites need have only one viewer, the home’s new owner. Consequently, the use of specialized real-estate search engines is essential.  These sites generally have their own agendas but usually provide for a “property site” link or “listing site” link that can be used to advertise a home’s property website. Another concern is the problem of what to do with a website once its home has been sold. One possibility is to include the website as part of the home. The busy new owners will be grateful for pictures to share with friends and relatives. And if the new owners are blog-inclined, the property site can be the start of a new family website. A third concern is that putting the street address in a URL may be premature from a sales perspective. URLs such as “secludedWoodedLot.com” or “breathTakingOceanfrontCottege.com“, may be preferable, though perhaps not if the website is destined to become a family blog. Conclusion. Single-property sites represent a powerful new paradigm whose effectiveness depends on how it is applied. For more ideas on this new paradigm, see jgwilliamsdesign.com.  
Posted in Property Websites | 7 Comments

Where to Find Stuff in WordPress 2010

  I’ve been getting acquainted with Word­Press; if you’re new to Word­Press too, this might help. The undo button. There doesn’t seem to be any, even though the Word­Press database keeps lots of backed up versions.  Preview your site frequently. Do your more serious editing in an external editor, and watch out for unintended line breaks. The uninstall/reinstall button.  There doesn’t seem to be any from Word­Press, and it’s surprisingly easy to totally trash the site. Unzip an extra backup copy of Word­Press for restoring individual files. Worst case, your ISP may have a repair/reinstall button for Word­Press. If not, you can use the uninstall procedures at tipsandtricks-hq.com and reinstall. Support for SEO meta tags. There isn’t any, but that may not be important.  The best SEO defense is a well-organized website; see Jim Westergren’s blog regarding SEO. Alternatively, there’s lots of downloadable SEO packs.

Here’s where to find specific site content, listed top-down and left-to-right on your main WordPress page:

Site Info Location of Content and Formatting
Background Color/Image Content: (lower left) Appearance → Background Format: (lower left) Appearance → Background
Site Title Content: (bottom left) Settings → General Format: (lower left) Appearance → Editor  → (lower right) style.css → #site-title a {
Site Description Content: (bottom left) Settings → General Format: (lower left) Appearance → Editor (lower right) → style.css (bottom of file) .site-description { ….
Banner Image Content: (lower left) Appearance → Header Format: edit to 940 × 198 pixels first
Menu Bar below Banner Content: (lower left) Appearance → Menus → Custom links Format: Appearance → Editor → #access a { ….
Blog Post Title Content: (upper left) Posts … Format: (lower left) Appearance → Editor → .entry-title
Blog Post Text Content: (upper left) Posts → Add New Format: (lower left) Appearance → Editor → .entry-content, e.g.
Right Sidebar Titles Content: (left) Appearance → Widgets Format: (lower left) Appearance → Editor → .widget-container h3.widget-title
Right Sidebar Links Content: (left) Appearance → Widgets Format: (lower left) Appearance → Editor → .widget-container li a { ….
Right Sidebar Categories Content: (upper left) Posts → Categories → Name Format: (left) Appearance → Widgets → (lower right) Categories
About yourself Content: (far upper right) yourself → About Yourself
Leave a Comment Format: (lower left) Appearance → Editor → .comments-link
Finally, I would be interested to know if someone could tell me whether buying ThemeDreamer reduces the need for the above kinds of information.
Posted in Uncategorized, WordPress | 4 Comments