The Ajax lookup was quick work with jQuery UI’s Autocomplete widget. The harder part was figuring out the most simple table structure and an appropriate SQL query.

A flawed beginning

My people table contains a “first_name” column and a “last_name” column, nothing uncommon there. To get the project out the door, I wrote a PHP function that ran two ALTER TABLE queries on the people table to create two additional columns for pre-formatted strings (column “firstlast”, to be formatted as “Firstname Lastname”, and column “lastfirst”, to be formatted as “Lastname, Firstname”), added indexes on these columns, and then walked through each record in the table, populating these new fields. I then wrote a very straight forward SQL query to perform a lookup on both fields. The PHP and query looked something like this:

// The jQuery UI Autocomplete widget passes the user input as a value for the parameter "name"
$name= $_GET['name'];

// This SQL query uses argument swapping
$query = sprintf("SELECT * FROM people WHERE (`firstlast` LIKE '%1\$s' OR `lastfirst` LIKE '%1\$s') ORDER BY `lastfirst` ASC",
mysql_real_escape_string($name. "%", $link));

This was effective, accurate, and pretty fast, but the addition of columns bothered me and I didn’t like that I needed to run a process to generate those pre-formatted fields each time a record was added to the table (or if a change was made to an existing record). One possible alternative was to watch the input and match either lastname or firstname until the user entered a comma or a space, then explode the string on the comma or space and search more precisely. Once a comma or a space was encountered, I felt pretty sure that I would be able to accurately determine which part of the input was the first name and which was the last name. But this had that same inefficient, clunky bad-code-smell as the extra columns. (Explode is one of those functions that I try to avoid using.) Writing lots of extra PHP didn’t seem necessary or right.

I’m much more comfortable with PHP than with MySQL queries, but I realize that one can do some amazing things within the SQL query, and that it’s probably faster to use SQL to perform some functions. So, I decided that I’d try to work up a query that solved my problem, rather than write more lines of PHP.

CONCAT_WS to the rescue

I Googled around for a bit and settled on using CONCAT_WS to concatenate the first names and last names into a single string be matched, but found it a bit confusing to work with. I kept trying to use it to create an alias, “lastfirst”, and then use the alias in the WHERE clause, which doesn’t work, or I was getting the literal column names back instead of the values. Eventually, I hit upon the correct usage.

The PHP and query now looks like this:

// The jQuery UI Autocomplete widget passes the user input as a value for the parameter "name"
$name= $_GET['name'];

// This SQL query uses argument swapping
$query = sprintf("SELECT *, CONCAT_WS(  ', ',  `last_name`,  `first_name` ) as lastfirst FROM people WHERE (CONCAT_WS(  ', ',  `last_name`,  `first_name` ) LIKE '%1\$s' OR CONCAT_WS(  ' ',  `first_name`,  `last_name` ) LIKE '%1\$s') ORDER BY lastfirst ASC",
mysql_real_escape_string($name. "%", $link));

The first instance of CONCAT_WS isn’t needed for the lookup. The first instance allows me to order the results alphabetically and provides me an array key of “lastfirst” with a value of the person’s name already formatted as “Lastname, Firstname”, so I don’t have to do it later with PHP. The lookup comes from the two instances of CONCAT_WS in the WHERE clause. I haven’t done any performance measuring here, but the results of the lookup get back to the user plenty fast enough, if not just as quickly as the method using dedicated columns.

The result of the query is output back to the page as JSON-formatted data for use in the jQuery Autocomplete.

The end result works exactly as I had hoped. A user of the form is able to type a person’s name in whatever way is comfortable to them, as “Bob Smith” or “Smith, Bob”, and the matches are found either way. The only thing it doesn’t do is output the matches back to the autocompleter in the same format that the user is using. But I can live with that for now.

A history of spam fighting

The most successful and simple method I found was a server-side system for reducing comment spam by using a handshake method involving timestamps on hidden form fields. The general idea was that a bot would submit a comment more quickly than a human visitor, so if the comment was submitted too soon after the page was loaded, it was rejected. A human caught in this trap would be able to click the Back button on the browser to resubmit. This had proven to be very effective on ardamis.com, cutting the number of spam comments intercepted by Akismet per day to nearly zero. For a long time, the only problem was that it required modifying a core WordPress file, wp-comments-post.php. Each time WordPress was updated, the core file was replaced. If I didn’t then go back and make my modifications again, I would lose the spam protection until I made the changes. As it became easier to update WordPress (via the admin panel) and I updated it more frequently, editing the core file became more of a nuisance.

A huge facepalm

When Google began weighting page load times as part of its ranking algorithm, I implemented the WP Super Cache caching plugin on ardamis.com and configured it to use .htaccess and mod_rewrite to serve cache files. Page load times certainly decreased, but the amount of spam detected by Akismet increased. After a while, I realized that this was because the spam bots were submitting comments from static, cached pages, and the timestamps on those pages, which had been generated server-side with PHP, were already minutes old when the page was requested. The form processing script, which normally rejects comments that are submitted too quickly to be written by a human visitor, happily accepted the timestamps. Even worse, a second function of my anti-spam method also rejected comments that were submitted 10 minutes or more after the page was loaded. Of course, most of the visitors were being served cached pages that were already more than 10 minutes old, so even legitimate comments were being rejected. Using PHP to generate my timestamps obviously was not going to work if I wanted to keep serving cached pages.

JavaScript to the rescue

Generating real-time timestamps on cached pages requires JavaScript. But instead of a reliable server clock setting the timestamp, the time is coming from the visitor’s system, which can’t be trusted to be accurate. Merely changing the comment form to use JavaScript to generate the first timestamp wouldn’t work, because verifying a timestamp generated on the client-side against one generated with a server-side language would be disastrous.

Replacing the PHP-generated timestamps with JavaScript-generated timestamps would require substantial changes to the system.

Traditional client-side form validation using JavaScript happens when the form is submitted. If the validation fails, the form is not submitted, and the visitor typically gets an alert with suggestions on how to make the form acceptable. If the validation passes, the form submission continues without bothering the visitor. To get our two timestamps, we can generate a first timestamp when the page loads and compare it to a second timestamp generated when the form is submitted. If the visitor submits the form too quickly, we can display an alert showing the number of seconds remaining until the form can be successfully submitted. This should hopefully be invisible to most visitors who choose to leave comments, but at the very least, far less irritating than a CAPTCHA system.

It took me two tries to get it right, but I’m going to discuss the less successful method first to point out its flaws.

Method One (not good enough)

Here’s how the original system flowed.

1. Generate a first JS timestamp when the page is loaded.
2. Generate a second JS timestamp when the form is submitted.
3. Before the form is submitted, compare the two, and if enough time has passed, write a pre-determined passcode to a hidden INPUT element, then submit the form.
4. On the form processing page, use server-side logic to verify that the passcode is present and valid.

The problem was that it seemed that certain bots could parse JavaScript enough to drop the pre-determined passcode into the hidden form field before submitting the form, circumventing the timestamps completely and defeating the system.

It also failed to adhere to one of the basic tenants of form validation – that the input must be checked on both the client-side and the server-side.

Method Two (better)

Rather than having the server-side validation be merely a check to confirm that the passcode is present, method two goes back to comparing the timestamps a second time on the server side. Instead of a single hidden input, we now have two – one for each timestamp. This is intended to prevent a bot from figuring out the ultimate validation mechanism by simply parsing the JavaScript. Finally, the hidden fields are added to the form via jQuery, which makes it easier to implement and may act as another layer of obfuscation.

1. Generate a first JS timestamp when the page is loaded and write it to a hidden form field.
2. Generate a second JS timestamp when the form is submitted and write it to a hidden form field.
3. Before the form is submitted, compare the two, and if enough time has passed, submit the form (client-side validation).
4. On the form processing page, use server-side logic to compare the timestamps a second time (server-side validation).

The timestamp handshake works more like it did in the server-side-only method. We still have to pass something from the comment form to the processing script, but it’s not too obvious from the HTML what is being done with it.

The same downside plagues me

Unfortunately, if we want to have any server-side validation at all, and we do, the core file wp-comments-post.php will still have to be modified. In my experience, the system is not sufficiently effective using just client-side validation.

The code

Two files must be modified to implement the validation.

File 1: The theme’s comments.php file (older themes) or wp-includes\comment-template.php (newer themes)

Your comment form lives somewhere. My theme is based on Kubrick, the old default WordPress theme, and my comment form is in my theme folder, in a file named comments.php. If your theme is newer and based on the current default theme, twentyeleven, the form is in wp-includes\comment-template.php. If your theme isn’t based on either of these, all bets are off. I know it’s confusing. Sorry.

Add the JavaScript that creates and populates the timestamp fields. Be sure to confirm that your comment form has an ID of commentform. I’m using jQuery to help fire functions when the page loads.

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){
	ardGenTS1();
});

function ardGenTS1() {
	// prepare the form
	$('#commentform').append('<input type="hidden" name="ardTS1" id="ardTS1" value="1" />');
	$('#commentform').append('<input type="hidden" name="ardTS2" id="ardTS2" value="1" />');
	$('#commentform').attr('onsubmit', 'return validate()');
	// set a first timestamp when the page loads
	var ardTS1 = (new Date).getTime();
	document.getElementById("ardTS1").value = ardTS1;
}

function validate() {
	// read the first timestamp
	var ardTS1 = document.getElementById("ardTS1").value;
//	alert ('ardTS1: ' + ardTS1);
	// generate the second timestamp
	var ardTS2 = (new Date).getTime();
	document.getElementById("ardTS2").value = ardTS2;
//	alert ('ardTS2: ' + document.getElementById("ardTS2").value);
	// find the difference
	var diff = ardTS2 - ardTS1;
	var elapsed = Math.round(diff / 1000);
	var remaining = 10 - elapsed;
//	alert ('diff: ' + diff + '\n\nelapsed:' + elapsed);
	// check whether enough time has elapsed
	if (diff > 10000) {
		// submit the form
		return true;
	}else{
		// display an alert if the form is submitted within 10 seconds
		alert("This site is protected by an anti-spam feature that requires 10 seconds to have elapsed between the page load and the form submission.\n\nPlease close this alert window.  The form may be resubmitted successfully in " + remaining + " seconds.");
		// prevent the form from being submitted
		return false;
	}
}
</script>

File 2: The wp-comments-post.php file

The wp-comments-post.php file lives in the root of WordPress and handles the form processing. We need to add a few lines that check the contents of our new validation input field.

Somewhere after line 53 or so (where $comment_content is defined), insert the following code.

$ardTS1 = ( isset($_POST['ardTS1']) ) ? trim($_POST['ardTS1']) : 1;
$ardTS2 = ( isset($_POST['ardTS2']) ) ? trim($_POST['ardTS2']) : 2;
$ardTS = $ardTS2 - $ardTS1;

if ( $ardTS < 10000 ) {
// If the difference of the timestamps is not more than 10 seconds, exit
    wp_die( __('<strong>ERROR</strong>:  This site uses JavaScript validation to reduce comment spam.  Either your browser has JavaScript disabled, or the comment was not legitimately submitted.') );
}

That’s it. Not so bad, right?

Final thoughts

One advantage to this method over the old PHP-only method is that even if the core file is replaced and the server-side validation is lost, the client-side validation continues to work, perhaps providing some measure of protection.

The method is safe for use with caching systems that create purely static, HTML pages. I’ll follow up later and report on how effective it seems to be at stopping spam comments before they get to Akismet (and into the WordPress database).

Now, for a little extra protection, you can rename the wp-comments-post.php file and change the path in the comment form’s action attribute. I’ve posted logs showing that some bots just try to post spam directly to the wp-comments-post.php file, so renaming that file is an easy way to cut down on spam. Just remember to come back and delete the wp-comments-post.php file each time you update WordPress.

Objects are the building blocks of the application (ie: the workers in a factory)
Classes can be thought of as blueprints for the objects. Classes describe the objects, which are created in memory.
So, the programmer writes the classes and the PHP interpreter creates the objects from the classes.

A class may contain both variables and functions.
A variable inside a class is called a property.
A function inside a class is called a method.

Instantiation

To create an object, you instantiate a class (you create an instance of the class as an object).
For example, if we have a class named ‘person’ and want to instantiate it as the variable $oliver:

$oliver = new person();

The variable $oliver is referred to as the ‘handle’.

Accessing properties and methods

To access the properties and methods of a class, we use the object’s handle, followed by the arrow operator “->”.
For example, if our class has a method ‘get_name’, we can echo that to the page with:

echo $oliver->get_name();

Note that there are no single or double quotes used in instantiating a class or accessing properties and methods of a class.

Constructors

A class may have a special method called a constructor. The constructor method is called automatically when the object is instantiated.
The constructor method begins with two underscores and the word ‘construct’:

function __construct($variable) { }

One can pass values to the constructor method by providing arguments after the class name.
For example, to pass the name “John Doe” to the constructor method in the ‘person’ class:

$john = new person("John Doe");

! If a constructor exists and expects arguments, you must instantiate the class with the arguments expected by the constructor.

Access modifiers and visibility declarations

Properties must, and methods may, have one of three access modifiers (visibility declarations): public, protected, and private.
Public: can be accessed from outside the class, eg: $myclass->secret_variable;
Protected: can be accessed within the class and by classes derived from the class
Private: can be accessed only within the class

Declaring a property with var makes the property public.

Methods declared without an explicit access modifier are considered public.

! If you call a protected method from outside the class, any PHP output before the call is still processed, but you get an error message when the interpreter gets to that call:

Fatal error: Call to protected method...

Inheritance

Inheritance allows a child class to be created from a parent class, whereby the child has all of the public and protected properties and methods of the parent.

A child class extends a parent class:

class employee extends person {
}

A child class can redefine/override/replace a method in the parent class by reusing the method name.

! A child class’s method’s access modifier can not be more restrictive than that of the parent class. For example, if the parent class has a public set_name() method and the child class’s set_name() method is protected, the class itself will generate a fatal error, and no prior PHP output will be rendered. (In the error below, employee is the child class to person):

Fatal error: Access level to employee::set_name() must be public (as in class person) in E:\xampp\htdocs\tester\oop\class_lib.php on line 38

To differentiate between a method in a parent class vs the method as redefined in a child class, one must specifically name the class that contains the method you want to call using the scope resolution operator (::):

person::set_name($new_name);

The scope resolution operator allows access to static, constant, and overridden properties or methods of a class, generally, a parent class. This would be done inside the child class, after redefining a parent’s method of the same name.

It’s also possible to use ‘parent’ to refer to the child’s parent class:

parent::set_name($new_name);

(I’m still a bit vague on this and am looking for examples of situations in which this would be used.)

Classes inside classes

Just as it’s possible to instantiate a class and use the object in a view file, it’s possible to instantiate an object and call its methods from inside another class.

Static properties and methods

Declaring class properties or methods as static makes them accessible without needing an instantiation of the class. A property declared as static can not be accessed with an instantiated class object (though a static method can).

Resources
http://us2.php.net/manual/en/language.oop5.php
http://net.tutsplus.com/tutorials/php/oop-in-php/
http://www.phpfreaks.com/tutorial/oo-php-part-1-oop-in-full-effect
http://www.killerphp.com/tutorials/object-oriented-php/

In a meager effort to console myself for so far failing to be a billionaire, I’m assembling the short list of web-application type things I’ve built here.

1. A dice roller: rollforit. Enter a name, create a room, invite your friends, and start rolling dice. For people who want to play pen and paper, table-top RPG dice games with their distant friends.
2. A URL shortener: Minifi.de. Minifi.de comes with an API and a bookmarklet. It really works, too! The technical explanation has more details.
3. A social networking site: Snapbase. Snapbase is a social site that shows you what’s going on in your city or anywhere in the world as pictures are uploaded by your friends and neighbors. The application extracts location information from the EXIF data embedded in images and displays recent images taken near your present location.
4. A trouble-ticketing system for an IT help desk or technical support center. It’s really pretty extensive, with asset management, user accounts, salted encrypted passwords, and all sorts of nifty things. I really must write a full description of it at some point, but until then, the documentation is the next best thing.
5. An account-based invoice tracking and access system for grouping invoices according to clients, then sharing invoice history with those clients and allowing them to easily pay outstanding invoices via Paypal.
6. An account-based invoice access system where clients can view paid and unpaid invoices, and even easily pay an outstanding invoice via Paypal. I actually use this almost every day.
7. A simple method for protecting a download using a unique URL that can be emailed to authorized users. The URL can be set to expire after a certain amount of time or any number of downloads.
8. An update to the above download protection script to protect multiple downloads, generate batches of keys, leave notes about who received the key, the ability to specify per-key the allowable number of downloads and age, and some basic reporting.
9. An HTML auction template generator called Simple Auction Wizard. It helps you create HTML auction templates for eBay, and uses SWFUpload and tinyMCE.

I have another project in the works that promises to be more financially viable, but the most clever thing on that list is Snapbase. It’s in something akin to alpha right now; barely usable. I really wish I had the time to pursue it.

...
[01-Jun-2011 14:25:15] PHP Fatal error:  Call to undefined function  get_header() in /home/accountname/public_html/ardamis.com/wp-content/themes/ars/index.php on line 7
[01-Jun-2011 20:58:23] PHP Fatal error:  Call to undefined function  get_header() in /home/accountname/public_html/ardamis.com/wp-content/themes/ars/index.php on line 7
...

The first seven lines of my theme’s index.php file:

<?php ini_set('display_errors', 0); ?>
<?php
/**
 * @package WordPress
 * @subpackage Ars_Theme
*/
get_header(); ?>

I realized that the error was being generated each time that my theme’s index.php file was called directly, and that the error was caused by the theme’s inability to locate the WordPress get_header function (which is completely normal). Thankfully, the descriptive error wasn’t being output to the browser, but was only being logged to the error_log file, due to the inclusion of the ini_set(‘display_errors’, 0); line. I had learned this the hard way a few months ago when I found that calling the theme’s index.php file directly would generate an error message, output to the browser, that would reveal my hosting account username as part of the absolute path to the file throwing the error.

I decided the best way to handle this would be to check to see if the file could find the get_header function, and if it could not, simply redirect the visitor to the site’s home page. The code I used to do this:

<?php ini_set('display_errors', 0); ?>
<?php
/**
* @package WordPress
* @subpackage Ars_Theme
*/
if (function_exists('get_header')) {
	get_header();
}else{
    /* Redirect browser */
    header("Location: http://" . $_SERVER['HTTP_HOST'] . "");
    /* Make sure that code below does not get executed when we redirect. */
    exit;
}; ?>

So there you have it. No more fatal errors due to get_header when loading the WordPress theme’s index.php file directly. And if something else in the file should throw an error, ini_set(‘display_errors’, 0); means it still won’t be sent to the browser.

Tools for detecting infections on web sites

Google Webmaster Tools

Your first stop should be here, as I’ve personally witnessed alerts show up in Webmaster Tools, even when all the following tools gave the site a passing grade. If your site is registered here, and Google finds weird pages on your site, an alert will appear. You can also have the messages forwarded to your email account on file, by choosing the Forward option under the All Messages area of the Home page.

Google Webmaster Tools Hack Alert

Google Safe Browsing

The Google Safe Browsing report for ardamis.com: http://safebrowsing.clients.google.com/safebrowsing/diagnostic?site=ardamis.com

Norton Safe Web

https://safeweb.norton.com/

The Norton Safe Web report for ardamis.com: https://safeweb.norton.com/report/show?url=ardamis.com

Tools for analyzing a site for vulnerabilities

Sucuri Site Check

http://sitecheck.sucuri.net/scanner/

The Sucuri report for ardamis.com: http://sitecheck.sucuri.net/scanner/?scan=www.ardamis.com.

This was sufficiently effective, but it was also overly complicated, requiring four separate components:

1. an autoexec Word 2007 macro that runs each time Word is opened
2. a batch file that runs the registry merge file and writes an entry to a log file
3. the registry merge file that contains the correct LoadBehavior settings for the add-ins
4. a text file that acts as a log

This month, I decided to rewrite the macro to handle the registry changes and write to the log file. It was also a good opportunity to dig a bit deeper into VBA, and I also wanted to confirm that it would work in a more modern environment of Windows 7 and Office 2010 (that code is near the bottom of the post). The new system has only two components:

1. an autoexec Word 2007 macro that runs each time Word is opened
2. a text file that acts as a log

Background

First, a bit of background.

Many of the problems with Word 2007 are due to Word’s handling of add-ins. When something unexpected happens in Word, and Word attributes the problem to an add-in, Word will react by flagging it and prompting the user for a decision the next time Word opens. Depending on the severity of the problem and the user’s response, the add-in can be either ‘hard-disabled’ or ‘soft-disabled’.

Microsoft explains the differences between Hard Disabled vs Soft Disabled in a MSDN article at: http://msdn.microsoft.com/en-us/library/ms268871(VS.80).aspx.

I’ve explained a bit about the process by which Word disables add-ins at the end of this post, and I’ve written a shorter post about the basics behind the registry keys responsible for disabling add-ins.

Handling disabled add-ins programmatically

A Word macro can access the condition of an add-in via an Application.COMAddIns object, and it can read and write to the registry. This allows us to tell when an add-in has been disabled and re-enabled it.

My macro has some admittedly hackish parts that need to be cleaned up, there is the matter of unsetting variables to be addressed, and it could certainly be made more elegant, but it works. Note that a file named addinslog.txt must exist in the %TEMP% directory in order for the macro to write the log file. This is what the Word 2007 macro looks like, using the COM add-in installed with Adobe Acrobat 8 Standard as the required add-in…

Option Explicit

' Set up a function to search for a key and return true or false
Public Function KeyExists(key)
    Dim objShell
    On Error Resume Next
    Set objShell = CreateObject("WScript.Shell")
        objShell.RegRead (key)
    Set objShell = Nothing
    If Err = 0 Then KeyExists = True
End Function

Sub AutoExec()
'
' FixMissingAddins Macro
' Display a message box with any critical but not 'Connected' COM add-ins, then fix them programatically
'
' Oliver Baty
' June, 2010 - April, 2011
'
' Information on the Application.COMAddIns array
' http://msdn.microsoft.com/en-us/library/aa831759(v=office.10).aspx
'
' Running macros automatically
' http://support.microsoft.com/kb/286310
'
' Using Windows Scripting Shell (WshShell) to read from and write to the local registry
' http://technet.microsoft.com/en-us/library/ee156602.aspx

' Declare the WshShell variable (this is used to edit the registry)
    Dim WshShell

' Declare the fso and logFile variables (these are used to write to a txt file)
    Dim fso
    Dim logFile

' Create an instance of the WScript Shell object
    Set WshShell = CreateObject("WScript.Shell")

' Declare some other variables
   Dim MyAddin As COMAddIn
   Dim stringOfAddins As String
   Dim listOfDisconnectedAddins As String
   Dim requiredAddIn As Variant
   Dim msg As String

' Notes on deleting registry keys and values in VB
' http://www.vbforums.com/showthread.php?t=425483
' http://www.tek-tips.com/viewthread.cfm?qid=674375
' http://www.robvanderwoude.com/vbstech_registry_wshshell.php

' Create a string containing the names of all 'Connected' COM add-ins named "stringOfAddins"
   For Each MyAddin In Application.COMAddIns
      If MyAddin.Connect = True Then
          stringOfAddins = stringOfAddins & MyAddin.ProgID & " - "
      End If
   Next

' Create an array to hold the names of the critical (required) add-ins named "requiredAddIns"
' Example: change to "Dim requiredAddIns(0 To 4)" if the macro is checking 5 total add-ins)
   Dim requiredAddIns(0 To 0) As String

' Add each required AddIn to the array
   requiredAddIns(0) = "PDFMaker.OfficeAddin"
'   requiredAddIns(1) = ""
'   requiredAddIns(2) = ""
'   requiredAddIns(3) = ""
'   requiredAddIns(4) = ""

' Cycle through the array of required add-ins, and see if they exist in the connected add-ins list
   For Each requiredAddIn In requiredAddIns
      If InStr(stringOfAddins, requiredAddIn) Then
        ' The required add-in is in the string of connected add-ins
         msg = msg
      Else
        ' The required add-in is not in the string of connected add-ins, so add the add-in name to a string named "listOfDisconnectedAddins"
         msg = msg & requiredAddIn & vbCrLf
         listOfDisconnectedAddins = requiredAddIn & " " & listOfDisconnectedAddins
         listOfDisconnectedAddins = Trim(listOfDisconnectedAddins)
      End If
   Next

' If the msg variable is not blank (it contains at least one add-in's name) handle it, otherwise, do nothing
   If msg = "" Then
        ' There are no critical, unconnected add-ins (yay!)
        ' The script can now exit
   Else
        ' There are critical add-ins that are not connected, so handle this
        MsgBox "The following critical Word Add-In(s) are disabled: " & vbCrLf & vbCrLf & msg & vbCrLf & vbCrLf & "To correct this problem, please save any documents you are working on, then close Word and reopen Word."

            ' I find it extremely hackish to check for each possible key and delete it if found... need to research how to delete the tree
            ' One potential obstacle to this method is that I've seen a DocumentRecovery subkey under Resiliency (only once, while editing this macro), that I haven't researched yet

            ' Note: Since the WSH Shell has no Enumeration functionality, you cannot
            '       use the WSH Shell object to delete an entire "tree" unless you
            '       know the exact name of every subkey.
            '       If you don't, use the WMI StdRegProv instead.
            ' http://www.robvanderwoude.com/vbstech_registry_wshshell.php

            ' More info on WMI StdRegProv at:
            ' http://msdn.microsoft.com/en-us/library/aa393664(v=vs.85).aspx

        ' This is hackish, but it effectively deletes a registry key, if it exists
        If KeyExists("HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Resiliency\DisabledItems\") Then
            WshShell.RegDelete "HKCU\Software\Microsoft\Office\12.0\Word\Resiliency\DisabledItems\"
        ElseIf KeyExists("HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Resiliency\StartupItems\") Then
            WshShell.RegDelete "HKCU\Software\Microsoft\Office\12.0\Word\Resiliency\StartupItems\"
        ElseIf KeyExists("HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Resiliency\") Then
            WshShell.RegDelete "HKCU\Software\Microsoft\Office\12.0\Word\Resiliency\"
        End If

        ' To be completely thorough, we can also set the desired LoadBehavior for certain add-ins
        ' This can be done selectively, and only if the LoadBehavior was incorrect, but the quick and dirty way would be to just force the values

        WshShell.RegWrite "HKLM\SOFTWARE\Microsoft\Office\Word\Addins\PDFMaker.OfficeAddin\LoadBehavior", 3, "REG_DWORD"

        ' Release the WshShell object
        Set WshShell = Nothing

        ' Declare a few variables for the log file
        Dim user, machine, datetime, output

        Set WshShell = CreateObject("WScript.Shell")
        user = WshShell.ExpandEnvironmentStrings("%USERNAME%")
        machine = WshShell.ExpandEnvironmentStrings("%COMPUTERNAME%")
        temp = WshShell.ExpandEnvironmentStrings("%TEMP%")
        ' Convert the slashes in Now to hyphens to prevent a fatal error
        datetime = Replace(Now, "/", "-")
        ' Create the string that will be written to the log file
        output = datetime + ", " + user + ", " + machine + ", " + listOfDisconnectedAddins

        ' Write the event to a log file
        logfile = temp + "\addinslog.txt"
        ' http://msdn.microsoft.com/en-us/library/2z9ffy99(v=vs.85).aspx
        ' http://www.devguru.com/technologies/vbscript/quickref/filesystemobject_opentextfile.html
        Set fso = CreateObject("Scripting.FileSystemObject")
        Set logFile = fso.OpenTextFile(logfile, 8, True)
        logFile.WriteLine (output)
        logFile.Close
        Set logFile = Nothing
        Set fso = Nothing

        ' Should we clear the variables?

        ' Release the WshShell object
        Set WshShell = Nothing
   End If

   ' Ardamis.com - We're in your macros, fixing your COM add-ins.
End Sub

While working on this, I found that there were some gaps in my understanding of the sequence of events that occur when Word 2007 disables a COM add-in. Please comment if you find that any of this is inaccurate or incomplete.

What happens when Word launches

A critical key to the whole business of Word add-ins is HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Resiliency

When Word launches, it looks for data under the Resiliency key and a subkey: HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Resiliency\StartupItems

If the StartupItems subkey contains a REG_BINARY value that corresponds to an add-in, Word throws the familiar warning:

Microsoft Office Word
Word experienced a serious problem with the ‘[addin name]’ add-in. If you have seen this message multiple times, you should disable this add-in and check to see if an update is available. Do you want to disable this add-in?
[Yes] [No]

Choosing No at the prompt removes the Resiliency key and allows Word to continue to launch, leaving the LoadBehavior for that add-in unchanged.

Choosing No also writes an Error event to the Application Event Viewer log:

Event Type:	Error
Event Source:	Microsoft Office 12
Event Category:	None
Event ID:	2000
Date:		5/23/2011
Time:		3:15:29 PM
User:		N/A
Computer:	[WORKSTATION_NAME]
Description:
Accepted Safe Mode action : Microsoft Office Word.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Choosing Yes at the prompt removes the StartupItems subkey and creates a new DisabledItems subkey. This DisabledItems subkey will contain a different REG_BINARY value, the data of which contains information about the disabled add-in.

Choosing Yes also writes an Error event to the Application Event Viewer log:

Event Type:	Error
Event Source:	Microsoft Office 12
Event Category:	None
Event ID:	2001
Date:		5/23/2011
Time:		3:12:36 PM
User:		N/A
Computer:	[WORKSTATION_NAME]
Description:
Rejected Safe Mode action : Microsoft Office Word.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

At this point, the add-in is ‘hard-disabled’, but not ‘soft-disabled’.

Word then continues to launch, but without loading the add-in.

To see which add-ins have been hard-disabled, click on the Office Button | Word Options | Add-Ins, and scroll down to “Disabled Application Add-ins”.

To see which add-ins have been soft-disabled, click on the Office Button | Word Options | Add-Ins. Select “COM Add-Ins” in the Manage menu and click Go.

Word is somewhat tricky in this regard, as the add-in will not have a checkmark, but the LoadBehavior registry value will be unchanged. At any other time, the presence of a checkmark is an indication of the LoadBehavior, but when an add-in has been hard-disabled, the box will always be unchecked.

What users can do at this point

Going through Word Options and enabling the hard-disabled COM add-in will remove the Resiliency key. This may not make the add-in immediately available in Word, however.

To immediately load the add-in and gain its functionality, you can check the box. Otherwise, close and reopen Word, which will cause Word to launch with the add-in’s specified LoadBehavior.

In case you were curious about the keyboard shortcuts used to enable the first disabled add-in in the list of disabled add-ins (maybe you wanted to do something with SendKeys, for example), they are:
Alt+F, I, A, A, Tab, Tab, Tab, D, Enter, G, Space, Alt+E, C, Alt+F4.

In summary, deleting the Resiliency key after the “serious problem” prompt, then closing and reopening Word, returns Word to a normal operating state.

What I intend to accomplish with the macro is to re-enable the hard-disabled add-in, return any LoadBehavior values back to the desired settings, then prompt the user to save their work and close and reopen Word.

This should return Word to a working state.

Word 2010 on 64-bit Windows 7

As a bonus, here’s the same macro, with some minor adjustments to run in Word 2010 on Windows 7 64-bit, with Adobe Acrobat 9 Pro’s COM add-in acting as one of the required add-ins. The OneNote add-in is not enabled in Word by default, and the macro below does not attempt to enable it, but does consider it a required add-in. This is done to demonstrate the pop-up window. Note that a file named addinslog.txt must exist in the %TEMP% directory in order for the macro to write the log file.

Option Explicit

' Set up a function to search for a key and return true or false
Public Function KeyExists(key)
    Dim objShell
    On Error Resume Next
    Set objShell = CreateObject("WScript.Shell")
        objShell.RegRead (key)
    Set objShell = Nothing
    If Err = 0 Then KeyExists = True
End Function

Sub AutoExec()
'
' FixMissingAddins Macro
' Display a message box with any critical but not 'Connected' COM add-ins, then fix them programatically
'
' Oliver Baty
' June, 2010 - April, 2011
'
' Information on the Application.COMAddIns array
' http://msdn.microsoft.com/en-us/library/aa831759(v=office.10).aspx
'
' Running macros automatically
' http://support.microsoft.com/kb/286310
'
' Using Windows Scripting Shell (WshShell) to read from and write to the local registry
' http://technet.microsoft.com/en-us/library/ee156602.aspx

' Declare the WshShell variable (this is used to edit the registry)
    Dim WshShell

' Declare the fso and logFile variables (these are used to write to a txt file)
    Dim fso
    Dim logfile

' Create an instance of the WScript Shell object
    Set WshShell = CreateObject("WScript.Shell")

' Declare some other variables
   Dim MyAddin As COMAddIn
   Dim stringOfAddins As String
   Dim listOfDisconnectedAddins As String
   Dim requiredAddIn As Variant
   Dim msg As String

' Notes on deleting registry keys and values in VB
' http://www.vbforums.com/showthread.php?t=425483
' http://www.tek-tips.com/viewthread.cfm?qid=674375
' http://www.robvanderwoude.com/vbstech_registry_wshshell.php

' Create a string containing the names of all 'Connected' COM add-ins named "stringOfAddins"
   For Each MyAddin In Application.COMAddIns
      If MyAddin.Connect = True Then
          stringOfAddins = stringOfAddins & MyAddin.ProgID & " - "
      End If
   Next

' Create an array to hold the names of the critical (required) add-ins named "requiredAddIns"
' Example: change to "Dim requiredAddIns(0 To 4)" if the macro is checking 5 total add-ins)
   Dim requiredAddIns(0 To 1) As String

' Add each required AddIn to the array
   requiredAddIns(0) = "PDFMaker.OfficeAddin"
   requiredAddIns(1) = "OneNote.WordAddinTakeNotesService"
'   requiredAddIns(2) = ""
'   requiredAddIns(3) = ""
'   requiredAddIns(4) = ""

' Cycle through the array of required add-ins, and see if they exist in the connected add-ins list
   For Each requiredAddIn In requiredAddIns
      If InStr(stringOfAddins, requiredAddIn) Then
        ' The required add-in is in the string of connected add-ins
         msg = msg
      Else
        ' The required add-in is not in the string of connected add-ins, so add the add-in name to a string named "listOfDisconnectedAddins"
         msg = msg & requiredAddIn & vbCrLf
         listOfDisconnectedAddins = requiredAddIn & " " & listOfDisconnectedAddins
         listOfDisconnectedAddins = Trim(listOfDisconnectedAddins)
      End If
   Next

' If the msg variable is not blank (it contains at least one add-in's name) handle it, otherwise, do nothing
   If msg = "" Then
        ' There are no critical, unconnected add-ins (yay!)
        ' The script can now exit
   Else
        ' There are critical add-ins that are not connected, so handle this
        MsgBox "The following critical Word Add-In(s) are disabled: " & vbCrLf & vbCrLf & msg & vbCrLf & vbCrLf & "To correct this problem, please save any documents you are working on, then close Word and reopen Word."

            ' I find it extremely hackish to check for each possible key and delete it if found... need to research how to delete the tree
            ' One potential obstacle to this method is that I've seen a DocumentRecovery subkey under Resiliency (only once, while editing this macro), that I haven't researched yet

            ' Note: Since the WSH Shell has no Enumeration functionality, you cannot
            '       use the WSH Shell object to delete an entire "tree" unless you
            '       know the exact name of every subkey.
            '       If you don't, use the WMI StdRegProv instead.
            ' http://www.robvanderwoude.com/vbstech_registry_wshshell.php

            ' More info on WMI StdRegProv at:
            ' http://msdn.microsoft.com/en-us/library/aa393664(v=vs.85).aspx

        ' This is hackish, but it effectively deletes a registry key, if it exists
        If KeyExists("HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Word\Resiliency\DisabledItems\") Then
            WshShell.RegDelete "HKCU\Software\Microsoft\Office\14.0\Word\Resiliency\DisabledItems\"
        ElseIf KeyExists("HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Word\Resiliency\StartupItems\") Then
            WshShell.RegDelete "HKCU\Software\Microsoft\Office\14.0\Word\Resiliency\StartupItems\"
        ElseIf KeyExists("HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Word\Resiliency\") Then
            WshShell.RegDelete "HKCU\Software\Microsoft\Office\14.0\Word\Resiliency\"
        End If

        ' To be completely thorough, we can also set the desired LoadBehavior for certain add-ins
        ' This can be done selectively, and only if the LoadBehavior was incorrect, but the quick and dirty way would be to just force the values

        WshShell.RegWrite "HKCU\Software\Microsoft\Office\Word\Addins\PDFMaker.OfficeAddin\LoadBehavior", 3, "REG_DWORD"

        ' Release the WshShell object
        Set WshShell = Nothing

        ' Declare a few variables for the log file
        Dim user, machine, temp, datetime, output

        Set WshShell = CreateObject("WScript.Shell")
        user = WshShell.ExpandEnvironmentStrings("%USERNAME%")
        machine = WshShell.ExpandEnvironmentStrings("%COMPUTERNAME%")
        temp = WshShell.ExpandEnvironmentStrings("%TEMP%")
        ' Convert the slashes in Now to hyphens to prevent a fatal error
        datetime = Replace(Now, "/", "-")
        ' Create the string that will be written to the log file
        output = datetime + ", " + user + ", " + machine + ", " + listOfDisconnectedAddins

        ' Write the event to a log file
        logfile = temp + "\addinslog.txt"
        ' http://msdn.microsoft.com/en-us/library/2z9ffy99(v=vs.85).aspx
        ' http://www.devguru.com/technologies/vbscript/quickref/filesystemobject_opentextfile.html
        Set fso = CreateObject("Scripting.FileSystemObject")
        Set logfile = fso.OpenTextFile(logfile, 8, True)
        logfile.WriteLine (output)
        logfile.Close
        Set logfile = Nothing
        Set fso = Nothing

        ' Should we clear the variables?

        ' Release the WshShell object
        Set WshShell = Nothing
   End If

   ' Ardamis.com - We're in your macros, fixing your COM add-ins.
End Sub

Page Speed Online

http://pagespeed.googlelabs.com/

Google’s wonderful Page Speed tool, once only available as a Firefox browser Add-on, finally arrives as an online tool. Achieving a high score (ardamis.com is a 96/100) should be on every web developer’s list of things to do before the culmination of a project.

Enter a URL and Page Speed Online will run performance tests based on a set of best practices known to reduce page load times.

Optimizing caching – keeping your application’s data and logic off the network altogether
Minimizing round-trip times – reducing the number of serial request-response cycles
Minimizing request overhead – reducing upload size
Minimizing payload size – reducing the size of responses, downloads, and cached pages
Optimizing browser rendering – improving the browser’s layout of a page

WebPagetest

http://www.webpagetest.org/

WebPagetest is an excellent application for users who want the same sort of detailed reporting that one gets with Page Speed.

Load time speed test on first view (cold cache) and repeat view (hot cache), first byte and start render
Optimization checklist
Enable keep-alive, HTML compression, image compression, cache static content, combine JavaScript and CSS, and use of CDN
Waterfall
Response headers for each request

Load Impact

http://loadimpact.com/pageanalyzer.php

Load Impact is an online load testing service that lets you load- and stress test your website over the Internet. The page analyzer analyzes your web page performance by emulating how a web browser would load your page and all resources referenced in it. The page and its referenced resources are loaded and important performance metrics are measured and displayed in a load-bar diagram along with other per-resource attributes such as URL, size, compression ratio and HTTP status code.

ByteCheck

http://www.bytecheck.com/

ByteCheck is a super minimal site that return your page’s all-important time to first byte (TTFB). Time to first byte is the time it takes for a browser to start receiving information after it has started to make the request to the server, and is responsible for a visitor’s first impression that a page is fast- or slow-loading.

Web Page Analyzer

http://websiteoptimization.com/services/analyze/

My opinion is that the Web Page Analyzer report is good for beginners without much technical knowledge of things like gzip compression and Expires headers. It’s a bit dated, and is primarily concerned with basics like how many images a page contains. It tells you how fast you can expect your page to load for dial-up visitors, which strikes me as quaint and not particularly useful.

Total HTTP requests
Total size
Total size per object type (CSS, JavaScript, images, etc.)
Analysis of number of files and file size as compared to recommended limits.

The Performance Grader

http://www.joomlaperformance.com/component/option,com_performance/Itemid,52/

This is another simplistic analysis of a site, like Web Page Analyzer, that returns its analysis in the form of pass/fail grades on about 14 different tests. I expect that it would be useful for developers who want to show a client a third-party’s analysis of their work, if the third-party is not terribly technically savvy.

One unique thing about this tool, though, is that it totals up the size of all images referenced in CSS files (even those that the current page isn’t using).

HTML Size
Total Size
Total Requests
Generation Time
Number of Hosts
Number of Images
Size of Images
Number of CSS Files
Size of CSS Files
Number of Script Files
Size of Script Files
HTML Encoding
Valid HTML
Frames

Frameworks and scripts

HTML5 Boilerplate is the professional badass’s base HTML/CSS/JS template for a fast, robust and future-proof site.

scriptsrc.net is a collection of script tags of the latest versions of a range of JavaScript libraries.

Modernizr adds classes to the <html> element which allow you to target specific browser functionality in your stylesheet. You don’t actually need to write any Javascript to use it.

Images

placehold.it is a quick and simple image placeholder service.

Text manipulation

TextFixer is a collection of online text tools. Remove line breaks from text, alphabetize text, capitalize the first letter of sentences, remove whitespaces, and uppercase text or lowercase text.

XHTML

HTML Minifier will minify HTML (or XHTML), and any CSS or JS included in your markup.

CSS

CSS3 Generator is an awesome code generator for CSS3 snippets, and shows the minimum browser versions required to display the effects.

proCSSor is an advanced CSS prettifier with tons of formatting options.

JavaScript

Online javascript beautifier will reformat and reindent bookmarklets, ugly javascript, unpack scripts packed by the popular Dean Edward’s packer, as well as deobfuscate scripts processed by javascriptobfuscator.com. The source code for the latest version is always available on github, and you can download the beautifier for local use (zip, tar.gz) as well.

Fonts and Typography

Fontshop.com has written A Field Guide to Typography to get you excited about fonts and typography.

Typetester is an online app for comparing different fonts for the screen, you can test up to three fonts at a time and choose the one you like. Its primary role is to make web designer’s life easier.

A quick chart of the fonts common to all versions of Windows and the Mac equivalents, or a more extensive matrix of fonts bundled with Mac and Windows operating systems, Microsoft Office and Adobe Creative Suite.

<html>ipsum has Lorem ipsum already wrapped in HTML tags. Pre-made paragraphs, lists, etc…

More resources at: 50 Useful Design Tools For Beautiful Web Typography and 21 Typography and Font Web Apps You Can’t Live Without.

Colors

Color Scheme Designer.

Markup

Google Webmaster Tools’ Rich Snippets Testing Tool.

Use the Rich Snippets Testing Tool to check that Google can correctly parse your structured data markup and display it in search results.

A great explanation of why compression helps the web deliver a better user experience is at betterexplained.com.

Two authoritative articles on the subject are Google’s Performance Best Practices documentation | Enable compression and Yahoo’s Best Practices for Speeding Up Your Web Site | Gzip Components.

Compressing PHP files

If your Apache server does not have mod_gzip or mod_deflate enabled (Godaddy and JustHost shared hosting, for example), you can use PHP to compress pages on-the-fly. This is still preferable to sending uncompressed files to the browser, so don’t worry about the additional work the server has to do to compress the files at each request.

Option 1: PHP.INI using zlib.output_compression

The zlib extension can be used to transparently compress PHP pages on-the-fly if the browser sends an “Accept-Encoding: gzip” or “deflate” header. Compression with zlib.output_compression seems to be disabled on most hosts by default, but can be enabled with a custom php.ini file:

[PHP]

zlib.output_compression = On

Credit: http://php.net/manual/en/zlib.configuration.php

Check with your host for instructions on how to implement this, and whether you need a php.ini file in each directory.

Option 2: PHP using ob_gzhandler

If your host does not allow custom php.ini files, you can add the following line of code to the top of your PHP pages, above the DOCTYPE declaration or first line of output:

<?php if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')) ob_start("ob_gzhandler"); else ob_start(); ?>

Credit: GoDaddy.com

For WordPress sites, this code would be added to the top of the theme’s header.php file.

According to php.net, using zlib.output_compression is preferred over ob_gzhandler().

For WordPress or other CMS sites, an advantage of zlib.output_compression over the ob_gzhandler method is that all of the .php pages served will be compressed, not just those that contain the global include (eg.: header.php, etc.).

Running both ob_gzhandler and zlib.output_compression at the same time will throw a warning, similar to:

Warning: ob_start() [ref.outcontrol]: output handler ‘ob_gzhandler’ conflicts with ‘zlib output compression’ in /home/path/public_html/ardamis.com/wp-content/themes/mytheme/header.php on line 7

Compressing CSS and JavaScript files

Because the on-the-fly methods above only work for PHP pages, you’ll need something else to compress CSS and JavaScript files. Furthermore, these files typically don’t change, so there isn’t a need to compress them at each request. A better method is to serve pre-compressed versions of these files. I’ll describe a few different ways to do this, but in both cases, you’ll need to add some lines to your .htaccess file to send user agents the gzipped versions if they support the encoding. This requires that Apache’s mod_rewrite be enabled (and I think it’s almost universally enabled).

Option 1: Compress locally and upload

CSS and JavaScript files can be gzipped on the workstation, then uploaded along with the uncompressed files. Use a utility like 7-Zip (quite possibly the best compression software around, and it’s free) to compress the CSS and JavaScript files using the gzip format (with extension *.gz), then upload them to your server.

For Windows users, here is a handy command to compress all the .css and .js files in the current directory and all sub directories (adjust the path to the 7-Zip executable, Zz.exe, as necessary):

for /R %i in (*.css *.js) do "C:\Program Files (x86)\7-Zip\7z.exe" a -tgzip "%i.gz" "%i" -mx9

Note that the above command is to be run from the command line. The batch file equivalent would be:

for /R %%i in (*.css *.js) do "C:\Program Files (x86)\7-Zip\7z.exe" a -tgzip "%%i.gz" "%%i" -mx9

Option 2: Compress on the server

If you have shell access, you can run a command to create a gzip copy of each CSS and JavaScript file on your site (or, if you are developing on Linux, you can run it locally):

find . -regex ".*\(css\|js\)$" -exec bash -c 'echo Compressing "{}" && gzip -c --best "{}" > "{}.gz"' \;

This may be a bit too technical for many people, but is also much more convenient. It is particularly useful when you need to compress a large number of files (as in the case of a WordPress installation with multiple plugins). Remember to run it every time you automatically update WordPress, your theme, or any plugins, so as to replace the gzip’d versions of any updated CSS and JavaScript files.

The .htaccess (for both options)

Add the following lines to your .htaccess file to identify the user agents that can accept the gzip encoded versions of these files.

<files *.js.gz>
  AddType "text/javascript" .gz
  AddEncoding gzip .gz
</files>
<files *.css.gz>
  AddType "text/css" .gz
  AddEncoding gzip .gz
</files>
RewriteEngine on
#Check to see if browser can accept gzip files.
ReWriteCond %{HTTP:accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
#make sure there's no trailing .gz on the url
ReWriteCond %{REQUEST_FILENAME} !^.+\.gz$
#check to see if a .gz version of the file exists.
RewriteCond %{REQUEST_FILENAME}.gz -f
#All conditions met so add .gz to URL filename (invisibly)
RewriteRule ^(.+) $1.gz [QSA,L]

Credit: opensourcetutor.com

I’m not sure it’s still necessary to exclude Safari.

For added benefit, minify the CSS and JavaScript files before gzipping them. Google’s excellent Page Speed Firefox/Firebug Add-on makes this very easy. Yahoo’s YUI Compressor is also quite good.

Verify that your content is being compressed

Use the nifty Web Page Content Compression Verification tool at http://www.whatsmyip.org/http_compression/ to confirm that your server is sending the compressed files.

Speed up page load times for returning visitors

Compression is only part of the story. In order to further speed page load times for your returning visitors, you will want to send the correct headers to leverage browser caching.