Tidybot Home Page

Introduction

Tidybot is a cross-platform batch (X)HTML syntax-checker and report-generator. It traverses a directory tree of (X)HTML files on your hard disk, and generates a web page listing all the errors and warnings it encounters.

Tidybot can be used in two ways: from the command-line, or through a user-friendly GUI application. The output will be identical in either case.

Downloading Tidybot

Unix

tidybot-1.6.1.tar.gz (5 November 2005, 58 Kb)

Microsoft Windows

Tidybot 1.6.1 Installer (5 November 2005, 4 Mb)

You can have a look at the installation instructions, at the version history of user-visible changes, or at some screenshots.

Using Tidybot

Using the command-line version

The command-line version of Tidybot is (on all platforms) an executable program called tidybot.

You can run the command tidybot --help from a suitable shell or command prompt to see a summary of the available options.

Using the GUI version

On Unix platforms, the GUI version of Tidybot is installed as tidybot_gui, and can be executed directly from the command line.

On MS Windows, the installer will have installed the GUI version as the program Tidybot GUI, which can be executed from the Start Menu: Start > Programs > Tidybot > Tidybot GUI.

In addition (depending on what you instructed the installer to do) you may also have a Tidybot GUI icon on your Desktop, and/or in the “Quick Launch” area of your Taskbar, that you can use to start the program.

The GUI version of Tidybot allows you to specify exactly the same options as the command-line version, just in a more convenient point-and-click fashion. The GUI version also has some functionality the command-line version lacks:

The less often-used options are initially hidden. They can be made visible by choosing the Show Advanced Options menu item from the Options menu.
If the Display results in browser checkbox is checked, Tidybot will send the results of the Tidybot run to your default browser.
Settings and values are saved when you quit the application, and restored the next time you start it up. Some fields maintain a history of previously entered values, which can be selected from a drop-down menu.

Tidybot Options

Command-line usage:

tidybot OPTION... <source directory>...

That is, zero or more options followed by one or more source directories.

GUI usage:

All Tidybot options have correspondingly labelled versions as fields in the GUI screen (except for --version and --help, which are menu items instead, and --verbose, which is on the TODO list).

Options:

--version
Show the program’s version number and exit.
(GUI: About Tidybot menu item from the Help menu.)
--help, -h
Show a short help message and exit.
(GUI: Tidybot Documentation menu item from the Help menu.)
--verbose
Turn on verbose mode. (GUI: not implemented yet.)
--xml-pass
Have Tidybot make two passes over each file. Apart from the normal XHTML syntax check, Tidy will also check the file for XML well-formedness. The results from each pass are merged together in the final report. The XML pass is identical to the one performed when the tidy-option input-xml:true is given. The --xml-pass option is a handy shortcut for having the two types of check done in a single run and getting the results on a single page.
--output-directory=DIR
The HTML report files generated by Tidybot will be placed in DIR. The main file will be named ‘tidybot.html’. DIR must be an existing directory.

--include=LIST, -i LIST
Include certain files in the

<source
      directories>

for processing by Tidybot. LIST is a comma-separated list of filename patterns. If a filename matches any of these patterns, the file will be checked by Tidybot. Filename patterns are Unix shell style:

`*`	matches everything
`?`	matches any single character
`seq`	matches any character in seq
`!seq`	matches any char not in seq

The default include list is "*.html, *.htm", i.e. all files with the extension '.html' or '.htm'.

--include-from=FILE
Similar to --include, but the inclusion patterns are now read from text file FILE, which should contain one pattern per line.
--exclude=LIST, -e LIST
Exclude certain files in the <source directories> from being processed by Tidybot. LIST is a comma-separated list of strings. If a filename matches or contains any of these strings the file is skipped and not checked. Exclude strings, unlike include strings, are not filename patterns, but literal substrings. Wildcards such as * are not interpreted. The matching is against the full path of the file, so e.g. entire subdirectories can be skipped just by --exclude’ing the name of the subdirectory. See the next section for examples.
--exclude-from=FILE
Similar to --exclude, but the exclusion strings are now read from text file FILE, which should contain one exclusion string per line.
--suppress=LIST, -s LIST
Suppress certain Tidybot messages from being shown on the generated HTML result pages. LIST is a comma-separated list of strings. If a Tidybot message matches or contains any of these strings the message is not displayed. The string matching is case-sensitive. Suppression strings are literal, like --exclude strings. This option can be used to get Tidybot to shut up about certain XML/XHTML warnings you are not interested in having clutter up the reports. See the next section for examples.
--suppress-from=FILE
Similar to --suppress, but the suppression strings are now read from text file FILE, which should contain one suppression string per line.
--tidy-options=LIST, -t LIST
Pass options directly to the underlying Tidy Library engine. LIST is a comma-separated list of strings, each of the form key:value, where ‘key’ is a Tidy-specific option. This option is typically used to fine-tune the behaviour of the Tidy Library that Tidybot is built on top of. A list of valid options can be found in the Tidy documentation. See the next section for examples.
--tidy-options-from=FILE
Similar to --tidy-options, but the Tidy options are now read from text file FILE, which should contain one key:value string per line.
--stylesheet=URL
Have the generated HTML files reference the CSS stylesheet found at URL. This option can be used to change the look-and-feel of the results pages generated by Tidybot.
--mailto=ADDRESS
Mail a brief synopsis of the Tidybot run to email address ADDRESS when done. This option relies on the existence of a sendmail executable in your search path (or in the environment variable SENDMAIL) to send the email. It is therefore a Unix-only option. On the off chance that there exist MS Windows systems that do have sendmail installed, this option is not disabled for the Windows version of Tidybot.
--note=NOTE, -n NOTE
Include string NOTE in the report mailed by the --mailto option.

If both the LIST and FILE versions of an option are specified on the command-line, the option values will be merged. Multiple LIST options (or multiple FILE options) are allowed, but the value of the last one specified will override all the earlier values.

Examples

The directory ‘example-sources’ distributed with Tidybot contains three sample XHTML files: ‘good.html’, ‘bad-1.html’, and ‘bad-2.html’. The first file contains correct XHTML, the latter two contain various errors.

The GUI version of Tidybot will start up with the ‘example-sources’ as the default value for the ‘Source Directories’ field.

The following examples are expressed as command-line invocations of Tidybot. They all assume the user has cd’d into the directory that contains the ‘example-sources’ subdirectory.

These examples can easily be replicated in the GUI version by placing the values in the corresponding fields. (In fact, Tidybot GUI makes it a lot easier to experiment with the options and see what kind of effect they have.)

Default behaviour: check all the files in ‘example-sources’ and place the resulting html report files in the current directory:

tidybot example-sources

As before, but place the report files in /tmp:

tidybot --output-directory=/tmp example-sources

Do not process bad-1.html and bad-2.html (both invocations will have this effect):

tidybot --exclude=bad-1.html,bad-2.html example-sources

tidybot --exclude=bad example-sources

Do not include any Tidy messages containing the word ‘Warning’ in the reports:

tidybot --suppress=Warning example-sources

Additionally, suppress one very specific type of error message:

tidybot --suppress="Warning, Error: <spanstyle> is not recognized" example-sources

(This also illustrates why the various -from options exist — specifying too many options directly on the command-line can get unwieldy very fast.)

Instruct the underlying Tidy library to use an XML parser rather than a HTML parser:

tidybot --tidy-options="input-xml:yes" example-sources

Have Tidybot send mail after it has finished (Unix-only):

tidybot --mailto=maintainers@kronto.org \ --note "Report at: <http://foo.bar/tidybot.html>" example-sources

The --mailto option is typically used when Tidybot is running in command-line mode as an automatically scheduled application (e.g. via the Unix crontab).

Caveats

Tidybot, written in the high-level scripting language Python, is a typical ‘glue’ program that creates a more convenient user-interface and nicer output around already existing, lower-level components.

Tidybot was written because I needed its specific functionality and it was easier (and more fun) to cobble together my own utility than to try and adapt something else. Once it existed, I felt I might as well make it available to others.

By design, Tidybot offers access to only a small subset of what the underlying Tidy library is capable of. The original Tidy’s strong suit is actually fixing your legacy HTML files, converting them into shiny, valid XHTML files. In contrast, Tidybot purely exists to check existing files — it does not convert anything.

Another limitation is that Tidy is not a formal XHTML or XML validator. Its syntax checking, while extensive, does not guarantee 100% correctness. Similarly, there are certain exotic uses of XML/XHTML (namespaces are one example) which, while valid, are not supported by the Tidy library, and may therefore be incorrectly flagged as errors. Tidybot is good for keeping an eye on things and quickly spotting problems. It should never be used as a replacement for real validation checks.

There are quite a few other freeware wrappers and GUIs for TidyLib available that might suit your needs much better than Tidybot. A good starting point is the TidyLib Project’s Tidy Binaries page.

So why would you want to use Tidybot? Well, I wrote it for a medium-sized website (around 3200 static XHTML pages), that has sections being maintained by about a dozen different people, who can each upload their own files to the central repository. We run Tidybot automatically every night, and make the report pages available as part of our developer's section. This way, there is always a current validation report available, and everyone can double-check their uploads and make sure that any errors that have crept in are quickly fixed.

Credits

Source Code

Tidybot uses the following components:

Scripting language: Python, by the Python Software Foundation
XHTML/XML checking: The Tidylib C library, by the HTML Tidy Library Project
C-Python interface: The ctypes module, by Thomas Heller
TidyLib Python wrapper: µTidylib, by Cory Dodt
HTML report generation: HTMLTemplate, by Hamish Sanderson

In addition, the GUI version of Tidybot uses:

GUI toolkit: wxPython, by Robin Dunn
Error reporting: Mongoose, by Philip Lindsay

On MS Windows, Tidybot also uses:

Creating stand-alone .EXE file: py2exe, by Thomas Heller
Python Win32 Extensions: win32api, by Mark Hammond
Installer application: Inno Setup, by Jordan Russell

Finally, Tidybot’s name, design and functionality were much inspired by (but have by now also vastly deviated from) Hans de Graaff’s Checkbot, a link checker for web pages.

Infrastructure

The Tidybot source code is stored in a Subversion repository. Public repository viewing is provided by the FishEye browser, with additional statistics courtesy of mpy svn stats. Issue tracking is done with FogBugz project management software.

Bugs

If you encounter a problem with Tidybot (any problem -- even something as trivial as say a typo in the documentation!) or if you have a feature request, please consider filing a bug report. Just plain e-mail is of course also always welcome, as is posting a message to the Tidybot User Forum.

Development

Anyone can browse the Tidybot repository.

You can also do a proper subversion checkout of the latest Tidybot sources by using the command:

% svn checkout https://svn.kronto.org/tidybot/trunk/ tidybot-svn

Once you've done this, you can stay current by cd'ing into the created tidybot-svn subdirectory and saying:

% svn update

License

Tidybot is free software. It and its source code are released under the MIT License, which means they can be freely used, copied, modified, and distributed. Consult the license for full details.

Other Languages

To the best of my knowledge, Tidybot has not been translated to any other languages, but at least this documentation web page is now also available in German translation.