Maintenance Procedures

Scott Clark

The webmaster must be intimately familiar with The WDVL Website. Responsibilities include:-

integrate new content;
update navigation aids;
validate and link check;
perform log file analysis;
updating the public server;
participation in discussion list;
providing first-level user support;
running scripts and checking output;
liaising with other internet.com staff;
coordinatng with WDVL staff and authors.

Familiarity with HTML 4, CGI / Perl, UNIX, and JavaScript needed. Look around, read about us, and become acquainted with our content and style.

Almost all of our HTML files are generated by some script. Most of our procedures generate public HTML files either as by-product or end-product. Most of our scripts are Perl, some are UNIX shells. All scripts were written by us.

The central component of our procedures is a Perl script called ht; it forms the basis for our daily operations. It converts ht files into html files (statically, not at server access time) by adding boilerplate stuff like ad code and navigation menus. Whenever run, it adds an entry to a file used by our mirroring procedure. It could be modified, e.g. to lookup file path and return ad.

Daily

Respond to email from WDVL team.

Spot check new or modified pages:

Look for missing images.
Validate normally by clicking the W3C/HTML4 button at the bottom of the page.
Check new/modified pages in various browsers on various platforms with and without style sheets on.

Provide first-level user support.

Participate in the Forum.
Respond to Comments when you can with confidence, all others get an acknowledgement and told their question has been forwarded to the appropriate person

Generate the Top 100 page

In our private stats directory, t100.sh is a Unix shell script that copies the previous day's access log file to the local directory and calls the Perl programs to generate the new files.

log.pl and top.pl are run to generate the (public) Top 100 - a very useful file for us and visitors.
ref_cnt.pl and rtop.pl create the top 100 referer page (private)

Look for problems, suggest improvements, etc.

Within the same directory as the programs to generate the Top 100 is a Perl program, find_error.pl, to find bad links. This program looks for 404s (missing files) in the access log with the referring page from within the WDVL. An output file is created giving the missing URL and the URL of the WDVL page calling it.

Update Public Server

Every day (sometimes more, sometimes less) we run mirror.sh, a UNIX script that

runs make_mirror.pl which
- gets the new/updated files from a file appended to by ht and, occassionally, uw.
- generates a shell (mir_tar.sh) to create the tar file, a script (do_ht.pl) to run ht on all the new or modified files, a file (check.lst) of new files, and Update.ht, a public file to let visitors know which files have changed or appeared.
runs all the .ht files included in check.lst through the links checker (vlinks.pl).
creates the tar file, which is gzipped and copied over and untarred.
runs the do_ht.pl script to create the new html files.

Finally all new/modified pages are validated and checked in various browsers on various platforms with and without style sheets on.

Weekly

Check for broken links. To find broken links before our users do, the links checker program, vlinks.pl, needs to be run. The input file is a list of .ht or .html files within a directory (generated by the Unix find command and slightly modified) and the output file is an html file containing live links to the suspect URLs and a list of the pages calling that URL. Ideally the entire site is checked weekly, but a two week cycle is more realistic.

Update the navigation aids (e.g. the ToC, site maps)

Write entries for the Glossary.

Verify that links from the site are live and go to what they claim.

Publish New Articles.

Each week we add the new articles from our various authors. This is accomplished in the following fashion:

Check the development server for the target URL.
Change directories to the parent directory of the target dir.
Move the article's directory to the new (target) directory.
Change directories so that you are in the (target) directory.
ht all of the files by running "htt."
Change directories to place yourself at the top of the dev tree.
Start the mirror process by running "mirror.sh."
If there were broken links, visit broken.html and click on the links to see if they're actually broken. The plain text tells you which file(s) they're in.
Fix or delete them and then run "mirror2.sh."
Visit the article's URL on the public server and check it.
When all is well, put the article blurb on the home page, nearest index, lists, etc. Change to the Table of Contents dir, and run "do_map."
Copy the home page over to the public server and re-validate it.