End User's Corner - September 1997


Teri BoomsmaSite Management Solutions of the Internet Scout Project

Teri Boomsma, Webmaster


September 1997


This month we turn the End User's Corner over to Teri Boomsma, Internet Scout's Webmaster, for an in-depth view of managing the Internet Scout web site. As you will see, maintaining the server, design, and management tools that keep an over 3,000 page site running is a large task. However, it is only one of the many duties Teri performs for Internet Scout. She researches the Net for web site management and design tools, using them in the maintenance of the Internet Scout site, as well as reporting them to the community in the Net Tools section of The Scout Report (http://scout.cs.wisc.edu/scout/report/). She also maintains the Web Tools and Macintosh and Windows Webzines sections of the Scout Toolkit (http://scout.cs.wisc.edu/scout/toolkit/). [Jack Solock]


The Internet Scout Project (http://scout.cs.wisc.edu/) offers a wide variety of services with one focus in mind: to help you "surf smarter, not longer." For those unfamiliar with the project, our services include

  • the Scout Report, a weekly publication supplying the latest and best Internet resources;
  • our newest addition, the Scout Report Signpost, a browsable and searchable catalog of Internet resources from the Scout Report;
  • Net-happenings, a daily Internet announcements list;
  • the Scout Toolkit, an informational resource which addresses Internet searching, web tools, online publications, etc.; and
  • the KIDS Report, a biweekly publication of annotated Internet sites selected by students for students.

Each of these services entails its own site management needs. For example, the Scout Report can be more easily maintained with the implementation of templates and server-side includes (dynamic commands that are replaced as each page is served), whereas the Scout Report Signpost requires an interface that turns database searches into formatted web pages on the fly.

As the Webmaster for the Internet Scout Project, I am concerned with everything from server solutions to site management tools and site design. Website management is a vast and dynamic area involving numerous issues and solutions. Thus it is essential for me to keep up with the latest technology. While this is a difficult and daunting task at times, the rewards are plentiful. Due to the project's core responsibility of providing resources for research, testing various tools is one of my main focuses. This allows me to address both our site's needs and readers' technology interests. Some of my favorite informational resources include webreference.com (http://www.webreference.com/), ZDNet (http://www.zdnet.com/), and C|NET (http://cnet.com/) including C|NET services such as NEWS.COM and BUILDER.COM. These sites help webmasters tackle one of the key components of management: choosing the right combination of tools. Carefully selected tools make it easier to offer a consistent, well-organized, and eye-pleasing presentation of information, and reduce the time it takes to design, create, and maintain the site.

Currently we employ two different server solutions for our services. The Scout Report, Scout Toolkit, and KIDS Report are served via a Power Macintosh 7600 and the Scout Report Signpost via a Workgroup Server 8550. Both machines run WebStar web server software (for Mac only) from StarNine Technologies, Inc., a Quarterdeck company (http://www.starnine.com/webstar/). Net-happenings is currently served from a UNIX server at Global Internet, although it will soon be relocated to a machine here at the Internet Scout Project. Combined, the Scout Report, Scout Report Signpost, Scout Toolkit, and KIDS Report encompass more than 3,200 static HTML and PDF documents. This number would be even larger if the documents created on the fly in response to user requests through the Scout Report Signpost were included. Furthermore, the Scout Report, Scout Report Signpost, Scout Toolkit, and KIDS Report together receive an average of over 36,000 hits per day, with the Scout Report receiving approximately 50-60% of that total. Net-happenings receives 15,000 requests on average per day. Even with this relatively heavy load, technical problems and server errors are minimal due to the stability of WebStar and the Macs.

An analysis of server traffic helps Internet Scout team members plan enhancements to our services, but keeping track of this many hits can be a challenge. Luckily, there are a number of tools available for analyzing server log files. I use a great little freeware package called Analog (http://www.statslab.cam.ac.uk/~sret1/analog/) to determine the number of requests and errors. This utility, written by Dr. Stephen Turner, a postdoctoral Research Fellow at Sidney Sussex College in the University of Cambridge, is also available for DOS/Windows, RiscOS, UNIX, and VMS. It may seem surprising that we are not using a commercial program, but Analog really does the job. It offers customizable hourly, daily, weekly, monthly, domain, directory, file type, request, host, referrer, browser, and error reports. The feature that really sets it apart from the others I tested, such as ServerStat (http://www.kitchen-sink.com/) and Funnel Web (http://www.royalsoftware.com/), is its ability to analyze the data by page requests (.html files only) rather than general requests (.html, .gif, .jpg, etc.). ServerStat and Funnel Web, however, do offer nice features, such as time analysis and excellent graphical reports, respectively, which Analog lacks.

In regard to "site management" tools, there are a number of options available depending upon individual needs. A basic decision to be made is between an all-in-one program and several tools that work well together. I chose the latter. Since I prefer to have direct control over HTML code, my editor of choice is BBEdit, a Mac only product by Bare Bones Software, Inc. (http://www.barebones.com/). I have also recently decided to use a third-party BBEdit extension called CometPage for BBEdit (http://www.digitalcomet.com/) by Digital Comet, Inc. (also available for Alpha, BSDI, IRIX, NT, and Solaris). This extension adds a great deal of site management functionality to BBEdit in the form of dynamic Comet commands (Digital Comet's custom HTML extensions) such as macros, variables, insert commands, and if/then/else commands. To implement the server-side includes created with ComePage on an on-the-fly basis, we also use the CometSite Plug-in for WebStar. Together these tools are a powerful and timesaving combination. With the addition of CometPage and CometSite, the most valuable feature I am able to implement is standardized headers, footers, and other commonly used HTML segments through the use of macros. This allows for one-time changes within a given macro, rather than page-by-page manual changes too complex for BBEdit's find/replace function.

If you are looking for site management tools with capabilities such as automatic universal changes or a WYSIWYG HTML editor, you might want to look into applications such as Adobe PageMill/SiteMill (http://www.adobe.com/prodindex/pagemill/), DeltaPoint QuickSite (http://www.deltapoint.com/qs/), GoLive CyberStudio (http://www.golive.com/), Microsoft FrontPage (http://www.microsoft.com/frontpage/), and NetObjects Fusion (http://www.netobjects.com/). Keep in mind when choosing a site management tool that it is important that the tool not only meets your site's needs, but also allows your site to evolve as new technology arises. For example, you may want to choose a tool which includes an option to export or publish your site, as CometPage does. In most cases, this will allow your site to be transferred to a new site management tool more easily.

Maintaining hyperlinks is another aspect of site management, which, although extremely important, can be very time consuming. Here at Internet Scout we use a great $10 shareware for the Mac called Big Brother (http://pauillac.inria.fr/~fpottier/brother.html.en) created by Francois Pottier, a student of theoretical computer science at Projet Cristal, INRIA (The French National Institute for Research in Computer Science and Control). Checking links within a web document(s) is as easy as providing a URL (file or directory level) or opening a local file/folder on your hard drive. The output, generated before your eyes, includes a detailed summary of the links including the text which was hyperlinked, URL, source, and a diagnosis. The output can also be exported as an HTML document allowing for an interactive, convenient way to check hyperlink errors. If applicable, the diagnosis will offer an alternative URL, which can greatly reduce the time it takes to make corrections. The process time is quite fast, checking 25 or more links simultaneously. Other features Big Brother offers include configuration options for quick opening of sites and scripting to automate checking.

Effective design is the critical component that ties together everything discussed above. Here is where one's creativity as well as organizational, graphic design, and HTML skills come into play. I tend to use borderless tables and simple, crisp graphics throughout the Internet Scout site. These, along with the organizational structure of the information, offer ease of navigation for the end user. Currently, we have chosen not to use multimedia enhancements due to our emphasis on information. We have also decided not to use frames for the present. Frames can be very effective when used to creatively save space and reduce download time, and are especially nice when the user cannot immediately tell that they are frames. However, common frame frustrations include the inability to bookmark individual pages within frames and legal questions surrounding opening a site inside another site's frame. The first of these problems could be easily solved by including a link on every page that will open that same page, without frames, in the same browser window or another browser window. As far as framing another site, webmasters must be sure that links to other sites open in a new browser window. There are many resources available to help webmasters consider the breadth of design issues. If you are new to HTML, try the Beginner's Guide to HTML (http://www.ncsa.uiuc.edu/General/Internet/WWW/HTMLPrimer.html) offered by the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign. C|NET, ZDNet, and webreference.com also often offer great tutorials and informational articles on design.

The graphics included on our site are all created using Adobe Photoshop (http://www.adobe.com/prodindex/photoshop/) and Illustrator (http://www.adobe.com/prodindex/illustrator/), products of Adobe Systems, Inc.,which are also available for IRIX, Solaris, and Windows. Colors are extracted using a web-safe color palette called ColorWeb by Pantone, Inc. (http://www.pantone.com/). Doing so ensures that the colors will appear the same on all monitors whether they support 256 colors or millions of colors. As this versatility is very important, MacOS 8 now includes an HTML color palette as part of its Color Picker options.

The HTML documents on our Mac servers draw on a pool of just over 200 graphics, most of which are navigational and interface-related. As a courtesy to our readers, particularly those with lower bandwidth connections, the Scout Report, Scout Toolkit, and KIDS Report are also available in text-only versions. When deciding on the use of graphics, one must weigh several issues. On one hand, graphics can be used very effectively in the site design to organize information and eliminate unutilized page space. On the other hand, a highly graphical page is very frustrating for the end user with a 14.4 or 28.8 Kbps modem. This problem can be somewhat relieved by GIF Wizard (http://useast.gifwizard.com/), provided by Raspberry Hill Publishing, Inc.. It's a great online utility that can reduce the file size of graphics by up to 90%.

There is a great deal of information regarding site management on the Internet. The best way to begin is to visit some of the resources listed above. Our site also offers information on site management and various other tools. To best access this information simply search or browse the Scout Report Signpost (http://www.signpost.org/), browse the Scout Report Bimonthly Compilations under Network Tools (http://scout.cs.wisc.edu/scout/report/bimonth/), look through previous issues of the Scout Report under Network Tools (http://scout.cs.wisc.edu/scout/report/archive/), and/or visit the Web Tools section of the Scout Toolkit (http://scout.cs.wisc.edu/scout/toolkit/webtools/). Best of luck with your site. : )


InterNIC News

This article originally appeared as part of the End User's Corner, a featured column of InterNIC News, which was published monthly by Network Solutions, Inc. and InterNIC from May 1996 through March 1998. As of April 1998, End User's Corner will be published by the Internet Scout Project.


Copyright Susan Calcari and the University of Wisconsin Board of Regents, 1994-1998. Permission is granted to make and distribute verbatim copies of the End User's Corner provided the copyright notice and this paragraph is preserved on all copies. The Internet Scout Project provides information about the Internet to the US research and education community under a grant from the National Science Foundation, number NCR-9712140. The Government has certain rights in this material.

Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the University of Wisconsin - Madison or the National Science Foundation.


Internet Scout

A Publication of the Internet Scout Project

Comments, Suggestions, Feedback

Use our feedback form or send email to scout@cs.wisc.edu.

© 1997 Internet Scout Project

Information on reproducing any publication is available on our copyright page.