End User's Corner - February 1998

Jack SolockMinding the Web to Keep Current on the Internet

Jack Solock, Special Librarian

February 1998

With so much information on the Internet, it is a difficult task to find quality sites in your field of interest. In previous articles, as well as in the Scout Report (http://scout.cs.wisc.edu/scout/report/) and Scout Toolkit (http://scout.cs.wisc.edu/scout/toolkit), we have discussed methods and provided URLs and site descriptions to help users in this somewhat daunting task. Another problem for users, once they have mastered the art of finding quality sites, keeping track of additions to these sites.

This task is made difficult because Internet content evolves and changes so rapidly, and that even though the amount of quality information on the Net is a tiny proportion of all the information there, it is still massive, much larger than even the largest set of bookmarks can handle.

There are, however, techniques by which both the savvy and novice user can stay current. This article will give you a few examples.

In order to understand what is to come, it is necessary to understand the concepts "pull" and "push." Pull, in terms of the Internet, means that the user must go out onto the Net, by whatever access protocol, and find relevant content. Once found, he or she uses the protocol, whether it be the Web, gopher, telnet, or FTP, to "pull" the desired information to the desktop. Most often, this takes the form of searching sites like indexes, subject catalogs, or subject guides to find appropriate resources. When the user becomes savvy enough, a select list of appropriate known quality metasites can be used to even greater advantage. In any event, it takes action on the part of the user to find content. That action takes energy. Finding information is a fairly labor intensive process this way. Keeping track of updates to this information is extremely difficult this way. When is the information updated? How is it updated?

Push, on the other hand, is a mechanism whereby the user is passive. Filtered information simply appears on the desktop, whether it be in the form of email updates, or a piece of software such as the PointCast Network (http://www.pointcast.com/). Once an information profile of some sort is set up, the user simply sits back and waits for information to appear. Much less energy is expended in this process, although initiating an information push mechanism can sometimes be difficult for novice users.

The process of keeping current, as might be expected, is a combination of push and pull. The more push one can exploit, the easier it will be.

We will point out just a few methods of keeping current. Once the kernel of the idea is understood, it will be fairly easy to incorporate it into use.


The key to pull is to turn it into push. How is that accomplished? As with almost anything on the Internet, there are several sites that "mind" desired sites for the user. One of the best is NetMind's URL-Minder (http://www.netmind.com/html/url-minder.html). The user simply enters his or her email address and a list of URLs to be watched. A simple demonstration of this will show its power. If you are interested in US government documents, for example, you know that one of the largest repositories of them is the Government Printing Office. Conveniently, the GPO Pathway Services site maintains a Browse Electronic Titles (http://www.access.gpo.gov/su_docs/dpos/btitles.html) page, which contains links to thousands of government documents by agency. It also maintains a What's New section that is available as a link from the main page. It is this section that is of interest to you if you want to keep current. The easiest way to keep track of new government documents available from the GPO is to URL-Mind this What's New (http://www.access.gpo.gov/su_docs/dpos/new.html) page. The page has a simple structure, changing weekly. You could, of course, bookmark the page, but then you probably would forget to check it. URL-Minder will send you an email message telling you when the page changes. In this way, you can easily keep track of new government documents.

Seems simple, doesn't it? Well, not quite so simple as you think. First, you must find the correct page that will give you the "What's New" information at a site. Second, the URL of the What's New page must remain stable. In the case of Pathway Services, you might ask why not just URL-Mind the New Additions listing? If you look closely at the URL of the New Additions page, you will see that it ends in a date. URL-Minding this page will be useless, because it will not change. Next week's What's New additions will be on a separate, new page that ends in a different date.

Another problem is that page changes do not necessarily reflect relevant content changes. While URL-Minder does have an advanced form that lets the user track changes to parts of a page, this can still be a tricky problem. An example is the American Memory (http://lcweb2.loc.gov/ammem/) site at US Library of Congress (LOC). A quick look at the design of this page shows that it isn't very easy to "mind" additions to the collection. When a new collection is added, old ones are not necessarily removed, and additions are not incorporated on their own page. Notice also that the page is also the home to "Today in History," another LOC Web service, which means that URL-Minding this page will guarantee that you will receive lots of changes that are not related to new American Memory exhibits. In this case, the solution is simple, but a bit subtle. If you click on the text only American Memory Site (http://lcweb2.loc.gov/ammem/amhometxt.html), you quickly realize that minding this page will inform you of additions to the American Memory collection, without informing you every time "Today in History" changes.

This is more an art than science, and it takes considerable trial and error, as well as study of page design to exploit this tool to its utmost. Don't be discouraged if you find yourself minding a page and receiving changes in its design rather than content. Study the page and related pages and try to determine if there is a better page to mind. Minding tools can be frustrating, but they can also be extremely valuable.

Another pull method is the well known bookmark. When you find the proper What's New page, simply bookmark it. Of course, it then becomes your responsibility to go back to the site again and again.


While "minding" is a useful technique, even more useful is direct pushing of changes in content. There has been much talk recently about push technology and software, but the best technology for this at present is still email. URL-Minder is an example of how email helps turn pull into push. Direct push depends on the site. If you find a site that you are interested in and it offers you email updates, by all means take advantage of this feature. It allows you to make the Net work for you. A few examples will show this.

The US Census Bureau (http://www.census.gov/), is a prolific provider of information. If you wanted to keep current on everything new at this site, you would have to mind many pages. However, the Bureau has provided an email notification service (http://www.census.gov/mp/www/subscribe.html), a selection of any of four different notifications of various additions to Census Bureau services.

The US National Science Foundation (NSF) (http://www.nsf.gov/) provides a Custom News Service (CNS) (http://www.nsf.gov/home/cns/start.htm) that allows you to specify exactly what new NSF information you want to be notified of, based on a very specific information profile. CNS will even create a user web page that will provide those updates in addition to, or instead of email notification. CNS is one of the most powerful current awareness services available on the Internet.

These are two examples of email push services that deliver newly available content at a site or agency. There are also metasites that allow you to track additions via email, and this is an excellent service when available. One example is BUBL LINK (Bulletin Board for Libraries: Libraries of Networked Knowledge) (http://bubl.ac.uk/link/), a product of Strathclyde University's (Scotland) Andersonian Library. BUBL LINK maintains a large and growing collection of annotated subject resources arranged by Dewey Decimal Classification. Interested users can be notified of updates via the lis-link (http://www.mailbase.ac.uk/lists/lis-link/) mailing list. Site additions are made available via email approximately every ten days. Another example is FindLaw's (http://www.findlaw.com/) FindLaw Wire, which allows users to keep track of new site additions to this massive law related metasite.


By using these simple techniques of push-pull current awareness, you can make the Internet work for you over and over again, with much less effort on your part. The above examples are just a few of the ways you can keep current. There are hundreds of others. Always analyze sites closely to see if there is an email push notification service available. If there is, use it. If not, analyze the site again to see just what pages you might want to "mind." Just a little bit of work on your part can pay off many times over in helping you "mind" the Net and keeping current.

InterNIC News

This article originally appeared as part of the End User's Corner, a featured column of InterNIC News, which was published monthly by Network Solutions, Inc. and InterNIC from May 1996 through March 1998. As of April 1998, End User's Corner will be published by the Internet Scout Project.

Copyright Susan Calcari and the University of Wisconsin Board of Regents, 1994-1998. Permission is granted to make and distribute verbatim copies of the End User's Corner provided the copyright notice and this paragraph is preserved on all copies. The Internet Scout Project provides information about the Internet to the US research and education community under a grant from the National Science Foundation, number NCR-9712140. The Government has certain rights in this material.

Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the University of Wisconsin - Madison or the National Science Foundation.

Internet Scout

A Publication of the Internet Scout Project

Comments, Suggestions, Feedback

Use our feedback form or send email to scout@cs.wisc.edu.

© 1998 Internet Scout Project

Information on reproducing any publication is available on our copyright page.