JANET Bandwidth Management Advisory Service

BMAS Home | About BMAS | Bandwidth Management | Technology Papers and Guidance | Product Evaluations | Calls for Partipation

 

Web Caching

Here you will find a wealth of resources about the subject of caching; an effective method of bandwidth management. Learn about what caching is and why it could be beneficial to your organisation, as well as information on some of the solutions available today.

Introduction to Web Caching

The aim of this document is to clarify what caching is and what its advantages are. Please also refer to the "Caching solutions" section of the website where you will find details of caching hardware and software solutions, together with some guides for popular caching software.

What is a web cache?

In Worldwide Web terms, a cache is a place where temporary copies of web objects are kept. Normally when a user accesses the internet, web object requests go directly to the site of origin with the web object then being returned back to the user.


With a local cache in operation, user web object requests go via the local cache which then retains a copy of the said web object. This results in all subsequent requests for the same object being fulfilled from the local cache instead of from the site of origin. This process of web caching minimises the amount of times identical web objects are transferred from remote websites by retaining copies of requested URLs in a cache.


A web cache can be installed utilising both software and / or hardware, and can run on various different platforms. The BMAS website contains details of technical specifications and caching solutions which should provide a wide enough range of information to suit most budgets and areas of expertise.

What are the advantages of creating a local web cache?

With a local cache in operation subsequent requests for previously cached URLs result in the cached copy of the object being returned to the user; creating little or no extra network traffic, improving efficiency and reducing waiting time. This can be particularly effective when you consider the number of requests sent to the same popular websites.


Without a cache all your users requests go directly to the remote site generating web traffic and using up bandwidth.

Top of page

Technical Specification

There are many different solutions available to you if you need to set up your own local cache.

The purpose of this page is to provide you with an outline of what is required to create a local cache. For more detailed information about the majority of caching software packages available (listed by platform) and proprietary hardware available, please see the "Caching Solutions" section below.

We normally recommend that you use a machine specification similar to the one stated below together with software which will run on the operating system with which you are most familiar, especially if you are new to caching. Larger or smaller scale institutions may need to differ the specification.

Please see the "Technical Guides" section for configuration information for the most popular caching software, i.e.Squid (UNIX systems) and MS Proxy Server/Internet Security and Acceleration Server (Windows systems)

Hardware specifications -

700Mhz PIII CPU
6 x Ultra2 LVD SCSI cache disks
512 MB RAM
We recommend that you run your system on about 512MB RAM (certainly no less than 128) with the fastest
SCSI cache disks you can afford.

Operating systems

Caching software is available for the Apple Macintosh, Novell, Sun Solaris, most flavours of UNIX, and Windows operating systems. See "Caching Solutions" for more details.

Top of page

 

Caching Solutions - software and proprietary hardware

Below we have listed all the main caching software solutions, sorted by platform. Many of these packages are actually web server packages with inherant caching facilities, although some, such as Squid, are dedicated caching packages.

Dedicated Caching software - Unix platform

The FULL Squid configuration guide is available on the Squid support website in HTML (compressed tar file) and PDF formats.

The full Squid Users guide can be found on the Squid website.

Software Solutions:- Windows only

Software solutions Multi-platform: - Apple Mac, Netware, Solaris, Unix and Windows

Top of page

Caching Resources

Below is a list of caching resources which should be useful. The list has links which cover all aspects of caching.

  • Caching.com - Excellent resource with lots of information and links about proprietary caching devices. Includes links and information about many of the main commercial caching products such as Microsoft ISA server, Lucent Technologies 'imminet Web Cache' etc
  • Caching the web with Linux - David Guerrero. A good all round guide to setting up a cache on Linux
  • Cache Now! is a campaign to raise awareness about web caching and what individuals and organisations can do to save time, money & bandwidth for themselves and others.
  • IRCache Home Page - details the IRCache project which was formed in 1995, and has many useful resources.
  • O'Reilly Books - Web Caching by Duane Wessels, available to buy online from the O'Reilly website
  • The Squid Home page  Go here for the latest version of Squid, as well as everything you need to know about installing and configuring Squid caching software.
  • Squid Users Guide   A useful guide to getting Squid up and running, and pointers as to problems to expect.
  • Terena (the Trans-European Research and Education Networking association) - has many caching links which can be found through a searchable front end.
  • Web cache.com - A fairly up to date and comprehensive list of caching resources
Top of page

Non-cacheable Objects

The Internet provides access to many different types of file - some of these do not lend themselves to being cached. In order to use a local cache effectively, there are some web objects that should not be cached - a list of these current types of objects is shown below.

 

Caching software can be easily configured to route directly to the URL of origin for web objects such as those described below.

 

Nb. There are also some URLs which do not allow caching for various reasons. A list of such sites can be made available in the form of an Access Control List for Squid users.

 
WEB OBJECT EXAMPLE DESCRIPTION
Dynamically generated web objects .php . asp Web objects which are returned from the result of input into a database
SSL - a.k.a. https Sites beginning "https" Sites using Secure Socket Layer (SSL - aka https) normally bypass the cache. This is primarily for security reasons, as it is not recommended that sites containing personal data be cached.
If an https site is goes through the cache, the user often sees an error message indicating that the server or dns cannot be found.
Real Audio and video    
Sites that use IP authentication Sites such as electronic journals etc Requests have to go directly to the server of origin to enable authentication. If these sites were cached, your request would arrive via another machine resulting in the IP address being unidentified by the end server.
Local Sites or those within the academic domain (.ac.uk) Local and .ac.uk domain sites should not be cached to reduce latency to the request. Cost efficiency issues are not an issue with local sites.
Top of page

Caching software guides

Top of page

Frequently Asked Questions

  • Squid cache configuration FAQ - this page explains some of the problems that you may encounter when setting up your own local cache. NB This page refers to the use of Squid software. For other caching software solutions, please see the 'caching' section and follow relevant links.
  • Cachemaster FAQ - this page can help you resolve one the most common student access problems that local cachemasters come across.

 

 

Any problems, comments or suggestions regarding this page, please e-mail the BMAS service manager bmas@ja.net

 

 

 

 

Privacy Policy | Web Admin
© The JNT Association,
JANET(UK), Lumen House, Library Avenue
Harwell Science and Innovation Campus, Didcot, Oxfordshire
OX11 0SG