The following article appeared in the Sydney Morning Herald.

A New King Of The Block

Link: http://smh.com.au/articles/2004/10/11/1097406487197.html

“Spam, spam, spam.” As the old Monty Python song goes, so goes the modern world.

The first spam message was sent by now-defunct computer company Digital in 1978 announcing the demonstration of a new computer system. Unsolicited emails of various types became common over the next 15 years, but it was only with the widespread use of the Web for commercial purposes in the 1990s, and the explosion in the use of email, that spam became widespread.

Spam now comprises more than 50 per cent of all emails. Internet pioneer Brad Templeton has written an interesting little history of the origins of the term, and the first spam messages, at www.templetons.com/brad/spamterm.html.

Spam is such a problem that some people have forecast the death of email. It is not just its sheer volume, or its often objectionable content, but the fact that it is often used to carry viruses and other nasties. And it’s expensive – email security firm MessageLabs estimates the annual cost to a large Australian corporation or government department over $1000 a user. Similar studies elsewhere have shown similar results.

Add downtime and productivity losses from spam-borne viruses and the figures become very large. British consultancy mi2g puts the worldwide damage from spam in the first half of 2004 alone at nearly $2 billion, from more than a trillion spam messages.

There have been a number of suggestions and measures made to reduce the problem. They can be divided into a number of categories. The first major division is between legislative and technical approaches.

Many countries, including Australia and the USA, have introduced anti-spam laws prohibiting certain kinds of unsolicited emails. These measures have been largely ineffectual, because spam is an international problem. The largest source of spam messages (around 30-40 per cent) is the USA, but there have been very few successful prosecutions. The US laws are in any case not very rigid, and any large scale spammer can easily go elsewhere.

Australia's laws are generally regarded as among the best in the world, but with less than one per cent of the world's emails originating in this country they are of little use. The head of the Australian Communications Authority, Bob Horton, recently chaired a meeting in Geneva of the International Telecommunications Union that proposed standardised international anti-spam laws.

Most experts on the subject believe the legislative approach will work only in conjunction with effective technical approaches. Such approaches can themselves be divided into two main types – filtering and blocking.

Filtering software picks up on certain word patterns or other characteristics of email to identify it as spam. We have all received spam with words like “Viagra” spelt in strange ways, examples of attempts to get round spam filters.

Filters can work at the client level or at the server level. Client-level packages like iHateSpam are popular, and the latest version of Microsoft Outlook has a similar filter built in. These filters can also be applied at the level of an organisation's mail server, or at the ISP (Internet Service Provider) level.

Some ISPs or mail server administrators have gone to the extent of blocking emails from particular domain names that are identified as common spam sources. This can be helpful, but because many spammers use various clever methods to shield their email addresses, and even to hijack (or “spoof”) legitimate ones, it is not totally effective.

Microsoft has proposed a system called “Sender ID Framework” for identifying the senders of emails. It goes some way towards helping identify spoofed domain names, but it depends on a central authority to administer it. The Internet, by its nature, has no such authority, and AOL recently rejected an approach by Microsoft to cooperate on such a system because it would have given Microsoft too much power.

The other method is blocking. I have been trialling a blocking system, from a small Australian company called TotalBlock (www.totalblock.net) for over a month now, and it's been very effective. The approach is completely different to filtering.

All my email goes to the TotalBlock server, which allows through only mail from senders I have previously authorised. This is my entire Outlook address book, and anyone I send emails to is added automatically. Any other mail is blocked, and a sender is challenged with a simple question. If they answer it, the mail is delivered, if not I never see it. Send me an email (my address is at the bottom of this column) and you'll see how it works.

I can at any time go to the server and see what emails have been blocked. I did this every couple of hours at first, worried that I was missing important mail. I would routinely get well over 100 spam messages a day, and I kind of missed them at first. Now it's out of my life.

There are some problems with the blocking/authorisation approach. If you are sent an automatically generated email – such as a password when you subscribe to a newsletter or buy some software – it will be blocked, and you have to go to the server to unblock it. And I probably miss a few messages.