Introduction To Filesharing

HomePage | RecentChanges | Preferences

Note: in the following document 'server' means the computer that has the file(s) you want and 'client' means the computer you want the file on.

'Classic' Filesharing Apps

The simplest file transfer programs simply allow one person to send a file to another. Most Instant Messenger programs allow you to do this, as does email. You don't have to worry about strangers using your bandwidth, but you can't search through the server's files to see what's available.

One step up in functionality are systems that allow searching and requesting of files. There are simple email systems that allow this, but at a more advanced level there is FTP and any networked operating system. These allow the client to open a folder (or set of folders), scroll through them to find the required file and then copy it to their own computer. From a security standpoint, these are easily locked down to only allow certain users to have access, but the site providing the files is easily identified. Searching of an individual server's files is simple, providing that the files are organised into a folder structure (i.e. band/album), but there is no index. Searching across multiple users at once is impossible.

Napster kickstarted the file-sharing revolution by adding a single function to this - the ability to search across thousands of users at once. Individual servers reported the files they were sharing to a central database and clients looking for a file searched that database rather than searching each individual server. Advantage: much easier to find the file you want. Disadvantages: single point of failure (through technical problems or being sued).

GNUtella took Napster and decentralised it. Rather than a single central point being responsible for indexing, requests for files were passed from point to point through the network, with each server responding individually if it had the file. Advantages: no single point of failure, no single entity to sue. Disadvantages: Network was easily swamped by traffic if it got too large (because every client talked to every other client), more freedom by individual servers to return faulty results (causing network disruption).

Kazaa, eDonkey and Grokster took this concept and partially re-centralised it. Individual servers with sufficient bandwidth and processor power were automatically promoted to being Supernodes. These behaved like the centralised Napster systems but had the advantage that if one was taken away another just popped up. They also added file-swarming. This meant that you could download different parts of a single file from different servers, reducing the load on each one and speeding up the downloads. This is the most popular kind of file-sharing system at the moment. Advantages: no single point of failure, quicker searches than fully decentralised model, no single entity to sue. Disadvantages: Supernodes are still high profile, allowing them to be sued individually.


BitTorrent kept the swarming functionality, but discarded everything else. This was intended to allow servers to host large files without incurring high bandwidth costs - as clients downloaded each part of the file they became servers for that part. This distributed the load across multiple servers, meaning that the main server wasn't crippled by the load. In fact, the more people were downloading the file, the less the original server had to do. BitTorrent has become very popular for sharing large files (movie files, operating systems), and is the only recent file-sharing technology to be substantially used for non-copyright-infringing purpouses.

Freenet decided to ignore all the searching problems in favour of secrecy and resistance to intervention. Freenet servers cannot tell you what files they contain, nor do they generate indexes. Instead, whenever a file is added it is assigned a unique ID. To get the file, the client must know the ID. When a file is requested from a Freenet server, if it has all the parts of it, it will simply return the file, otherwise it will ask all of its neighbours. They, in turn will ask their neighbours - but they will _not_ tell them that they are asking on behalf of another server. This means that no server can betray another. In addition, it is impossible to prove that any Freenet Server operator intended to have certain items on their server - it may have been requested by another server and was simply being passed along. Freenet has failed to really catch on, as it doesn't fulfill the requirement of most file-sharers - the ability to type "Britney Spears" into a box and then choose a song to listen to. Advantages: Utterly secret. Very, very hard to block. Disadvantages: All indexes are handcrafted (and frequently passed across Freenet itself, making them hard to get hold of in the first place). File transfers are slow, as they may have to cross multiple intervening servers in order to get to the client.

WASTE was another system from the original creator of Gnutella - it allows for the creation of private, encrypted P2P networks. Each member of the network has access to the encryption keys of the other members - nobody outside the network can tell what the members are sharing, but everyone inside has access to each other's files. Theoretically, this will allow for for the addition of any other features available in other P2P systems (limited only to the members of the network). It has, however, failed to take off for most users, as they frequently want files that none of their friends have. It also has problems with some of its default settings that can lead to you adding one friend to your network and suddenly gaining their 4000 other friends.

A Few Notes

Some features are incompatible with other features. A decentralised system means that searches take longer, but individual servers are less likely to be taken down (and the system as a whole cannot be taken down). Complete Proxying (where you connect directly only to trusted machines, and all traffic to other machines is carried through those machines) is extremely secure and anonymous, but slows communication down massively.

Imagine, for a moment, that the internet is a bar, filled with both rebels and the secret police. The rebels want to talk to each other, but don't want the police to know what they're talking about. You could stand on the table and shout, but you'd quickly be arrested by the police. You could only tell your two close friends, trusting them to pass the message on (but not that you said it), but that would take hours. Or you could join in huddles of people that looked like rebels, speaking freely with them, and hoping there were no police in that particular huddle.

The choice you make would depend on how paranoid you were about the police, how important the information was to you, the penalties for being caught and how patient you were. So far, most people have plumped for the third option - hoping that there aren't too many policemen about, and being too impatient to use any other methods. Crackdown by the police might cause people to move to more paranoid methods of communication, or to decide that those methods are more annoying to use than the information warrants. Alternatively, there might be a change of government, meaning the rebels don't need to communicate secretly at all.

But I wouldn't hold your breath.

Creative Commons License
This work is licensed under a Creative Commons License.

HomePage | RecentChanges | Preferences
This page is read-only | View other revisions
Last edited October 30, 2004 7:11 am by (diff)