\maketitle\r
\r
\begin{abstract}\r
-The Internet has become an cost-effective\r
-vehicle for software development and release, particular in the free software society.\r
+The Internet has become a cost-effective\r
+vehicle for software development and release, particular in the free software community.\r
Given the free nature of these software, there are often a number of users\r
motivated by altruism to help out with the distribution, so as to promote the healthy development\r
of this voluntary society. It is thus naturally expected that peer-to-peer distribution can be implemented,\r
the volunteers.\r
\r
Unfortunately, this application scenario has many unique characteristics, which\r
-make a straightforward adoption of such existing peer-to-peer systems for file sharing as BitTorrent suboptimal. In particular,\r
+make a straightforward adoption of existing peer-to-peer systems for file sharing such as BitTorrent suboptimal. In particular,\r
a software release often consists of a large number of packages, which are difficult to be distributed individually, but the archive is\r
too large to be distributed in its entirety. The packages are also being constantly\r
-updated by the loosely-managed developed; yet the interest in a particular version of a package can be very\r
+updated by the loosely-managed developers, and the interest in a particular version of a package can be very\r
limited depending on the computer platforms and operating systems used.\r
\r
-In this paper, we propose a novel peer-to-peer assisted distributor design that\r
+In this paper, we propose a novel peer-to-peer assisted distribution system design that\r
addresses the above challenges. It enhances the existing distribution systems by providing compatible and yet more efficient downloading and updating services\r
for software packages. Our design leads to \texttt{apt-p2p}, a practical implementation that extends the popular \texttt{apt} distributor. \texttt{apt-p2p} has been used in conjuction with Debian-based distribution of Linux\r
software packages and will also be available in the next release of Ubuntu. We have addressed the key design issues in \texttt{apt-p2p}, including indexing table customization,\r
response time reduction, and multi-value extension. They together ensure\r
-that the altruistic users' resources be effectively utilized and thus significantly reduce the currently\r
-sheer bandwidth requirements of hosting the software, as confirmed by our existing real user statistics.\r
+that the altruistic users' resources are effectively utilized and thus significantly reduces the currently\r
+large bandwidth requirements of hosting the software, as confirmed by our existing real user statistics.\r
\end{abstract}\r
\r
%%%%%%%%%%%%%%%%%%%%%%%%%%% Section %%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
\section{Introduction}\r
\label{intro}\r
\r
-With the widespread penetration of broadband accesses, the Internet has become an cost-effective\r
+With the widespread penetration of broadband access, the Internet has become a cost-effective\r
vehicle for software development and release. This is particularly true for the free software\r
-society whose developers and users are worldwide distributed and work asynchronously. The ever increasing power of\r
-modern programming languages, computer platforms, and operating systems has made these software extremely large and complex,\r
-which are often divided into a huge number of small packages.\r
+community whose developers and users are distributed worldwide and work asynchronously. The ever increasing power of\r
+modern programming languages, computer platforms, and operating systems has made this software extremely large and complex,\r
+though it is often divided into a huge number of small packages.\r
Together with their popularity among users,\r
-an efficient and reliable management and distribution of these software over the Internet has become a challenging task.\r
+an efficient and reliable management and distribution of these software packages over the Internet has become a challenging task.\r
\r
-The existing distributor for free software are mostly based on the client/server model, e.g.,\r
+The existing distribution for free software is mostly based on the client/server model, e.g.,\r
the Advanced Package Tool (\texttt{apt}) for Linux, which suffers from the well-known bottleneck problem.\r
-Given the free nature of these software, there are often a number of users\r
+Given the free nature of this software, there are often a number of users\r
motivated by altruism to help out with the distribution, so as to promote the healthy development\r
of this voluntary society.\r
We thus naturally expect that peer-to-peer distribution can be implemented in\r
-this context, which scale well with large user bases and can easily explore the network resources made available by\r
+this context, which will scale well with the currently large user bases and can easily explore the network resources made available by\r
the volunteers.\r
\r
Unfortunately, this application scenario has many unique characteristics, which\r
-make a straightforward adoption of such existing peer-to-peer systems for file sharing as BitTorrent suboptimal. In particular,\r
+make a straightforward adoption of existing peer-to-peer systems for file sharing such as BitTorrent suboptimal. In particular,\r
there are too many packages to distribute each individually, but the archive is\r
too large to distribute in its entirety. The packages are also being constantly\r
-updated by the loosely-managed developed; yet the interest in a particular version of a package can be very\r
+updated by the loosely-managed developers, and the interest in a particular version of a package can be very\r
limited. They together make it very difficult to efficiently create and manage torrents/trackers. The random downloading nature of BitTorrent-like systems is also different from the\r
-sequential order used in existing software package distributors. This in turn suppresses interaction with users\r
+sequential order used in existing software package distributions. This in turn suppresses interaction with users\r
given the difficulty in tracking speed and downloading progress.\r
\r
-In this paper, we propose a novel peer-to-peer assisted distributor design that\r
+In this paper, we propose a novel peer-to-peer assisted distribution system design that\r
addresses the above challenges. It enhances the existing distribution systems by providing compatible and yet more efficient downloading and updating services\r
for software packages. Our design leads to the development of \texttt{apt-p2p}, a practical implementation based on the Debian \footnote{Debian - The Universal Operating System: {\it http://www.debian.org/}} package\r
distribution system. We have addressed the key design issues in \texttt{apt-p2p}, including indexing table customization,\r
response time reduction, and multi-value extension. They together ensure\r
-that the altruistic users' resources be effectively utilized and thus significantly reduce the currently\r
-sheer bandwidth requirements of hosting the software.\r
+that the altruistic users' resources are effectively utilized and thus significantly reduces the currently\r
+large bandwidth requirements of hosting the software.\r
\r
\texttt{apt-p2p} has been used in conjuction with Debian-based distribution of Linux\r
-software packages and will also be available in the next release of Ubuntu. We have evaluated our currently deployment to\r
+software packages and will also be available in the next release of Ubuntu. We have evaluated our current deployment to\r
determine how effective it is at meeting our goals, and to see what\r
effect it is having on the Debian package distribution system. In particular, our existing real user statistics\r
have suggested that it responsively interacts with clients and substantially reduces server cost.\r
The rest of this paper is organized as follows. The background and motivation are presented in Section~\ref{situation}, including an analysis of BitTorrent's use for this purpose in Section~\ref{bittorrent}. We propose\r
our solution in Section~\ref{opportunity}. We then detail our sample\r
implementation for Debian-based distributions in Section~\ref{implementation},\r
-including an in-depth look at our DHT\r
-customizations in Section~\ref{custom_dht}. The performance of our implementation is evaluated in Section~\ref{analysis}. We examine some related work in Section~\ref{related}, and then\r
+including an in-depth look at our system optimization\r
+in Section~\ref{custom_dht}. The performance of our implementation is evaluated\r
+in Section~\ref{analysis}. We examine some related work in Section~\ref{related}, and then\r
Section~\ref{conclusions} concludes the paper and offers some future directions.\r
\r
%%%%%%%%%%%%%%%%%%%%%%%%%%% Section %%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
\texttt{pkgtools}, and FreeBSD has a suite of command-line tools,\r
all of which download these tarballs from web servers.\r
\r
-Similar tools have been used for other software packages. CPAN\r
-distributes software packages for the PERL\r
+Similar tools have been used for other types of software packages. CPAN\r
+distributes packaged software for the PERL\r
programming language, using SOAP RPC requests to find and download\r
files. Cygwin provides many of the\r
standard Unix/Linux tools in a Windows environment, using a\r
\r
Given the free nature of this software, there are often a number of users\r
motivated by altruism to want to help out with the distribution.\r
-This is particularly true considering that many of this software is used by\r
+This is particularly true considering that much of this software is used by\r
groups that are staffed mostly, or sometimes completely, by\r
volunteers. They are thus motivated to contribute their network resources, so as to promote the healthy development\r
of the volunteer community that released the software.\r
-We also naturally expect that peer-to-peer distribution can be implementation in\r
-this context, which scale well with large user bases and can easily explore the network resources made available by\r
+We also naturally expect that peer-to-peer distribution can be implemented in\r
+this context, which will scale well with the currently large user bases\r
+and can easily explore the network resources made available by\r
the volunteers.\r
\r
\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{apt_p2p_simulation-size_CDF.eps}\r
+\includegraphics[width=0.80\columnwidth]{apt_p2p_simulation-size_CDF.eps}\r
\caption{The CDF of the size of packages in a Debian system, both\r
for the actual size and adjusted size based on the popularity of\r
the package.}\r
current Debian distribution. While 80\% of the packages are less than\r
512~KB, some of the packages are hundreds of megabytes. The entire\r
archive consists of 22,298 packages and is approximately 119,000 MB\r
-in size. Most of the packages are to be installed in any computer environment, but there are\r
+in size. Many of the packages are to be installed in any computer environment, but there are\r
also OS- or architecture-specific packages.\r
\r
\subsubsection{Package Updates}\r
or the distributor updating their packaging of the\r
software to meet new requirements. Even if the distributor\r
periodically makes \emph{stable} releases, which are snapshots of\r
-all the packages in the archive at a certain time, updates are still\r
+all the packages in the archive at a certain time, later updates are still\r
released for security issues or serious bugs.\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{size-quarter.eps}\r
+\includegraphics[width=0.9\columnwidth]{size-quarter.eps}\r
\caption{The amount of data in the 119,000 MB Debian archive that is\r
updated each day, broken down by architecture.}\r
\label{update_size}\r
\subsubsection{Limited Interest}\r
\r
Finally, though there are a large number of packages and a large number of\r
-users, the interest in a particular version of a package can be very\r
+users, the interest in a particular package, or version of a package, can be very\r
limited. Specifically, there are core packages that every user has to download, but most\r
packages would fall in the category of optional or extra, and so are\r
interesting to only a limited number of people.\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{apt_p2p_popularity-cdf.eps}\r
+\includegraphics[width=0.80\columnwidth]{apt_p2p_popularity-cdf.eps}\r
\caption{The CDF of the popularity of packages in a Debian system.}\r
\label{popularity_CDF}\r
\end{figure}\r
some kind of indication of speed and completeness for users to\r
monitor. Since previous client-server downloads occurred in a sequential\r
fashion, the package management software also measures the speed\r
-based on sequential downloading. To offer comparable user experience, it is naturally to expect that\r
+based on sequential downloading. To offer comparable user experience, it is natural to expect that\r
the new peer-to-peer solution be reasonably responsive at retrieving packages,\r
-preferably enabling sequential downloading order, too.\r
+preferably in a sequential downloading order too.\r
\r
\subsection{Why BitTorrent Doesn't Work Well}\r
\label{bittorrent}\r
quite difficult given the unique characteristic of free software packages.\r
First, there is no obvious way to divide the packages into torrents.\r
Most of the packages are too small, and there are too many packages\r
-in the entire archive to create individual torrents for each one.\r
+in the entire archive, to create individual torrents for each one.\r
On the other hand, all the packages together are too large to track\r
efficiently as a single torrent. Hence, some division of the archive's\r
packages into torrents is obviously necessary, but wherever that\r
content. In addition, a small number of the packages can be updated every\r
day which would add new files to the torrent, thereby changing its\r
\emph{infohash} identifier and making it a new torrent. This will\r
-severely fracture the download population, even though peers in the\r
+severely fracture the download population, since even though peers in the\r
new torrent may share 99\% of the packages in common with peers in the\r
-old torrent.\r
+old torrent, they will be unable to communicate.\r
\r
Other issues also prevent BitTorrent from being a good solution to\r
this problem. In particular, BitTorrent's fixed piece sizes (usually 512 KB) that disregard file\r
\r
On the other hand, there are aspects of BitTorrent that are no\r
longer critical. Specifically, with altruistic peers\r
-and all files being available without uploading, incentives to share\r
+and all files being available to download without uploading, incentives to share\r
become a less important issue. Also, the availability of seeds are\r
-not critical either, as the mirror sites serve in that capacity\r
-already.\r
+not critical either, as the serves are already available to serve in\r
+that capacity.\r
\r
%%%%%%%%%%%%%%%%%%%%%%%%%%% Section %%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
\r
\section{Peer-to-Peer Assisted Distributor: An Overview}\r
\label{opportunity}\r
\r
-We now present the design of our peer-to-peer assisted distributor for free software package releases and\r
+We now present the design of our peer-to-peer assisted distribution system for free software package releases and\r
updates. A key principle in our design is that the new functionalities implemented in our distributor should be transparent to users,\r
-thus offering the same experience as using conventional software management systems, despite enhanced efficiency.\r
+thus offering the same experience as using conventional software management systems, but with enhanced efficiency.\r
That said, we assume that the user is still attempting to download packages from a\r
-server, but the requests will be proxied by our peer-to-peer program. The server is always available and has all of the package files.\r
+server, but the requests will be proxied by our peer-to-peer program.\r
+We further assume that the server is always available and has all of the package files.\r
In addition, the cryptographic hash of the packages will be available\r
separately from the package itself, and is usually contained in an\r
\emph{index} file which also contains all the packages' names,\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{model_simple.eps}\r
+\includegraphics[width=0.9\columnwidth]{model_simple.eps}\r
\caption{The different phases of functionality of our peer-to-peer distribution model.}\r
\label{model}\r
\end{figure}\r
packages. Later, in Phase~2, upon receiving a request from the user\r
to download a package (5), our program will search the index files\r
for the package being requested and find its hash (6). This hash can\r
-then be looked up recursively in an indexing structure (a Distributed Hash Table [] in our implementation) (7), which will return a\r
+then be looked up recursively in an indexing structure (a Distributed Hash Table in our implementation) (7), which will return a\r
list of peers that have the package already (8). The package can\r
then be downloaded from the peers (11,12), it can be verified using\r
the hash (13), and if valid can be returned to the user (14). The\r
In steps (11,12), the fact that this package is also available to download for free\r
from a server is very important to our proposed model. If the package hash\r
can not be found in the DHT, the peer can then fallback to\r
-downloading from the original location (i.e. the network of\r
-mirrors). The mirrors thus, with no modification to their\r
-functionality, serve as seeds for the packages in the peer-to-peer\r
+downloading from the original location (i.e. the server).\r
+The server thus, with no modification to its\r
+functionality, serves as a seed for the packages in the peer-to-peer\r
system. Any packages that have just been updated, or that are very\r
rare, and so don't have any peers available, can always be found on\r
-the mirror. Once the peer has completed the download from the mirror\r
+the server. Once the peer has completed the download from the server\r
and verified the package, it can then add itself to the DHT as the\r
first peer for the new package, so that future requests for the package\r
-will not need the mirror.\r
+will not need the server.\r
\r
This sparse\r
interest in a large number of packages undergoing constant updating\r
of data, for which the cryptographic hashes used by these package\r
management systems are perfect for. The stored and retrieved strings\r
can then be pointers to the peers that have the package that hashes\r
-to that key. A downloading peer can lookup the package hash in the\r
-DHT and, if it is found, download the file from those peers and\r
-verify the package with the hash. Once the download is complete, the\r
-peer will add its entry to the DHT indicating that it now has the\r
-package.\r
+to that key.\r
\r
Note that, despite downloading the package from untrustworthy peers,\r
the trust of the package is always guaranteed through the use\r
from a trusted source (i.e. the distributors server). Most distributors use index\r
files that contain hashes for a large number of the packages in\r
their archive, and which are also hashed. After retrieving the\r
-index's hash from the mirror, the index file can be downloaded from\r
+index's hash from the server, the index file can be downloaded from\r
peers and verified. Then the program has access to all the hashes of\r
the packages it will be downloading, all of which can be verified\r
with a \emph{chain of trust} that stretches back to the original\r
distributor's server.\r
\r
-\subsection{Implementation Options}\r
-\label{imp_options}\r
-\r
-There are several ways to implement the desired peer-to-peer functionality\r
-into the existing package management software. The functionality can\r
-be directly integrated through modifications to the software, though\r
-this could be difficult as the peer-to-peer functionality should be running\r
-at all times. This is needed both for efficient lookups and to\r
-support uploading of already downloaded packages. Unfortunately, the\r
-package management tools typically only run until the download and\r
-install request is complete.\r
-\r
-Many of the package management software implementations use HTTP\r
-requests from web servers to download the packages, which makes it\r
-possible to implement the peer-to-peer aspect as an almost standard HTTP\r
-caching proxy. This proxy will run as a daemon in the background,\r
-listening for requests from the package management tool for package\r
-files, as well as serving (uploading) cached package to other peers.\r
-It will get uncached requests first from the peer-to-peer system, or\r
-fall back to the normal HTTP request from a server should it not\r
-be found. For methods that don't use HTTP requests, other types of\r
-proxies may also be possible.\r
-\r
-\subsection{Downloading From Peers}\r
+% \subsection{Implementation Options}\r
+% \label{imp_options}\r
+% \r
+% There are several ways to implement the desired peer-to-peer functionality\r
+% into the existing package management software. The functionality can\r
+% be directly integrated through modifications to the software, though\r
+% this could be difficult as the peer-to-peer functionality should be running\r
+% at all times. This is needed both for efficient lookups and to\r
+% support uploading of already downloaded packages. Unfortunately, the\r
+% package management tools typically only run until the download and\r
+% install request is complete.\r
+% \r
+% Many of the package management software implementations use HTTP\r
+% requests from web servers to download the packages, which makes it\r
+% possible to implement the peer-to-peer aspect as an almost standard HTTP\r
+% caching proxy. This proxy will run as a daemon in the background,\r
+% listening for requests from the package management tool for package\r
+% files, as well as serving (uploading) cached package to other peers.\r
+% It will get uncached requests first from the peer-to-peer system, or\r
+% fall back to the normal HTTP request from a server should it not\r
+% be found. For methods that don't use HTTP requests, other types of\r
+% proxies may also be possible.\r
+\r
+\subsection{Peer Downloading Protocol}\r
\label{downloading}\r
\r
Although not necessary, we recommend implementing a download\r
protocol that is similar to the protocol used to fetch packages from\r
-the distributor's servers. This simplifies the peer-to-peer program, as it\r
-can then treat peers and mirrors almost identically when requesting\r
-packages. In fact, the mirrors can be used when there are only a few\r
+the distributor's server. This simplifies the peer-to-peer program, as it\r
+can then treat peers and the server almost identically when requesting\r
+packages. In fact, the server can be used when there are only a few\r
slow peers available for a file to help speed up the download\r
process.\r
\r
addition to storing the file download location in the DHT (which\r
would still be used for small files), a peer will store a\r
\emph{torrent string} containing the peer's hashes of the pieces of\r
-the larger files. These piece hashes are compared ahead of time\r
+the larger files (similar to (15) in Phase 3 of Figure~\ref{model}).\r
+These piece hashes will be retrieved and compared ahead of time by\r
+the downloading peer ((9,10) in Phase 2 of Figure~\ref{model})\r
to determine which peers have the same piece hashes (they all\r
should), and then used during the download to verify the pieces of\r
the downloaded package.\r
implement trackerless operation. The communication is all handled by\r
UDP messages, and RPC (remote procedure call) requests and responses\r
are all \emph{bencoded} in the same way as BitTorrent's\r
-\texttt{.torrent} files. Khashmir uses the high-level Twisted\r
-event-driven networking engine \cite{twisted}, so we also use\r
-Twisted in our sample implementation for all other networking needs.\r
+\texttt{.torrent} files.\r
+% Khashmir uses the high-level Twisted\r
+% event-driven networking engine \cite{twisted}, so we also use\r
+% Twisted in our sample implementation for all other networking needs.\r
More details of this customized DHT can be found below in\r
Section~\ref{custom_dht}.\r
\r
\r
%%%%%%%%%%%%%%%%%%%%%%%%%%% Section %%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
\r
-\section{Customized DHT / System Optimization}\r
+\section{System Optimization}\r
\label{custom_dht}\r
\r
-A large contribution of our work is in the customization and use of\r
+Another contribution of our work is in the customization and use of\r
a Distributed Hash Table (DHT). Although our DHT is based on\r
Kademlia, we have made many improvements to it to make it suitable\r
for this application. In addition to a novel storage technique to\r
incorporated some improvements from BitTorrent's tracker-less DHT\r
implementation.\r
\r
-\subsection{Kademlia DHT Background}\r
+\subsection{DHT Details}\r
\label{dht}\r
\r
-DHT's operate by storing (\emph{key}, \emph{value}) pairs in a\r
-distributed fashion such that no node will, on average, store more\r
+DHTs operate by storing (\emph{key}, \emph{value}) pairs in a\r
+distributed fashion such that no node will, on average, store more or have to work harder\r
than any other node. They support two primitive operations:\r
\emph{put}, which takes a key and a value and stores it in the DHT;\r
and \emph{get}, which takes a key and returns a value (or values)\r
that was previously stored with that key. These operations are\r
-recursive, as each node does not know about all the other nodes in a\r
+recursive, as each node does not know about all the other nodes in the\r
DHT, and so must recursively search for the correct node to put to\r
or get from.\r
\r
-The Kademlia DHT, like most other DHTs, assigns IDs to peers from\r
+The Kademlia DHT, like most other DHTs, assigns IDs to peers randomly from\r
the same space that is used for keys. The peers with IDs closest to\r
-the desired key will then store the values for that key. Lookups are\r
-recursive, as nodes are queried in each step that are exponentially\r
-closer to the key than in the previous step.\r
-\r
-Nodes in a Kademlia system support four primitive requests.\r
+the desired key will then store the values for that key.\r
+Nodes support four primitive requests.\r
\texttt{ping} will cause a peer to return nothing, and is only used\r
-to determine if a node is still alive, while \texttt{store} tells a\r
+to determine if a node is still alive. \texttt{store} tells a\r
node to store a value associated with a given key. The most\r
important primitives are \texttt{find\_node} and\r
\texttt{find\_value}, which both function recursively to find nodes\r
close to a key. The queried nodes will return a list of the nodes\r
they know about that are closest to the key, allowing the querying\r
-node to quickly traverse the DHT to find the nodes close to the\r
+node to quickly traverse the DHT to find the nodes closest to the\r
desired key. The only difference between \texttt{find\_node} and \texttt{find\_value} is that the\r
\texttt{find\_value} query will cause a node to return a value, if\r
it has one for that key, instead of a list of nodes.\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{apt_p2p_improvements-find_value.eps}\r
+\includegraphics[width=0.80\columnwidth]{apt_p2p_improvements-find_value.eps}\r
\caption{The distribution of average response times PlanetLab nodes\r
experience for \texttt{find\_value} queries. The original DHT\r
implementation results are shown, as well as the successive\r
\label{improvements}\r
\end{figure}\r
\r
-To test our changes during development, we ran the customized DHT\r
+To test our changes during development, we ran our customized DHT\r
for several hours after each major change on over 300 PlanetLab nodes\r
\cite{planetlab}. Though the nodes are not expected to be firewalled\r
or NATted, some can be quite overloaded and so consistently fail to\r
respond within a timeout period, similar to NATted peers. The\r
resulting distribution of the nodes' average response times is shown\r
in Figure~\ref{improvements}. Each improvement successfully reduced\r
-the response time, for a total reduction of more than half. The\r
+the response time, for a total reduction of more than 50\%. The\r
final distribution is also narrower, as the improvements make the\r
system more predictable. However, there are still a large number of\r
outliers with higher average response times, which are the\r
the node is too overloaded for the program to be able to check for a\r
timeout very often.\r
\r
-\subsection{Extension with Multiple Values}\r
+\subsection{Multiple Values Extension}\r
\label{multiple_values}\r
\r
The original design of Kademlia specified that each key would have\r
it could choose to only request values from the nodes that are\r
closest to the key being searched for.\r
\r
-\r
+% \subsection{\textbf{OPTIONAL}: BitTorrent's Improvements}\r
+% \label{bittorrent_dht}\r
+% \r
+% In the several years that some BitTorrent clients have been using a\r
+% Kademlia-based DHT for tracker-less operation, the developers have\r
+% made many enhancements which we can take advantage of. One of the\r
+% most important is a security feature added to stop malicious nodes\r
+% from subscribing other nodes as downloaders. When a node issues a\r
+% request to another node to store its download info for that key, it\r
+% must include a \emph{token} returned by the storing node in a recent\r
+% \texttt{find\_node} request. The token is created by hashing the IP\r
+% address of the requesting peer with a temporary secret that expires\r
+% after several minutes. This prevents the requesting peer from faking\r
+% its IP address in the store request, since it must first receive a\r
+% response from a \texttt{find\_node} on that IP.\r
+% \r
+% We also made some BitTorrent-inspired changes to the parameters of\r
+% the DHT originally specified by the authors of Kademlia. Since, as\r
+% we will show later, peers stay online in our system for much longer\r
+% periods of time, we reduced Kademlia's \emph{k} value from 20 to 8.\r
+% The value is supposed to be large enough such that any given\r
+% \emph{k} nodes are unlikely to fail within an hour of each other,\r
+% which is very unlikely in our system given the long uptimes of\r
+% nodes (see Figure~\ref{duration_online_1} in Section~\ref{analysis}.\r
+% We also increased the number of concurrent outstanding\r
+% requests allowed from 3 to 6 to speed up the recursive key finding\r
+% processes.\r
+% \r
+% \subsection{\textbf{OPTIONAL}: Other Changes}\r
+% \label{other_changes}\r
+% \r
+% In addition, we have allowed peers to store values in the DHT, even\r
+% if the hash they are using is not the correct length. Most of the\r
+% keys used in the DHT are based on the SHA1 hash, and so they are 20\r
+% bytes in length. However, some files in the targetted Debian package\r
+% system are only hashed using the MD5 algorithm, and so the hashes\r
+% retrieved from the server are 16 bytes in length. The DHT will still\r
+% store values using these keys by padding the end of them with zeroes\r
+% to the proper length, as the loss of uniqueness from the 4 less\r
+% bytes is not an issue. Also, the DHT will happily store hashes that\r
+% are longer than 20 bytes, such as those from the 32 byte SHA256\r
+% algorithm, by truncating them to the correct length. Since the\r
+% requesting peer will have the full length of the hash, this will not\r
+% affect its attempt to verify the downloaded file.\r
\r
%%%%%%%%%%%%%%%%%%%%%%%%%%% Section %%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{AptP2PWalker-peers.eps}\r
+\includegraphics[width=0.80\columnwidth]{AptP2PWalker-peers.eps}\r
\caption{The number of peers found in the system, and how many are\r
behind a firewall or NAT.}\r
\label{walker_peers}\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{AptP2PDuration-peers.eps}\r
+\includegraphics[width=0.80\columnwidth]{AptP2PDuration-peers.eps}\r
\caption{The CDF of how long an average session will last.}\r
\label{duration_peers}\r
\end{figure}\r
for other peer-to-peer systems, which had 50\% of Napster and Gnutella\r
sessions lasting only 1 hour.\r
\r
-\begin{figure}\r
-\centering\r
-\includegraphics[width=\columnwidth]{AptP2PDuration-ind_peers.eps}\r
-\caption{\textbf{OPTIONAL}: The CDF of the average time individual peers stay in the\r
-system.}\r
-\label{duration_ind_peers}\r
-\end{figure}\r
-\r
-\textbf{OPTIONAL}: We also examined the average time each individual peer spends in the\r
-system. Figure~\ref{duration_peers} shows the cumulative\r
-distribution of how long each individual peer remains in the system.\r
-Here we see that 50\% of peers have average stays in the system\r
-longer than 10 hours.\r
+% \begin{figure}\r
+% \centering\r
+% \includegraphics[width=0.80\columnwidth]{AptP2PDuration-ind_peers.eps}\r
+% \caption{\textbf{OPTIONAL}: The CDF of the average time individual peers stay in the\r
+% system.}\r
+% \label{duration_ind_peers}\r
+% \end{figure}\r
+% \r
+% \textbf{OPTIONAL}: We also examined the average time each individual peer spends in the\r
+% system. Figure~\ref{duration_peers} shows the cumulative\r
+% distribution of how long each individual peer remains in the system.\r
+% Here we see that 50\% of peers have average stays in the system\r
+% longer than 10 hours.\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{AptP2PDuration-online_1.eps}\r
+\includegraphics[width=0.80\columnwidth]{AptP2PDuration-online_1.eps}\r
\caption{The fraction of peers that, given their current duration in\r
the system, will stay online for another hour.}\r
\label{duration_online_1}\r
will remain online another hour, compared with around 50\% for\r
Gnutella.\r
\r
-\begin{figure}\r
-\centering\r
-\includegraphics[width=\columnwidth]{AptP2PDuration-online_6.eps}\r
-\caption{\textbf{OPTIONAL}: The fraction of peers that, given their current duration in\r
-the system, will stay online for another 6 hours.}\r
-\label{duration_online_6}\r
-\end{figure}\r
-\r
-\textbf{OPTIONAL}: Since our peers are much longer-lived than other peer-to-peer systems, we\r
-also looked at the fraction of peers that stay online for another 6\r
-hours. Figure~\ref{duration_online_6} shows that over 60\% of peers\r
-that are online for 10 hours will stay online for another 6.\r
-However, we see an interesting decrease in this fraction between 8\r
-and 12 hours, which can also be seen in\r
-Figure~\ref{duration_online_1}. We believe this to be due to desktop\r
-users, who regularly turn off their computers at night.\r
+% \begin{figure}\r
+% \centering\r
+% \includegraphics[width=0.80\columnwidth]{AptP2PDuration-online_6.eps}\r
+% \caption{\textbf{OPTIONAL}: The fraction of peers that, given their current duration in\r
+% the system, will stay online for another 6 hours.}\r
+% \label{duration_online_6}\r
+% \end{figure}\r
+% \r
+% \textbf{OPTIONAL}: Since our peers are much longer-lived than other peer-to-peer systems, we\r
+% also looked at the fraction of peers that stay online for another 6\r
+% hours. Figure~\ref{duration_online_6} shows that over 60\% of peers\r
+% that are online for 10 hours will stay online for another 6.\r
+% However, we see an interesting decrease in this fraction between 8\r
+% and 12 hours, which can also be seen in\r
+% Figure~\ref{duration_online_1}. We believe this to be due to desktop\r
+% users, who regularly turn off their computers at night.\r
\r
\subsection{Peer Statistics}\r
\label{peer_stats}\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{AptP2PDownloaded-peers.eps}\r
+\includegraphics[width=0.80\columnwidth]{AptP2PDownloaded-peers.eps}\r
\caption{The number of peers that were contacted to determine their\r
bandwidth, and the total number of peers in the system.}\r
\label{down_peers}\r
\r
\begin{figure}\r
\centering\r
-\includegraphics[width=\columnwidth]{AptP2PDownloaded-bw.eps}\r
-\caption{\textbf{OPTIONAL}: The bandwidth of data that the contacted peers have\r
+\includegraphics[width=0.80\columnwidth]{AptP2PDownloaded-bw.eps}\r
+\caption{The bandwidth of data that the contacted peers have\r
downloaded and uploaded.}\r
\label{down_bw}\r
\end{figure}\r
bandwidth. The actual numbers are only a lower bound, since we have\r
only contacted 30\% of the peers in the system, but we can estimate\r
that \texttt{apt-p2p} has already saved the mirrors 15 GB of\r
-bandwidth, or 1 GB per day.\r
+bandwidth, or 1 GB per day. Considering the current small number of\r
+users this savings is quite large, and is expected to grow\r
+considerably as more users participate in the P2P system.\r
\r
We also collected the statistics on the measured response time peers\r
were experiencing when sending requests to the DHT. We found that\r
the recursive \texttt{find\_value} query, which is necessary before\r
a download can occur, is taking 17 seconds on average. This\r
-indicates that, on average, requests are experiencing almost 2\r
+indicates that, on average, requests are experiencing almost 2 full\r
stalls while waiting for the 9 second timeouts to occur on\r
-unresponsive peers. This is longer than our target of 10 seconds,\r
+unresponsive peers. This time is longer than our target of 10 seconds,\r
although it will only lead to a slight average delay in downloading\r
of 1.7 seconds when the default 10 concurrent downloads are\r
occurring.This increased response time is due to the number of peers\r
\r
We were also concerned that the constant DHT requests and responses,\r
even while not downloading, would overwhelm some peers' network\r
-connections. However, we found that peers are using 200 to 300 bytes\r
-per second of bandwidth in servicing the DHT. These numbers are\r
+connections. However, we found that peers are using 200 to 300 bytes/sec\r
+of bandwidth in servicing the DHT. These numbers are\r
small enough to not affect any other network services the peer would\r
be running.\r
\r
considers all the files available to users to download, and makes\r
use of the existing infrastructure unmodified.\r
\r
-\textbf{OPTIONAL}: Others have also used DHTs to support this type of functionality.\r
-Kenosis \cite{kenosis} is a peer-to-peer Remote Procedure Call\r
-client also based on the Kademlia DHT, but it is meant as a peer-to-peer\r
-primitive system on which other tools can be built, and so it has no\r
-file sharing functionality at all. Many have used a DHT as a drop-in\r
-replacement for the tracker in a BitTorrent system\r
-\cite{bittorrent-dht, azureus-dht}, but such systems only use the\r
-DHT to find peers for the same torrent, so the file sharing uses\r
-traditional BitTorrent and so is not ideal for the reasons listed\r
-in Section~\ref{bittorrent}.\r
+% \textbf{OPTIONAL}: Others have also used DHTs to support this type of functionality.\r
+% Kenosis \cite{kenosis} is a peer-to-peer Remote Procedure Call\r
+% client also based on the Kademlia DHT, but it is meant as a peer-to-peer\r
+% primitive system on which other tools can be built, and so it has no\r
+% file sharing functionality at all. Many have used a DHT as a drop-in\r
+% replacement for the tracker in a BitTorrent system\r
+% \cite{bittorrent-dht, azureus-dht}, but such systems only use the\r
+% DHT to find peers for the same torrent, so the file sharing uses\r
+% traditional BitTorrent and so is not ideal for the reasons listed\r
+% in Section~\ref{bittorrent}.\r
\r
% \cite{deltacast}\r
\r
projects / quix0rs-apt-p2p.git / commitdiff