Improve the peer ranking and fix a minor bug.

[quix0rs-apt-p2p.git] / TODO

diff --git a/TODO b/TODO
index f72699275cc161b09c62fafc871a041e2c1f75fe..7a586fe9a2c4b15861764d4b103f9bcd22a2f5f0 100644 (file)
--- a/TODO
+++ b/TODO
@@ -1,22 +1,3 @@
-Evaluate and fix some errors in the ktable khashmir module.
-
-The KTable implementation has some possible errors in it. insertNode
-does not remove the original and use the new node when updating a node
-already in the table, as claimed by the comments. justSeenNode doesn't
-verify that the found node is the node that was being looked for, nor
-does it move the node to the end of the list of nodes (since they are
-supposed to be sorted by their lastSeen time) or update the bucket's
-last touched time.nodeFailed also doesn't verify the found node is the
-right node.
-
-
-Consider what happens when we are the closest node.
-
-In some of the actions it is unclear what happens when we are one of the
-closest nodes to the target key. Do we store values that we publish
-ourself?
-
-

 Add all cache files to the database.
 
 All files in the cache should be added to the database, so that they can
 Add all cache files to the database.
 
 All files in the cache should be added to the database, so that they can


@@ -30,35 +11,38 @@ Packages.diff files need to be considered.
 The Packages.diff/Index files contain hashes of Packages.diff/rred.gz 
 files, which themselves contain diffs to the Packages files previously 
 downloaded. Apt will request these files for the testing/unstable 
 The Packages.diff/Index files contain hashes of Packages.diff/rred.gz 
 files, which themselves contain diffs to the Packages files previously 
 downloaded. Apt will request these files for the testing/unstable 

-distributions. They need to either be ignored, or dealt with properly by 
+distributions. They need to be dealt with properly by 

 adding them to the tracking done by the AptPackages module.
 
 
 adding them to the tracking done by the AptPackages module.
 
 

-PeerManager needs to download large files from multiple peers.
-
-The PeerManager currently chooses a peer at random from the list of 
-possible peers, and downloads the entire file from there. This needs to 
-change if both a) the file is large (more than 512 KB), and b) there are
-multiple peers with the file. The PeerManager should then break up the 
-large file into multiple pieces of size < 512 KB, and then send requests 
-to multiple peers for these pieces.
-
-This can cause a problem with hash checking the returned data, as hashes 
-for the pieces are not known. Any file that fails a hash check should be 
-downloaded again, with each piece being downloaded from different peers 
-than it was previously. The peers are shifted by 1, so that if a peers 
-previously downloaded piece i, it now downloads piece i+1, and the first 
-piece is downloaded by the previous downloader of the last piece, or 
-preferably a previously unused peer. As each piece is downloaded the 
-running hash of the file should be checked to determine the place at 
-which the file differs from the previous download.
-
-If the hash check then passes, then the peer who originally provided the 
-bad piece can be assessed blame for the error. Otherwise, the peer who 
-originally provided the piece is probably at fault, since he is now 
-providing a later piece. This doesn't work if the differing piece is the 
-first piece, in which case it is downloaded from a 3rd peer, with 
-consensus revealing the misbehaving peer.
+Improve the downloaded and uploaded data measurements.
+
+There are 2 places that this data is measured: for statistics, and for
+limiting the upload bandwidth. They both have deficiencies as they
+sometimes miss the headers or the requests sent out. The upload
+bandwidth calculation only considers the stream in the upload and not
+the headers sent, and it also doesn't consider the upload bandwidth
+from requesting downloads from peers (though that may be a good thing).
+The statistics calculations for downloads include the headers of
+downloaded files, but not the requests received from peers for upload
+files. The statistics for uploaded data only includes the files sent
+and not the headers, and also misses the requests for downloads sent to
+other peers.
+
+
+Consider storing deltas of packages.
+
+Instead of downloading full package files when a previous version of
+the same package is available, peers could request a delta of the
+package to the previous version. This would only be done if the delta
+is significantly (>50%) smaller than the full package, and is not too
+large (absolutely). A peer that has a new package and an old one would
+add a list of deltas for the package to the value stored in the DHT.
+The delta information would specify the old version (by hash), the
+size of the delta, and the hash of the delta. A peer that has the same
+old package could then download the delta from the peer by requesting
+the hash of the delta. Alternatively, very small deltas could be
+stored directly in the DHT.

 
 
 Consider tracking security issues with packages.
 
 
 Consider tracking security issues with packages.