-Evaluate and fix some errors in the ktable khashmir module.
+Consider what happens when multiple requests for a file are received.
-The KTable implementation has some possible errors in it. insertNode
-does not remove the original and use the new node when updating a node
-already in the table, as claimed by the comments. justSeenNode doesn't
-verify that the found node is the node that was being looked for, nor
-does it move the node to the end of the list of nodes (since they are
-supposed to be sorted by their lastSeen time) or update the bucket's
-last touched time.nodeFailed also doesn't verify the found node is the
-right node.
+When another request comes in for a file already being downloaded,
+the new request should wait for the old one to finish. This should
+also be done for multiple requests for peer downloads of files with
+the same hash.
-Consider what happens when we are the closest node.
+Packages.diff files need to be considered.
-In some of the actions it is unclear what happens when we are one of the
-closest nodes to the target key. Do we store values that we publish
-ourself?
+The Packages.diff/Index files contain hashes of Packages.diff/rred.gz
+files, which themselves contain diffs to the Packages files previously
+downloaded. Apt will request these files for the testing/unstable
+distributions. They need to be dealt with properly by
+adding them to the tracking done by the AptPackages module.
-Add all cache files to the database.
+Improve the downloaded and uploaded data measurements.
-All files in the cache should be added to the database, so that they can
-be checked to make sure nothing has happened to them. The database would
-then need a flag to indicate files that are hashed and available, but
-that shouldn't be added to the DHT.
+There are 2 places that this data is measured: for statistics, and for
+limiting the upload bandwidth. They both have deficiencies as they
+sometimes miss the headers or the requests sent out. The upload
+bandwidth calculation only considers the stream in the upload and not
+the headers sent, and it also doesn't consider the upload bandwidth
+from requesting downloads from peers (though that may be a good thing).
+The statistics calculations for downloads include the headers of
+downloaded files, but not the requests received from peers for upload
+files. The statistics for uploaded data only includes the files sent
+and not the headers, and also misses the requests for downloads sent to
+other peers.
-Packages.diff files need to be considered.
+Rehash changed files instead of removing them.
+
+When the modification time of a file changes but the size does not,
+the file could be rehased to verify it is the same instead of
+automatically removing it. The DB would have to be modified to return
+deferred's for a lot of its functions.
+
+
+Consider storing deltas of packages.
+
+Instead of downloading full package files when a previous version of
+the same package is available, peers could request a delta of the
+package to the previous version. This would only be done if the delta
+is significantly (>50%) smaller than the full package, and is not too
+large (absolutely). A peer that has a new package and an old one would
+add a list of deltas for the package to the value stored in the DHT.
+The delta information would specify the old version (by hash), the
+size of the delta, and the hash of the delta. A peer that has the same
+old package could then download the delta from the peer by requesting
+the hash of the delta. Alternatively, very small deltas could be
+stored directly in the DHT.
-The Packages.diff/Index files contain hashes of Packages.diff/rred.gz
-files, which themselves contain diffs to the Packages files previously
-downloaded. Apt will request these files for the testing/unstable
-distributions. They need to either be ignored, or dealt with properly by
-adding them to the tracking done by the AptPackages module.
+Consider tracking security issues with packages.
-PeerManager needs to download large files from multiple peers.
-
-The PeerManager currently chooses a peer at random from the list of
-possible peers, and downloads the entire file from there. This needs to
-change if both a) the file is large (more than 512 KB), and b) there are
-multiple peers with the file. The PeerManager should then break up the
-large file into multiple pieces of size < 512 KB, and then send requests
-to multiple peers for these pieces.
-
-This can cause a problem with hash checking the returned data, as hashes
-for the pieces are not known. Any file that fails a hash check should be
-downloaded again, with each piece being downloaded from different peers
-than it was previously. The peers are shifted by 1, so that if a peers
-previously downloaded piece i, it now downloads piece i+1, and the first
-piece is downloaded by the previous downloader of the last piece, or
-preferably a previously unused peer. As each piece is downloaded the
-running hash of the file should be checked to determine the place at
-which the file differs from the previous download.
-
-If the hash check then passes, then the peer who originally provided the
-bad piece can be assessed blame for the error. Otherwise, the peer who
-originally provided the piece is probably at fault, since he is now
-providing a later piece. This doesn't work if the differing piece is the
-first piece, in which case it is downloaded from a 3rd peer, with
-consensus revealing the misbehaving peer.
+Since sharing information with others about what packages you have
+downloaded (and probably installed) is a possible security
+vulnerability, it would be advantageous to not share that information
+for packages that have known security vulnerabilities. This would
+require some way of obtaining a list of which packages (and versions)
+are vulnerable, which is not currently available.
Consider adding peer characteristics to the DHT.
projects / quix0rs-apt-p2p.git / blobdiff