-Change all print statements to log.msg() calls.
+Consider what happens when we are the closest node.
+In some of the actions it is unclear what happens when we are one of the
+closest nodes to the target key. Do we store values that we publish
+ourself?
-Add ability to search and hash and DHT-store other directories.
-The user should be able to specify a list of directories that will be
-searched for files to hash and add to the DHT.
+Add all cache files to the database.
+All files in the cache should be added to the database, so that they can
+be checked to make sure nothing has happened to them. The database would
+then need a flag to indicate files that are hashed and available, but
+that shouldn't be added to the DHT.
-Missing Kademlia implementation details are needed.
-The current implementation is missing some important features, mostly
-focussed on storing values:
- - values need to be republished (every hour?)
- - original publishers need to republish values (every 24 hours)
- - when a new node is found that is closer to some values, replicate the
- values there without deleting them
- - when a value lookup succeeds, store the value in the closest node
- found that didn't have it
- - make the expiration time of a value exponentially inversely
- proportional to the number of nodes between the current node and the
- node closest to the value
+Packages.diff files need to be considered.
+
+The Packages.diff/Index files contain hashes of Packages.diff/rred.gz
+files, which themselves contain diffs to the Packages files previously
+downloaded. Apt will request these files for the testing/unstable
+distributions. They need to either be ignored, or dealt with properly by
+adding them to the tracking done by the AptPackages module.
+
+
+PeerManager needs to download large files from multiple peers.
+
+The PeerManager currently chooses a peer at random from the list of
+possible peers, and downloads the entire file from there. This needs to
+change if both a) the file is large (more than 512 KB), and b) there are
+multiple peers with the file. The PeerManager should then break up the
+large file into multiple pieces of size < 512 KB, and then send requests
+to multiple peers for these pieces.
+
+This can cause a problem with hash checking the returned data, as hashes
+for the pieces are not known. Any file that fails a hash check should be
+downloaded again, with each piece being downloaded from different peers
+than it was previously. The peers are shifted by 1, so that if a peers
+previously downloaded piece i, it now downloads piece i+1, and the first
+piece is downloaded by the previous downloader of the last piece, or
+preferably a previously unused peer. As each piece is downloaded the
+running hash of the file should be checked to determine the place at
+which the file differs from the previous download.
+
+If the hash check then passes, then the peer who originally provided the
+bad piece can be assessed blame for the error. Otherwise, the peer who
+originally provided the piece is probably at fault, since he is now
+providing a later piece. This doesn't work if the differing piece is the
+first piece, in which case it is downloaded from a 3rd peer, with
+consensus revealing the misbehaving peer.
+
+
+Consider tracking security issues with packages.
+
+Since sharing information with others about what packages you have
+downloaded (and probably installed) is a possible security
+vulnerability, it would be advantageous to not share that information
+for packages that have known security vulnerabilities. This would
+require some way of obtaining a list of which packages (and versions)
+are vulnerable, which is not currently available.
+
+
+Consider adding peer characteristics to the DHT.
+
+Bad peers could be indicated in the DHT by adding a new value that is
+the NOT of their ID (so they are guaranteed not to store it) indicating
+information about the peer. This could be bad votes on the peer, as
+otherwise a peer could add good info about itself.
+
+
+Consider adding pieces to the DHT instead of files.
+
+Instead of adding file hashes to the DHT, only piece hashes could be
+added. This would allow a peer to upload to other peers while it is
+still downloading the rest of the file. It is not clear that this is
+needed, since peer's will not be uploading and downloading ery much of
+the time.
projects / quix0rs-apt-p2p.git / blobdiff