X-Git-Url: https://git.mxchange.org/?p=quix0rs-apt-p2p.git;a=blobdiff_plain;f=TODO;h=fc4158dfdfbabf9ded10538bca4dc0fb21473e04;hp=cfb65611ff30ea8bf7675a11051e86dce904be22;hb=151de5a26973474d2b1a8fc3f071615c09e9a62d;hpb=c107357b142b97125b289d1067fd3823e2c23464 diff --git a/TODO b/TODO index cfb6561..fc4158d 100644 --- a/TODO +++ b/TODO @@ -1,15 +1,44 @@ -Some last few things to do before release. - -- Handle/investigate the HTTP client pipeline errors -- DB should not always restat files (especially for expired hashes) -- remove missing files at startup (in DB's removeUntracked) -- when files modtime but not size changes, rehash them to be sure -- lengthen the expiry time for DHT entries -- remove files from the peer's download cache -- update the modtime of files downloaded from peers - - also set the Last-Modified header for the return to Apt -- make the DHT timeouts configuration parameters -- refresh expired DHT hashes concurrently instead of sequentially +Rotate DNS entries for mirrors more reliably. + +Currently the mirrors are accessed by DNS name, which can cause some +issues when there are mirror differences and the DNS gets rotated. +Instead, the HTTP Downloader should handle DNS lookups itself, store +the resulting addresses, and send requests to IP addresses. If there +is an error from the mirror (hash check or 404 response), the next IP +address in the rotation should be used. + + +Use GPG signatures as a hash for files. + +A detached GPG signature, such as is found in Release.gpg, can be used +as a hash for the file. This hash can be used to verify the file when +it is downloaded, and a shortened version can be added to the DHT to +look up peers for the file. To get the hash into a binary form from +the ascii-armored detached file, use the command +'gpg --no-options --no-default-keyring --output - --dearmor -'. The +hash should be stored as the reverse of the resulting binary string, +as the bytes at the beginning are headers that are the same for most +signatures. That way the shortened hash stored in the DHT will have a +better chance of being unique and being stored on different peers. To +verify a file, first the binary hash must be re-reversed, armored, and +written to a temporary file with the command +'gpg --no-options --no-default-keyring --output $tempfile --enarmor -'. +Then the incoming file can be verified with the command +'gpg --no-options --no-default-keyring --keyring /etc/apt/trusted.gpg +--verify $tempfile -'. + +All communication with the command-line gpg should be done using pipes +and the python module python-gnupginterface. There needs to be a new +module for GPG verification and hashing, which will make this easier. +In particular, it would need to support hashlib-like functionality +such as new(), update(), and digest(). Note that the verification +would not involve signing the file again and comparing the signatures, +as this is not possible. Instead, the verify() function would have to +behave differently for GPG hashes, and check that the verification +resulted in a VALIDSIG. CAUTION: the detached signature can have a +variable length, though it seems to be usually 65 bytes, 64 bytes has +also been observed. + Consider what happens when multiple requests for a file are received. @@ -43,6 +72,14 @@ and not the headers, and also misses the requests for downloads sent to other peers. +Rehash changed files instead of removing them. + +When the modification time of a file changes but the size does not, +the file could be rehased to verify it is the same instead of +automatically removing it. The DB would have to be modified to return +deferred's for a lot of its functions. + + Consider storing deltas of packages. Instead of downloading full package files when a previous version of