Generic Object-Sharing Protocol

Generic Object-Sharing Protocol
- Hub Identification
  - Every hub generates an id string by random
    - This is the Hub-Id (Hub Identifier)
    - Only on first use
    - This should be globally unique
    - It will be stored in hub's database for later reuse
    - A hash is being generated of it
      - Hashed data:
        Hub's IP number and hostname
        Some random characters
    - This id does not change as long as the database is not purged
  - Per session another id is generated
    - This is the SID (Session IDentifier)
    - It is being distributed to the hubs
    - It stored together with the Hub-Id
      - So other can validate bother together
    - Logging should only enabled for debugging purposes
  - Locking IPs or Hub-Ids on master-nodes is not planed
    - Censorship would be to very easy
      - Government agencies or enterprise parties
    - Censhorship makes no sence here
      - It can very easy be bypassed:
      - Delete Hub-Id in database
        A new one got generated
      - Locked IP or port number can be bypassed by proxies
      - One or two master-hubs should listen on ports commonly unblocked by firewalls
        Like 80/443/110/25
  - Hubs can be optionally registered by master-nodes
    - Increases karma because the hub admin is verified
    - Unregistered hubs does not receive negative votings
- Bootstrapping
  - At least one, better 3 to 4, master-nodes are required
    - Aka. "Bootstrap-Nodes"
  - Bootstrap-Nodes are working stand-alone
    - No central "Super-Node" is required
    - Too much traffic would have to flow through it
    - Attacks on the network by censorship are reduced
    - Traffic does not increase network-overall load
    - Small disadvantage:
      - Hubs must register with ...
      - ... more than one master-node ...
      - .. or connect with each other
  - Hub is fetching a list of other hubs
    - They must have at least X matching object types
    - Hashes of hub-lists should match
    - If to much are inconsistent:
      - No connect can happen
        Hub list is rejected
      - Or the bootstrap-nodes are working as regular hubs
      - Replication of the hub-list is required by all bootstrap-nodes
- Karma
  - Karma is given for validating entries in the DHT
    - Last activity in near past
      - Does not affect karma
    - Returned pings
      - Amount of sent pings
        If no reply it got dead-listed
      - Failed pings reduce karma
      - Slow responses reduce karma
    - Karma voting for other hubs is not to negative
      - Reduces manipulation chances
        Prefer karma votes of trusted hubs
        Negative karma votings for untrusted karma reduce own karma
    - To much "spam packages" reduce karma
    - Validated packages increase karma
    - Protocol version should not be to old
      - This affects karma only negativly
      - An up-to-date protocol does not increase karma
      - Does also serve as a "spam protection"
      - Received protocol version of hub is older than stored
        Karma is reduced
      - Received protocol version is much than from master-nodes
        Karma is reduced
    - Provided object types by the peer hub
      - This affectes karma only negativly
      - New types must first be known by masters
      - This should be configurable:
        Karma should be reduced...
        ... or peer hub should be black-listed
      - Because of every node can be a master-mode censorship is really hard
    - Correctly logging
      - Does not affect karma
      - Logout must be done by master hub and active hubs
        "Bye" message
      - Rotating of dynamic IPs should be considered
        Must be registered by master-node
        ID is registered as "Dynamic IP"
        So connects are still possible
        No negative votings by other hubs
        Current IP does spread good in network
        Query of the master-node only in doubt
- Update Messages
  - Will only be broadcasted from bootstrap- to master- and list-nodes
  - No hub will receive update messages due to heavy network load
    - Maybe only "good" hubs should receive this?
  - Contains update notes and importance level
- "Client" Connections
  - Should be interpreted as "application software"
  - Clients should also generate a "client id"
    - Both id and sid
  - Will also connect first to bootstrap-nodes
    - Ask for a hub-list as well
  - Do also receive karma from hubs
  - Dynamic IPs are also accepted and therefore must be registered
- Client<->Hub Communication
  - After a client has bootstrapped it announces all it's object types to the hubs
    - Including acceptance of broadcasts, poll-mode and Ping-POST
  - By this the hubs know clients and their accepted object types
  - Clients may download a hub-list for a specific object type
    - Distinct-List-Mode
    - After selecting a hub the client can request a list of clients from that hub
    - From these clients the client can accept objects from and send to
      - E.g. news by broadcast
  - Clients may send "broadcast" objects
    - Broadcast-Mode
    - Must be allowed by hubs
      - This consumes traffic
      - Acceptance of broadcasts is known to list-/master- and bootstrap-nodes
    - A client sends its broadcast to the master-nodes
      - They are distribute it to their fellow hubs
      - A hub knows which client accepts broadcasts and "deposits" it for the client
      - Clients are requesting such broadcasts by poll-mode or are "pinged"
        In poll-mode the client asks on a regular basis at the hub for new broadcasts
        A Ping-POST is being sent by the hub as a regular HTTP-POST request to the client
        This also happens on a regular basis
        A hub-admin may allow both types independly
        If none is allowed the hub acts as a "relay"
        And therefore it cannot accept clients with broadcast-functionality enabled
  - Client-Client Communication
    - May be done "anonymously" over the hub or directly with an other client
      - Communication of the hub is done in poll-mode or by Ping-POST
      - Direct client-client communication client "A" sends a Ping-POST directly to client "B"
    - Wrongly sent Ping-POSTs (e.g. the admin doesn't allow them) may be answered with a regular HTTP status '4XX'
- Usage of low-level protocols
  - Already existing low-level protocols like TCP/IP and UDP should be used
    - TCP should be used for "inter-communication"
    - UDP should be used for "streaming" the objects to other hubs
      - Parties are generating hashes of chunks for validation
      - Chunks should only be created for very big objects
        Total object size is larger than X KByte
      - The sender creates hashes and adds them to the chunk
        The receivers validates them
        No serial numbers a-la TCP are generated
      - The last chunk package contains both hashes
        Hash of itself and the final hash
      - If a hash fails to validate it is being collected
        After the final chunk was sent, failed chunks a re-requested
      - This is retried X times per hash
        But always at the end of the whole transaction and all together
        If still some hashes failed to transfer
        The object got dropped or fully requested
        This should be configurable by the admin
        To do so, the final hash and object type is submitted to the sender
        "Restransmit-Message"
        The sender is now trying smaller chunks
      - If all was successfully received
        The receivers sends a "done-message" to the sender with final hash and object type
    - There is also a "real" streaming mode
      - This is e.g. used for chat
      - For this TCP/IP is used and no hashes are generated
      - Also no chunks are generated
      - Only in this mode "multi-casting" is possible
- Fault Tolerance / Reliability
  - After X failed connection attempts a hub got removed
    - Other hubs report this to the master-node
    - The master-node probes the failed hub and removes it
  - Failed list-node
    - Hubs are reporting it to the master-node
    - The master-node probes the failed list-node and removes it
  - Failed master-node
    - List-nodes takeover the role of a master-node if no bootstrap-nodes are available
      - This takeover should not be entirely and should be defined
    - If there is no list-node, hubs look for an active master-node
      - They report the failed master-node to it
    - If additionally no master-node is up, a hub will be elected as new master-node
      - Doing so, all hubs are identifying the hub with...
        ... the best karma
        This is known to many hubs
        ... most votings
        A "vote" is a positive karma
        Also known to many hubs
      - The "election" should take place within a specific timeout
      - If no election is happening the hub with most connections got elected
  - If one of the bootstrap-nodes is up
    - The elected hubs notifies a some of it's fellow hubs that the bootstrap-node is back
    - The elected hub becomes a regular hub and notifies other hubs on connection attempts
  - Disadvantages:
    - A new hub with only knowlege about the bootstrap-nodes may not be able to connect to the hubs
      - Additional bootstrap-nodes on other server and/or continent may help here
- Object Types
  - New object types are only addable by updating the software
  - It also possible by 3rd-party
    - Must be known by master/bootstrap-nodes
  - Outdated object types are marked "deprecated" for a longer time
    - Master-nodes may accept or reject them
    - A "deprecation message" is always being sent
    - A note of a required update can optionally be added
  - After deprecation time they are treated as "unknown"
  - Other hubs should ask bootstrap-nodes
    - This compensate errors made by master-nodes
    - Wrongly deprecated object types by the master-node result in bad karma by the bootstrap-node

Generic Object-Sharing Protocol
+
- Node Identification
  +
  - Every node generates an id string by random
    +
    - This is the node id (node identifier)
    - Only on first use
    - This should be globally unique
    - It will be stored in node's database for later reuse
    - A hash is being generated of it
      +
      - Hashed data:
        +
        +
        Node's IP number and hostname
        +
        Some random characters
    - This id does not change as long as the database is not purged
  - Per session another id is generated
    +
    - This is the SID (Session IDentifier)
    - It is being distributed to the nodes
    - It stored together with the Node-Id
      +
      - So other can validate bother together
    - Logging should be enabled for debugging purposes only
  - Private key and its corresponding hash
    +
    - For signed and later encrypted traffic?
  - Locking IPs or Node-Ids on master-nodes is not planed
    +
    - Censorship would be to very easy
      +
      - Government agencies or enterprise parties
    - Censhorship makes no sence here
      +
      - It can very easy be bypassed:
      - Delete Node-Id in database
        +
        +
        A new one got generated
      - Locked IP or port number can be bypassed by proxies
      - One or two master-nodes should listen on ports commonly unblocked by firewalls
        +
        +
        Like 80/443/110/25
  - Hubs can be optionally registered by master-nodes
    +
    - Increases karma because the node admin is verified
    - Unregistered nodes does not receive negative votings
- Bootstrapping
  +
  - At least one, better 3 to 4, master-nodes are required
    +
    - Aka. "Bootstrap-Nodes"
    - They should be listed in the configuration for all applications
    - A comma-seperated list of node IPs with port numbers seperated by double-dot (:)
  - Bootstrap-Nodes are working stand-alone
    +
    - No central "Super-Node" is required
    - Too much traffic would have to flow through it
    - Attacks on the network by censorship are reduced
    - Traffic does not increase network-overall load
    - Small disadvantage:
      +
      - Hubs must register with ...
      - ... more than one master-node ...
      - .. or connect with each other
  - 1. Node checks if there is a list of master nodes already stored
    +
    - If so, it skips fetching the list of nodes step
  - 2. Node is announcing itself to the upper bootstrap hub(s)
    +
    - This should be done generic to keep things easy
    - Recommended is maybe an XML with all neccessary data
      +
      - Node status
        +
        +
        Mostly 'active' or 'reachable'
      - Session id (see below)
      - External and internal IP
        +
        +
        To allow LAN connections over connections on WAN IP
        +
        Allowing this aids networks where DHCP and IP masquerade is used
        +
        Most customer asynchronous DSL does not allow to connect from inside over the WAN IP back inside
      - TCP and UDP listening ports
    - The session id will be included here
      +
      - Other nodes shall always attempt to find a node by its current session id, not node it
      - The node id identifies the node globally
        +
        +
        Requires a well-randomized id to be created to avoid collisions with other nodes
        +
        First announcement dominates on all "upper" nodes
        +
        A maximum limit of node ids per IP address shall prevent id flooding
      - The session id identifies the node currently
        +
        +
        On every start a new one will be generated
        +
        This requires a mechanism to purge old session ids
  - 3. Node is fetching a list of other nodes
    +
    - They must have at least X matching object types
    - If a bootstrap node is full it forwards the node to an other bootstrap hub
    - If that node is again full the node will be forwarded to a list node
    - If wether no free bootstrap nor list node is available the node waits some time and tries it again
    - Hashes of node-lists distributed over the bootstrap and list nodes should match
      +
      - This can be ensured by a DHT
        +
        +
        DHT = Distributed Hash Table
        +
        
        Which format?
    - If to much are inconsistent:
      +
      - No connect can happen
        +
        +
        Node list is rejected
      - Or the bootstrap-nodes are working as regular nodes
      - Replication of the node-list is required by all bootstrap-nodes
  - 4. Node connects to gathered master nodes
    +
    - It again announces its object types to the master nodes
    - Again it provides the session id so the master node can map session id -> node id
  - 5. If all authorization steps are completed:
    +
    - The node starts to accept client connections
      +
      - (It already listens to them but rejects them)
    - Objects will now be shared with other nodes which accepts the same object types
- Karma
  +
  - Karma is given for validating entries in the DHT
    +
    - Last activity in near past
      +
      - Does not affect karma
    - Returned pings
      +
      - Amount of sent pings
        +
        +
        If no reply it got dead-listed
      - Failed pings reduce karma
      - Slow responses reduce karma
    - Karma voting for other nodes is not to negative
      +
      - Reduces manipulation chances
        +
        +
        Prefer karma votes of trusted nodes
        +
        Negative karma votings for untrusted karma reduce own karma
    - To much "spam packages" reduce karma
    - Validated packages increase karma
    - Protocol version should not be to old
      +
      - This affects karma only negativly
      - An up-to-date protocol does not increase karma
      - Does also serve as a "spam protection"
      - Received protocol version of node is older than stored
        +
        +
        Karma is reduced
      - Received protocol version is much than from master-nodes
        +
        +
        Karma is reduced
    - Provided object types by the peer hub
      +
      - This affectes karma only negativly
      - New types must first be known by masters
      - This should be configurable:
        +
        +
        Karma should be reduced...
        +
        ... or peer node should be black-listed
      - Because of every node can be a master-mode censorship is really hard
    - Correctly logging
      +
      - Does not affect karma
      - Logout must be done by master node and active nodes
        +
        +
        
        "Bye" message
      - Rotating of dynamic IPs should be considered
        +
        +
        Must be registered by master-node
        +
        +
        ID is registered as "Dynamic IP"
        +
        So connects are still possible
        +
        No negative votings by other nodes
        +
        Current IP does spread good in network
        +
        +
        Query of the master-node only in doubt
  - Karma influences peer state rating and vise versa
    +
    - What is peer state rating?
- Fault Tolerance / Reliability
  +
  - After X failed connection attempts a node got removed
    +
    - Other nodes report this to the master-node
    - The master-node probes the failed node and removes it
  - Failed list-node
    +
    - Hubs are reporting it to the master-node
    - The master-node probes the failed list-node and removes it
  - Failed master-node
    +
    - List-nodes takeover the role of a master-node if no bootstrap-nodes are available
      +
      - This takeover should not be entirely and should be defined
    - If there is no list-node, nodes look for an active master-node
      +
      - They report the failed master-node to it
    - If additionally no master-node is up, a node will be elected as new master-node
      +
      - Doing so, all nodes are identifying the node with...
        +
        +
        ... the best karma
        +
        +
        This is known to many nodes
        +
        ... most votings
        +
        +
        A "vote" is a positive karma
        +
        Also known to many nodes
      - The "election" should take place within a specific timeout
      - If no election is happening the node with most connections got elected
  - If one of the bootstrap-nodes is up
    +
    - The elected nodes notifies a some of it's fellow nodes that the bootstrap-node is back
    - The elected node becomes a regular node and notifies other nodes on connection attempts
  - Disadvantages:
    +
    - A new node with only knowlege about the bootstrap-nodes may not be able to connect to the nodes
      +
      - Additional bootstrap-nodes on other server and/or continent may help here
- Object Types
  +
  - New object types are only addable by updating the software
  - It also possible by 3rd-party
    +
    - Must be known by master/bootstrap-nodes
  - Outdated object types are marked "deprecated" for a longer time
    +
    - Master-nodes may accept or reject them
    - A "deprecation message" is always being sent
    - A note of a required update can optionally be added
  - After deprecation time they are treated as "unknown"
  - Other nodes should ask bootstrap-nodes
    +
    - This compensate errors made by master-nodes
    - Wrongly deprecated object types by the master-node result in bad karma by the bootstrap-node
- Update Messages
  +
  - Will only be broadcasted from bootstrap- to master- and list-nodes
  - No node will receive update messages due to heavy network load
    +
    - Maybe only "good" nodes should receive this?
  - Contains update notes and importance level
- "Client" Connections
  +
  - Should be interpreted as "application software"
  - Clients should also generate a "client id"
    +
    - Both id and sid
  - Will also connect first to bootstrap-nodes
    +
    - Ask for a node-list as well
  - Do also receive karma from nodes
  - Dynamic IPs are also accepted and therefore must be registered
- Client<->Node Communication
  +
  - After a client has bootstrapped it announces all it's object types to the nodes
    +
    - Including acceptance of broadcasts, poll-mode and Ping-POST
  - By this the nodes know clients and their accepted object types
  - Clients may download a node-list for a specific object type
    +
    - Distinct-List-Mode
    - After selecting a node the client can request a list of clients from that hub
    - From these clients the client can accept objects from and send to
      +
      - E.g. news by broadcast
  - Clients may send "broadcast" objects
    +
    - Broadcast-Mode
    - Must be allowed by nodes
      +
      - This consumes traffic
      - Acceptance of broadcasts is known to list-/master- and bootstrap-nodes
    - A client sends its broadcast to the master-nodes
      +
      - They are distribute it to their fellow nodes
      - A node knows which client accepts broadcasts and "deposits" it for the client
      - Clients are requesting such broadcasts by poll-mode or are "pinged"
        +
        +
        In poll-mode the client asks on a regular basis at the node for new broadcasts
        +
        A Ping-POST is being sent by the node as a regular HTTP-POST request to the client
        +
        +
        This also happens on a regular basis
        +
        A node-admin may allow both types independly
        +
        If none is allowed the node acts as a "relay"
        +
        +
        And therefore it cannot accept clients with broadcast-functionality enabled
  - Client-Client Communication
    +
    - May be done "anonymously" over the node or directly with an other client
      +
      - Communication of the node is done in poll-mode or by Ping-POST
      - Direct client-client communication client "A" sends a Ping-POST directly to client "B"
    - Wrongly sent Ping-POSTs (e.g. the admin doesn't allow them) may be answered with a regular HTTP status '4XX'
- Usage of low-level protocols
  +
  - Already existing low-level protocols like TCP/IP and UDP should be used
    +
    - TCP should be used for "inter-communication"
    - UDP should be used for "streaming" the objects to other nodes
      +
      - Parties are generating hashes of chunks for validation
      - Chunks should only be created for very big objects
        +
        +
        
        Total object size is larger than X KByte
      - The sender creates hashes and adds them to the chunk
        +
        +
        The receivers validates them
        +
        No serial numbers a-la TCP are generated
      - The last chunk package contains both hashes
        +
        +
        Hash of itself and the final hash
      - If a hash fails to validate it is being collected
        +
        +
        After the final chunk was sent, failed chunks a re-requested
      - This is retried X times per hash
        +
        +
        But always at the end of the whole transaction and all together
        +
        If still some hashes failed to transfer
        +
        +
        The object got dropped or fully requested
        +
        This should be configurable by the admin
        +
        To do so, the final hash and object type is submitted to the sender
        +
        +
        
        "Restransmit-Message"
        +
        The sender is now trying smaller chunks
      - If all was successfully received
        +
        +
        The receivers sends a "done-message" to the sender with final hash and object type
    - There is also a "real" streaming mode
      +
      - This is e.g. used for chat
      - For this TCP/IP is used and no hashes are generated
      - Also no chunks are generated
      - Only in this mode "multi-casting" is possible