Generic Object Sharing Protocol

Generic Object Sharing Protocol
- Hub Identification
  - Every hub generates an id string by random
    - This is the Hub-Id (Hub Identifier)
    - Only on first use
    - This should be globally unique
    - It will be stored in hub's database for later reuse
    - A hash is being generated of it
      - Hashed data:
        Hub's IP number and hostname
        Some random characters
    - This id does not change as long as the database is not purged
  - Per session another id is generated
    - This is the SID (Session IDentifier)
    - It is being distributed to the hubs
    - It stored together with the Hub-Id
      - So other can validate bother together
    - Logging should only enabled for debugging purposes
  - Locking IPs or Hub-Ids on master-nodes is not planed
    - Censorship would be to very easy
      - Government agencies or enterprise parties
    - Censhorship makes no sence here
      - It can very easy be bypassed:
      - Delete Hub-Id in database
        A new one got generated
      - Locked IP or port number can be bypassed by proxies
      - One or two master-hubs should listen on ports commonly unblocked by firewalls
        Like 80/443/110/25
  - Hubs can be optionally registered by master-nodes
    - Increases karma because the hub admin is verified
    - Unregistered hubs does not receive negative votings
- Bootstrapping
  - At least one, better 3 to 4, master-nodes are required
    - Aka. "Bootstrap-Nodes"
  - Bootstrap-Nodes are working stand-alone
    - No central "Super-Node" is required
    - Too much traffic would have to flow through it
    - Attacks on the network by censorship are reduced
    - Traffic does not increase network-overall load
    - Small disadvantage:
      - Hubs must register with ...
      - ... more than one master-node ...
      - .. or connect with each other
  - Hub is fetching a list of other hubs
    - They must have at least X matching object types
    - Hashes of hub-lists should match
    - If to much are inconsistent:
      - No connect can happen
        Hub list is rejected
      - Or the bootstrap-nodes are working as regular hubs
      - Replication of the hub-list is required by all bootstrap-nodes
- Karma
  - Karma is given for validating entries in the DHT
    - Last activity in near past
      - Does not affect karma
    - Returned pings
      - Amount of sent pings
        If no reply it got dead-listed
      - Failed pings reduce karma
      - Slow responses reduce karma
    - Karma voting for other hubs is not to negative
      - Reduces manipulation chances
        Prefer karma votes of trusted hubs
        Negative karma votings for untrusted karma reduce own karma
    - To much "spam packages" reduce karma
    - Validated packages increase karma
    - Protocol version should not be to old
      - This affects karma only negativly
      - An up-to-date protocol does not increase karma
      - Does also serve as a "spam protection"
      - Received protocol version of hub is older than stored
        Karma is reduced
      - Received protocol version is much than from master-nodes
        Karma is reduced
    - Provided object types by the peer hub
      - This affectes karma only negativly
      - New types must first be known by masters
      - This should be configurable:
        Karma should be reduced...
        ... or peer hub should be black-listed
      - Because of every node can be a master-mode censorship is really hard
    - Correctly logging
      - Does not affect karma
      - Logout must be done by master hub and active hubs
        "Bye" message
      - Rotating of dynamic IPs should be considered
        Must be registered by master-node
        ID is registered as "Dynamic IP"
        So connects are still possible
        No negative votings by other hubs
        Current IP does spread good in network
        Query of the master-node only in doubt
- Object Types
  - New object types are only addable by updating the software
  - It also possible by 3rd-party
    - Must be known by master/bootstrap-nodes
  - Outdated object types are marked "deprecated" for a longer time
    - Master-nodes may accept or reject them
    - A "deprecation message" is always being sent
    - A note of a required update can optionally be added
  - After deprecation time they are treated as "unknown"
  - Other hubs should ask bootstrap-nodes
    - This compensate errors made by master-nodes
    - Wrongly deprecated object types by the master-node result in bad karma by the bootstrap-node
- Update Messages
  - Will only be broadcasted from bootstrap- to master- and list-nodes
  - No hub will receive update messages due to heavy network load
    - Maybe only "good" hubs should receive this?
  - Contains update notes and importance level
- "Client" Connections
  - Should be interpreted as "application software"
  - Clients should also generate a "client id"
    - Both id and sid
  - Will also connect first to bootstrap-nodes
    - Ask for a hub-list as well
  - Do also receive karma from hubs
  - Dynamic IPs are also accepted and therefore must be registered
- Client<->Hub Communication
- Fault Tolerance / Reliability
  - After X failed connection attempts a hub got removed
    - Other hubs report this to the master-node
    - The master-node probes the failed hub and removes it
  - Failed list-node
    - Hubs are reporting it to the master-node
    - The master-node probes the failed list-node and removes it
  - Failed master-node
    - List-nodes takeover the role of a master-node if no bootstrap-nodes are available
      - This takeover should not be entirely and should be defined
    - If there is no list-node, hubs look for an active master-node
      - They report the failed master-node to it
    - If additionally no master-node is up, a hub will be elected as new master-node
      - Doing so, all hubs are identifying the hub with...
        ... the best karma
        This is known to many hubs
        ... most votings
        A "vote" is a positive karma
        Also known to many hubs
      - The "election" should take place within a specific timeout
      - If no election is happening the hub with most connections got elected
  - If one of the bootstrap-nodes is up
    - The elected hubs notifies a some of it's fellow hubs that the bootstrap-node is back
    - The elected hub becomes a regular hub and notifies other hubs on connection attempts
  - Disadvantages:
    - A new hub with only knowlege about the bootstrap-nodes may not be able to connect to the hubs
      - Additional bootstrap-nodes on other server and/or continent may help here

Generic Object Sharing Protocol
- Hub Identification
  - Every hub generates an id string by random
    - This is the Hub-Id (Hub Identifier)
    - Only on first use
    - This should be globally unique
    - It will be stored in hub's database for later reuse
    - A hash is being generated of it
      - Hashed data:
        Hub's IP number and hostname
        Some random characters
    - This id does not change as long as the database is not purged
  - Per session another id is generated
    - This is the SID (Session IDentifier)
    - It is being distributed to the hubs
    - It stored together with the Hub-Id
      - So other can validate bother together
    - Logging should only enabled for debugging purposes
  - Locking IPs or Hub-Ids on master-nodes is not planed
    - Censorship would be to very easy
      - Government agencies or enterprise parties
    - Censhorship makes no sence here
      - It can very easy be bypassed:
      - Delete Hub-Id in database
        A new one got generated
      - Locked IP or port number can be bypassed by proxies
      - One or two master-hubs should listen on ports commonly unblocked by firewalls
        Like 80/443/110/25
  - Hubs can be optionally registered by master-nodes
    - Increases karma because the hub admin is verified
    - Unregistered hubs does not receive negative votings
- Bootstrapping
  - At least one, better 3 to 4, master-nodes are required
    - Aka. "Bootstrap-Nodes"
  - Bootstrap-Nodes are working stand-alone
    - No central "Super-Node" is required
    - Too much traffic would have to flow through it
    - Attacks on the network by censorship are reduced
    - Traffic does not increase network-overall load
    - Small disadvantage:
      - Hubs must register with ...
      - ... more than one master-node ...
      - .. or connect with each other
  - Hub is fetching a list of other hubs
    - They must have at least X matching object types
    - Hashes of hub-lists should match
    - If to much are inconsistent:
      - No connect can happen
        Hub list is rejected
      - Or the bootstrap-nodes are working as regular hubs
      - Replication of the hub-list is required by all bootstrap-nodes
- Karma
  - Karma is given for validating entries in the DHT
    - Last activity in near past
      - Does not affect karma
    - Returned pings
      - Amount of sent pings
        If no reply it got dead-listed
      - Failed pings reduce karma
      - Slow responses reduce karma
    - Karma voting for other hubs is not to negative
      - Reduces manipulation chances
        Prefer karma votes of trusted hubs
        Negative karma votings for untrusted karma reduce own karma
    - To much "spam packages" reduce karma
    - Validated packages increase karma
    - Protocol version should not be to old
      - This affects karma only negativly
      - An up-to-date protocol does not increase karma
      - Does also serve as a "spam protection"
      - Received protocol version of hub is older than stored
        Karma is reduced
      - Received protocol version is much than from master-nodes
        Karma is reduced
    - Provided object types by the peer hub
      - This affectes karma only negativly
      - New types must first be known by masters
      - This should be configurable:
        Karma should be reduced...
        ... or peer hub should be black-listed
      - Because of every node can be a master-mode censorship is really hard
    - Correctly logging
      - Does not affect karma
      - Logout must be done by master hub and active hubs
        "Bye" message
      - Rotating of dynamic IPs should be considered
        Must be registered by master-node
        ID is registered as "Dynamic IP"
        So connects are still possible
        No negative votings by other hubs
        Current IP does spread good in network
        Query of the master-node only in doubt
- Object Types
  - New object types are only addable by updating the software
  - It also possible by 3rd-party
    - Must be known by master/bootstrap-nodes
  - Outdated object types are marked "deprecated" for a longer time
    - Master-nodes may accept or reject them
    - A "deprecation message" is always being sent
    - A note of a required update can optionally be added
  - After deprecation time they are treated as "unknown"
  - Other hubs should ask bootstrap-nodes
    - This compensate errors made by master-nodes
    - Wrongly deprecated object types by the master-node result in bad karma by the bootstrap-node
- Update Messages
  - Will only be broadcasted from bootstrap- to master- and list-nodes
  - No hub will receive update messages due to heavy network load
    - Maybe only "good" hubs should receive this?
  - Contains update notes and importance level
- "Client" Connections
  - Should be interpreted as "application software"
  - Clients should also generate a "client id"
    - Both id and sid
  - Will also connect first to bootstrap-nodes
    - Ask for a hub-list as well
  - Do also receive karma from hubs
  - Dynamic IPs are also accepted and therefore must be registered
- Client<->Hub Communication
  - After a client has bootstrapped it announces all it's object types to the hubs
    - Including acceptance of broadcasts, poll-mode and Ping-POST
  - By this the hubs know clients and their accepted object types
  - Clients may download a hub-list for a specific object type
    - Distinct-List-Mode
    - After selecting a hub the client can request a list of clients from that hub
    - From these clients the client can accept objects from and send to
      - E.g. news by broadcast
  - Clients may send "broadcast" objects
    - Broadcast-Mode
    - Must be allowed by hubs
      - This consumes traffic
      - Acceptance of broadcasts is known to list-/master- and bootstrap-nodes
    - A client sends its broadcast to the master-nodes
      - They are distribute it to their fellow hubs
      - A hub knows which client accepts broadcasts and "deposits" it for the client
      - Clients are requesting such broadcasts by poll-mode or are "pinged"
        In poll-mode the client asks on a regular basis at the hub for new broadcasts
        A Ping-POST is being sent by the hub as a regular HTTP-POST request to the client
        This also happens on a regular basis
        A hub-admin may allow both types independly
        If none is allowed the hub acts as a "relay"
        And therefore it cannot accept clients with broadcast-functionality enabled
  - Client-Client Communication
    - May be done "anonymously" over the hub or directly with an other client
      - Communication of the hub is done in poll-mode or by Ping-POST
      - Direct client-client communication client "A" sends a Ping-POST directly to client "B"
    - Wrongly sent Ping-POSTs (e.g. the admin doesn't allow them) may be answered with a regular HTTP status '4XX'
- Fault Tolerance / Reliability
  - After X failed connection attempts a hub got removed
    - Other hubs report this to the master-node
    - The master-node probes the failed hub and removes it
  - Failed list-node
    - Hubs are reporting it to the master-node
    - The master-node probes the failed list-node and removes it
  - Failed master-node
    - List-nodes takeover the role of a master-node if no bootstrap-nodes are available
      - This takeover should not be entirely and should be defined
    - If there is no list-node, hubs look for an active master-node
      - They report the failed master-node to it
    - If additionally no master-node is up, a hub will be elected as new master-node
      - Doing so, all hubs are identifying the hub with...
        ... the best karma
        This is known to many hubs
        ... most votings
        A "vote" is a positive karma
        Also known to many hubs
      - The "election" should take place within a specific timeout
      - If no election is happening the hub with most connections got elected
  - If one of the bootstrap-nodes is up
    - The elected hubs notifies a some of it's fellow hubs that the bootstrap-node is back
    - The elected hub becomes a regular hub and notifies other hubs on connection attempts
  - Disadvantages:
    - A new hub with only knowlege about the bootstrap-nodes may not be able to connect to the hubs
      - Additional bootstrap-nodes on other server and/or continent may help here