P2P Data Sync Overview

Data Objects handled by Hyperties are synchronised by using the Reporter - Observer Peer-to-Peer data synchronisation communication pattern. Such mechanism is provided by two main components in the Hyperty Runtime:

The Syncher is a singleton Component co-located with the Service Instance, which is in charge of handling all required procedures to manage data synchronisation at the Hyperty instance side, as a Reporter or a Observer service. The Syncher is included in the Hyperty deployable package that is published in the Catalogue.

The Runtime Sync Manager is a Core Runtime Component, which is in charge of handling authorisation requests to create Sync Data Objects from Reporters and subscription requests to Sync Data Objects from Observers. As soon as authorisation is granted the Sync Manager handles all required MessageBUS listeners in order to setup the Data Sync Stream routing path among Reporters and Observers. I.e., the Runtime Sync Manager provides a Messaging Framework Routing Manager functionality.

When the data synchronisation stream is not established with P2P Connections, the Message Node Subscription Manager functionality is used, to handle requests from Runtime Sync Managers in order to setup the Data Sync Stream routing path between the Reporter Runtime and Observers Runtimes.

The Hyperty Data Object synchronisation mechanism is provided according to the following main procedures:

To create a new Data Object, the Reporter Hyperty uses the Syncher to request the Runtime Sync Manager to authorise it. If authorised, the Sync Manager asks the Address Allocation component to get a new Data Object Address and asks the Runtime Registry to register the new Data Object instance in the Domain Registry in order to be discoverable (optional). The required routing paths to receive requests to subscribe, unsubscribe and read the Data Object, are also setup by adding the appropriate listeners in the Message Bus and by asking the Message Node Subscription Manager functionality to setup these routing paths at domain level. If all these procedures are successfuly performed, the Sync Manager answers the Syncher data object creation request with a 200 Ok response including in the body the new allocated Data Object Address. Optionaly, the Hyperty Reporter may invite other Hyperties to be observers or to announce the creation of the new Data Object. The full detailed description of the Data Object Creation process is provided here.

To be an Observer of a Data Object an Hyperty uses its Syncher to subscribe it and the subscription request is forwarded to local Sync Manager Core component. At this point, the Subscription process is handled between the Observer Sync Manager and the Reporter Sync Manager. First, the observer Sync Manager adds the required listeners to the Message Bus to ensure all the routing paths are established to handle the Data Synchronisation (for data object changes and data object delete). In case P2P Connection are not used this process also requires to ask the Message Node of the Reporter domain to establish these routing paths. Then, the Subscription request is sent to Reporter Sync Manager, which verifies if there is any pre-authorisation set by the Reporter, if not, the subscription is forwarded to Reporter Hyperty Syncher. If the subscription request is authorised, the Sync Manager establishes the required routing path from the Reporter side, in order to complete the full data synchronisation stream establishment between the two Hyperty Runtimes. If the Subscription process is successful, a response is sent back to the new observer, containing in the body the most updated version of the Data Object. At this point, any change performed by the Reporter in the Data Object, is immediately propagated till the new Observer throught the data synchronisation stream. The full detailed description of the Data Object Subscription process is provided here.

Anytime, the Observer role is ended by using the Syncher to unsubscribe it. Again, unsubscription process is performed between the Observer Sync Manager and the Reporter Sync Manager. During this process, the data synchronisation stream is ended by removing the listeners added in the subscription process from the messaging framework and the Reporter is notified about the unsubscription. The full detailed description of the Data Object Subscription process is provided here. To be noted that the Reporter also has the power to stop the data synchronisation for a certain Observer as fully described here.

The Data Object can be deleted by the Hyperty Reporter which is handled by its Sync Manager by removing all listeners added to the Messaging Framework during the Data Object Creation process. Then, all observers are notified by the Data Object deletion. On receiving the Data Object delete event, Observers should perform unsubscribe process to remove all listener added during the subscription process. The full detailed description of the Data Object Delete process is provided here.

A Data Object Child belongs to a Data Object Parent resource and can be created by any Observer of the Data Object Parent as well as by its Reporter. The Reporter - Observer rules still applies to Data Object Child ie there is only one Reporter that can update the Data Object Child, which can be an Observer of the Data Object Parent, as mentioned earlier. In order to support Data Object Childs, a Data Object Synchronisation stream has to be established for each children with address <DataObjectURL>/children/<nameOfChildren> during the creation and the subscription of Parent Data Object. Such stream will be used to publish among all involved Hyperties (observers and reporter), the creation, update and delete of Data Object Childs. To be noted that the name of the children must defined in the Data Object Schema. The full detailed description on how to manage a Data Object Child is provided here.

The Synchronisation of Data Objects can be Resumed between different runtime sessions e.g. when the browser is closed and opened again. Such resume will be automatically performed everytime the Syncher is instantiated in order to resume the sync of data objects created in previous sessions for the same Hyperty ie the HypertyURL is maintained along the different sessions. In addition, objects can also be resumed and reused from different sessions by explicitely asking the syncher to resume data synchronization that matches a certain criteria e.g. for all objects compliant with a certain data schema that were associated with the UserURL that is currently associated with the Hyperty instance. Data synchronisation resume is optional and has to be set during the Data Object creation and subscription process. It implies that Data Object Metadata is stored locally by the Data Object Storage Runtime Core Component. The resume of the Data Synchronisation is fully described from the Reporter side here and from the Observer side here.