1. A computer implemented system for synchronizing electronically stored files between a client file system and a cloud file system comprising:
a cloud watcher processor that receives a change log indicative of the status of the cloud file system, detects changes to the cloud file system, and generates work items in response to the detected cloud file system changes;
a local watcher processor that concurrently monitors the client file system and detects changes thereto and generates work items in response to the detected client file system changes;
a fetcher processor that serializes selected ones of the work items and assigns each of the selected ones of the work items to only one of a plurality of worker processors that concurrently perform assigned work items, wherein the fetcher processor references a dependency map prior to assigning selected work items to worker processors;
wherein the work items are file operations resulting in a synchronization between the client file system and the cloud file system.
2. The computer implemented system of claim 1 further comprising an event aggregator processor that aggregates a number of the detected client file system changes such that the aggregated changes are used to generate work items.
3. The computer implemented system of claim 1 further comprising a snapshot that indicates a current synchronization state between the client file system and the cloud file system.
4. The computer implemented system of claim 1 further comprising a blacklist of work items that are not to be assigned to worker processors.
5. The computer implemented system of claim 1 further comprising a local graph that indicates a state of the local file system.
6. The computer implemented system of claim 1 further comprising a cloud graph that indicates a state of the cloud file system.
7. The computer implemented system of claim 1 wherein the dependency map includes indications of particular inodes that are affected by current worker operations.
8. The computer implemented system of claim 1 wherein the dependency map includes indications of particular resourceIDs that are affected by current worker operations.
9. The computer implemented system of claim 1 wherein the dependency map includes filenames of files that are affected by current worker operations.
10. The computer implemented system of claim 1 wherein the fetcher processor includes a work queue of work items that are ordered from oldest to newest.
11. The computer implemented system of claim 1, wherein the fetcher processor assigns each of the selected ones of the work items to only one of the plurality of worker processors to prevent any two worker processors in the plurality of work processors from operating on conflicting or overlapping changes.
12. A computer implemented method for synchronizing electronically stored files between a client file system and a cloud file system comprising the steps of:
receiving a change log indicative of the status of the cloud file system, detecting changes to the cloud file system, and generating work items in response to the detected cloud file system changes;
concurrently monitoring the client file system, detecting changes to one or more files electronically stored in the client file system, and generating work items in response to the detected client file system changes;
serially fetching the generated work items such that each of the generated work items is assigned to only one of a plurality of worker processors, wherein a dependency map is referenced prior to assigning selected work items to worker processors; and
concurrently working the fetched work items at the assigned worker processors, such that changes detected to a file in one of the client file system or cloud file system are replicated to a file of the other file system.
13. The computer implemented system of claim 12 further comprising the step of aggregating a number of the detected client file system changes and then using the aggregated changes to generate work items.
14. The computer implemented method of claim 12 further comprising the step of comparing a current state of the cloud file system to a graph representing a previous state of the file system such that any changes to files in the file system are determined.
15. The computer implemented method of claim 12 wherein the fetching of work items includes ordering the work items in a work queue from oldest to newest order.
16. The computer implemented method of claim 12 wherein the fetching of work items includes determining whether a particular work item in the work queue can be processed.
17. The computer implemented method of claim 16 wherein the fetching of work items includes maintaining the ordering of the work items in the work queue from oldest to newest order.
18. A computer storage medium having computer executable instructions which when executed by a computer cause the computer to perform operations comprising:
receiving a change log indicative of the status of the cloud file system, detecting changes to the cloud file system, and generating file system operations in response to the detected cloud file system changes;
concurrently monitoring the client file system, detecting changes to one or more files electronically stored in the client file system, aggregating a number of the client file system changes, and using the aggregated changes to generate file system operations in response to the detected client file system changes;
serially ordering the file system operations from oldest to newest;
assigning each of the file system operations to only one of a plurality of worker processors, wherein a dependency map is referenced prior to assigning selected work items to worker processors; and
concurrently performing the file system operations at assigned worker processors, such that changes detected to a file in one of the file systems are replicated a file of the other file system.
19. The computer storage medium of claim 18 which further causes the computer to perform the following operation comprising:
comparing a current state of the cloud file system to a graph such that any changes to files in the cloud file system are determined.
20. The computer storage medium of claim 19 which further causes the computer to perform the following operation comprising:
comparing the determined changes to a snapshot structure that is indicative of a current synchronization state between the client file system and the cloud file system.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A communication system simultaneously providing scalable bundling for a plurality of communication links, supporting constant bit rate data and statisticalrandom data bit rate, said system comprised of:
an upper transmission bundling level including a constant bit rate bundler and a statistical traffic bundler for allocating bundled data based on received constant bit rate and statistical data;
a lower transmission bundling level including at least one superlink unit, which receives the allocated bundled data, performing \u201con the fly\u201d dynamic fragmentation for generating frames to be transmitted through multiple communication links;
a lower reception bundling level including at least one superlink unit which receives data frames from multiple communication links and reassembles the fragmented data into the bundled data, performing \u201con the fly\u201d dynamic synchronization;
a upper reception bundling level including a constant bit rate bundler and a statistical traffic bundler for reassembling the transmitted data from the bundled data.
2. The system of claim 1 wherein each transmission lower level superlink unit includes a framing machine, a memory of transmission buffer and at least eight link interfaces, wherein each link interface measures each communication channel link bit rate.
3. The system of claim 1 wherein each reception lower level superlink unit includes an extraction machine, a memory of reception buffer and at least eight link interfaces.
4. The system of claim 1 wherein the transmission upper level bundler units include a classifier, a queue prioritizer, a scheduler and at least two superlink buffers, wherein the classifier allocates frames from the prioritized queues to the superlink buffers.
5. The system of claim 1 wherein the reception upper level bundler units include an allocator, an indexed cyclic buffer, a scheduler and at least two superlink buffers.
6. The system of claim 1 wherein the generated frames include a control box, a constant bit rate channel Field, a statistical traffic channel, and a cyclic redundancy check, wherein the control box is identical for all communication links of the same superlink.
7. The system of claim 6 wherein the control box includes: a links activity field indicating active links, a sequence field for synchronizing between the frames, a constant bit rate Descriptor Field and a received cyclic redundancy check Status Field.
8. A communication method simultaneously providing scalable bundling for a plurality of communication links supporting constant bit rate data and statisticalrandom data bit rate, said method comprising the steps of:
allocating bundled data at an upper communication level based on received constant bit rate and statistical data;
generating frames from the allocated bundled data at the lower communication level to be transmitted through multiple communication links;
receiving data frames from multiple communication links and reconstructing the bundled data frames by a lower reception communication level;
reassembling the transmitted constant bit rate and statistical traffic data from the data bundles by an recention communication level.
9. The method of claim 8 further comprising the step of measuring each communication channel link bit rate and allocating the statistical data between the channel links accordingly.
10. The method of claim 8 further including the step of synchronizing received frames in accordance with sequence field embedded within each frame’s control box.
11. The method of claim 1 further comprising the step of managing priority queues for bundled data allocation.
12. The method of claim 8 further comprising the step of allocating statistical data in accordance with priority using a short header with priorityport field and frame number prior to the transmission.
13. The method of claim 8 wherein the generated frames include a control box, a constant bit rate channel Field, a statistical traffic channel, and a cyclic redundancy check, wherein the control box is identical for all communication links of the same superlink.
14. The method of claim 8 wherein the control box includes: a links activity field indicating active links, a sequence field for synchronizing between the frames, a constant bit rate Descriptor Field and a Received cyclic redundancy check Status Field.