1. An article of manufacture comprising:
shared memory;
program memory to store a plurality of instructions;
a counter to be clocked by a signal derived from a stratum traceable clock (STC) reference;
a processor to execute the plurality of instructions stored in the program memory and thereby (1) process voice payload into voice packets for one or more channels, the voice payload having been obtained from a plurality of network packets sent by a sender machine through a network, and (2) buffer the voice packets before writing them to the shared memory, wherein one or more of the buffered voice packets are written by the processor in response to and only if an interrupt is received from the counter;
a digital signal processor (DSP) system to read the plurality of voice packets from the shared memory before processing them; and
an interface to a bus, to transmit voice data of one or more channels, from the voice packets processed by the DSP system, over the bus and according to a bus clock, wherein the bus clock is derived from the STC reference signal.
2. The article of manufacture of claim 1 wherein the interface is to a time division multiplexed (TDM) bus, and the bus clock is a TDM bus clock.
3. The article of manufacture of claim 2 wherein the shared memory implements a first in first out (FIFO) structure, for storing the voice packets, that can be accessed independently by both the processor and the DSP system.
4. The article of manufacture of claim 3 wherein the shared memory is made of static RAM, and the program memory is made of dynamic RAM.
5. The article of manufacture of claim 1 wherein the counter and the processor are on the same chip.
6. The article of manufacture of claim 1 wherein the program memory includes further instructions which, when executed by the processor, cause the network packets to be processed at an application layer.
7. The article of manufacture of claim 6 wherein the program memory includes further instructions which, when executed by the processor, cause the network packets to be processed at an asynchronous transfer mode (ATM) adaptation layer.
8. The article of manufacture of claim 7 wherein the program memory includes further instructions which, when executed by the processor, cause the network packets to be disassembled to recover octets for individual voice tributaries.
9. The article of manufacture of claim 1 wherein some of the voice packets to be stored in the shared memory contain compressed speech data, and wherein the DSP system is to decompress such speech data before transmission over the bus.
10. The article of manufacture of claim 1 wherein the DSP system is to process the voice packets by decoding the packets, including one or more of format changing, decompression, and echo cancellation, before transmission over the bus.
11. The article of manufacture of claim 1 wherein the program memory includes further instructions which, when executed by the processor, cause the counter to be programmed to periodically interrupt the processor at a base rate, such that voice data for a given channel is transferred into the bus at essentially the same rate as voice data for the same channel is collected off a bus in the sender machine.
12. The article of manufacture of claim 1 further comprising:
a bridge coupled between the processor and the shared memory, to forward the voice packets from the processor to the shared memory and forward the interrupt from the counter to the processor.
13. The article of manufacture of claim 12 further comprising:
an embedded controller coupled to the bridge and containing a timer module, the counter being part of the timer module.
14. The article of manufacture of claim 13 wherein the signal that clocks the counter is provided by the DSP system, based upon the bus clock.
15. The article of manufacture of claim 1 further comprising:
a bridge coupled to forward the voice packets from the processor to the shared memory, and wherein the signal that clocks the counter is derived from the bus clock.
16. An article of manufacture comprising:
a machine-readable medium having instructions stored therein which when executed by a set of processors cause
(a) a counter to be programmed to repeatedly generate an interrupt to a first one of the processors at a base rate, the base rate being obtained by clocking the counter with a stratum traceable clock, STC-derived signal,
(b) voice payload, received from a sender machine through a packet-based network, to be processed by the first processor into a plurality of voice packets, and
(c) one or more of the voice packets to be repeatedly written from a buffer to a memory shared by the first processor and a second one of the processors, at a rate derived from the base rate.
17. The article of manufacture of claim 16 wherein the machine-readable medium includes further instructions which, when executed by the set of processors, cause the second processor to process one or more of the voice packets obtained from the shared memory before transmitting them over a bus according to a STC-derived clock.
18. The article of manufacture of claim 17 wherein the base rate is such that voice data in the voice packets, for a given channel, is transferred into the bus at essentially the same rate as voice data for the same channel is collected off a bus in the sender machine.
19. The article of manufacture of claim 18 wherein the machine-readable medium includes volatile and non-volatile semiconductor memory to store the instructions.
20. The article of manufacture of claim 16 wherein the instructions further provide a real time operating system for the first processor, the operating system having a first variable that is updated by an interrupt service routine, in response to each interrupt.
21. The article of manufacture of claim 20 wherein the instructions update a respective credit variable for each voice channel in response to the first variable being changed, the updating of the respective credit variable being a function of a packet size for each channel.
22. The article of manufacture of claim 21 wherein the instructions forward packets from a buffer to the shared memory, based upon the buffer contents satisfying predetermined thresholds between completely full and completely empty.
23. A method comprising:
programming a counter to repeatedly generate an interrupt to a first processor at a base rate, the base rate being obtained by clocking the counter with a stratum traceable clock, STC-derived signal;
processing voice payload, received from a sender machine through a packet-based network, into a plurality of voice packets; and
repeatedly writing one or more of the voice packets from a buffer to a memory shared by the first processor and a second processor, at a rate derived from the base rate.
24. The method of claim 23 further comprising:
processing one or more of the voice packets obtained from the shared memory, before transmitting them over a bus according to a STC-derived clock.
25. The method of claim 24 wherein the base rate is such that voice data in the voice packets, for a given channel, is transferred into the bus at essentially the same rate as voice data for the same channel is collected off a bus in the sender machine.
26. The method of claim 23 further comprising:
updating a first variable of a real time operating system in response to each interrupt.
27. The method of claim 26 further comprising:
updating a respective credit variable for each voice channel in response to the first variable being changed, the updating of the respective credit variable being a function of a packet size for each channel.
28. The method of claim 27 further comprising:
forwarding packets from the buffer to the shared memory, based upon the buffer contents satisfying predetermined thresholds between completely full and completely empty.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A computer implemented method for parallel data redundancy removal, the computer implemented method comprising:
computing a plurality of values for a record in a plurality of records stored in a storage device;
distributing the plurality of values for the record to corresponding queues in a plurality of queues, wherein each of the plurality of queues is associated with a corresponding section of a Bloom filter;
determining whether each value distributed to the corresponding queues for the record is indicated by a corresponding value in the corresponding section of the Bloom filter; and
identifying the record as a redundant record in response to a determination that each value distributed to the corresponding queues for the record is indicated by a corresponding value in the corresponding section of the Bloom filter.
2. The computer implemented method of claim 1, wherein the computing the plurality of values is performed in parallel by threads for multiple records in the plurality of records, and the distributing the plurality of values is performed in parallel by the threads for the multiple records to corresponding queues in the plurality of queues.
3. The computer implemented method of claim 1, further comprising:
determining whether the redundant record duplicates another record in a repository associated with the Bloom filter; and
removing the redundant record from the plurality of records in response to a determination that the redundant record duplicates another record in the repository.
4. The computer implemented method of claim 1, further comprising:
identifying the record as a unique record in response to a determination that each value distributed to the corresponding queues for the record is not indicated by the corresponding value in the corresponding section of the Bloom filter; and
storing the unique record in a repository associated with the Bloom filter.
5. The computer implemented method of claim 4, wherein storing the unique record in the repository comprises setting an indicator in the corresponding value in the corresponding section of the Bloom filter for each value distributed to the corresponding queues for the unique record.
6. The computer implemented method of claim 1, further comprising:
identifying any records in a preliminary plurality of records that are redundant relative to other records in the preliminary plurality of records;
removing any identified redundant records of the preliminary plurality of records from the preliminary plurality of records to create the plurality of records.
7. The computer implemented method of claim 6, wherein identifying any records in the preliminary plurality of records that are redundant relative to other records in the preliminary plurality of records comprises:
assigning a unique identifier to a record in the preliminary plurality of records;
computing a hash value for the record in the preliminary plurality of records based on the corresponding unique identifier;
storing the hash value to a corresponding hash table in a plurality of hash tables; and
merging the plurality of hash tables to identify at least one record of multiple records that correspond to values that are identical for the at least one record in different hash tables as redundant relative to other records in the preliminary plurality of records.
8. The computer implemented method of claim 6, further comprising:
computing a value for a record in an initial plurality of records based on a plurality of fields associated with each record in the initial plurality of records;
distributing the record in the initial plurality of records to a corresponding buffer in a plurality of buffers based on the value for the record in the initial plurality of records, wherein each of the plurality of buffers is associated with a corresponding Bloom filter;
assigning records of a buffer in the plurality of buffers as the preliminary plurality of records;
pre-fetching a second Bloom filter corresponding to the buffer to cache; and
assigning the second Bloom filter corresponding to the buffer as the Bloom filter.
9. A computer usable program product comprising a computer usable storage medium including computer usable code for parallel data redundancy removal, the computer usable code comprising:
computer usable code for computing a plurality of values for a record in a plurality of records stored in a storage device;
computer usable code for distributing the plurality of values for the record to corresponding queues in a plurality of queues, wherein each of the plurality of queues is associated with a corresponding section of a Bloom filter;
computer usable code for determining whether each value distributed to the corresponding queues for the record is indicated by a corresponding value in the corresponding section of the Bloom filter; and
computer usable code for identifying the record as a redundant record in response to a determination that each value distributed to the corresponding queues for the record is indicated by the corresponding value in the corresponding section of the Bloom filter.
10. The computer usable program product of claim 9, wherein the computing the plurality of values is performed in parallel by threads for multiple records in the plurality of records by threads, and the distributing the plurality of values is performed in parallel by the threads for the multiple records to corresponding queues in the plurality of queues by the threads.
11. The computer usable program product of claim 9, further comprising:
computer usable code for determining whether the redundant record duplicates one of the records in a repository associated with the Bloom filter; and
computer usable code for removing the redundant record from the plurality of records in response to a determination that the redundant record duplicates another record in the repository.
12. The computer usable program product of claim 9, further comprising:
computer usable code for identifying the record as a unique record in response to a determination that each value distributed to the corresponding queues for the record is not indicated by the corresponding value in the corresponding section of the Bloom filter; and
computer usable code for storing the unique record in a repository associated with the Bloom filter.
13. The computer usable program product of claim 12, wherein computer usable code for storing the unique record in the repository comprises computer usable code for setting an indicator in the corresponding value in the corresponding section of the Bloom filter for each value distributed to the corresponding queues for the unique record.
14. The computer usable program product of claim 9, further comprising:
computer usable code for identifying any records in a preliminary plurality of records that are redundant relative to other records in the preliminary plurality of records;
computer usable code for removing any identified redundant records in the preliminary plurality of records from the preliminary plurality of records to create the plurality of records.
15. The computer usable program product of claim 14, wherein computer usable code for identifying any records in the preliminary plurality of records that are redundant relative to other records of the preliminary plurality of records comprises:
computer usable code for assigning a unique identifier to a record in the preliminary plurality of records;
computer usable code for computing a hash value for the record in the preliminary plurality of records based on the corresponding unique identifier;
computer usable code for storing the hash value to a corresponding hash table in a plurality of hash tables; and
computer usable code for merging the plurality of hash tables to identify at least one record of multiple records that correspond to values that are identical for the at least one record in different hash tables as redundant relative to other records in the preliminary plurality of records.
16. The computer usable program product of claim 14, further comprising:
computer usable code for computing a value for a record in an initial plurality of records based on a plurality of fields associated with the record in the initial plurality of records;
computer usable code for distributing the record in the initial plurality of records to a corresponding buffer in a plurality of buffers based on the value for the record in the initial plurality of records, wherein each of the plurality of buffers is associated with a corresponding Bloom filter;
computer usable code for assigning records of a buffer in the plurality of buffers as the preliminary plurality of records;
computer usable code for pre-fetching a second Bloom filter corresponding to the buffer to cache; and
computer usable code for assigning the second Bloom filter corresponding to the buffer as the Bloom filter.
17. The computer usable program product of claim 9, wherein the computer usable code is stored in a computer readable storage medium in a data processing system, and wherein the computer usable code is transferred over a network from a remote data processing system.
18. The computer usable program product of claim 9, wherein the computer usable code is stored in a computer readable storage medium in a server data processing system, and wherein the computer usable code is downloaded over a network to a remote data processing system for use in a computer readable storage medium associated with the remote data processing system.
19. A data processing system for parallel data redundancy removal, the data processing system comprising:
a storage device including a storage medium, wherein the storage device stores computer usable program code; and
a processor, wherein the processor executes the computer usable program code, and wherein the computer usable program code comprises:
computer usable code for computing a plurality of values for a record in a plurality of records;
computer usable code for distributing the plurality of values for the record to corresponding queues in a plurality of queues, wherein each of the plurality of queues is associated with a corresponding section of a Bloom filter;
computer usable code for determining whether each value distributed to the corresponding queues for the record is indicated by a corresponding value in the corresponding section of the Bloom filter; and
computer usable code for identifying the record as a redundant record in response to a determination that each value distributed to the corresponding queues for the record is indicated by the corresponding value in the corresponding section of the Bloom filter.
20. The data processing system of claim 19, wherein the computing the plurality of values is performed in parallel by threads for multiple records in the plurality of records by threads, and the distributing the plurality of values is performed in parallel by the threads for the multiple records to corresponding queues in the plurality of queues by the threads.