1. An image-sound segment corresponding method comprising:
an image segment classification step that analyzes an input video to generate a plurality of image segment groups, each of the image segment groups including a plurality of image segments which include an identical object;
a sound segment classification step that analyzes the input video to generate a plurality of sound segment groups, each of the sound segment groups including a plurality of sound segments which include an identical object;
an inter-segment group score calculation step that calculates a similarity score between each image segment group and each sound segment group, based on a time duration on which at least one of the image segment group and the sound segment group is present; and
a segment group correspondence decision step that decides, using the scores, whether or not an object in the image segment groups and an object in the sound segment groups are the same.
2. The image-sound segment corresponding method according to claim 1, wherein the inter-segment group score calculation step comprises
calculating the similarity score between each image segment group and each sound segment group, based on a time duration on which the image segment group and the sound segment group are present at the same time.
3. The image-sound segment corresponding method according to claim 1, wherein the image segment classification step comprises
extracting an image feature of an object from each image segment in the input video and classifying a plurality of image segments each having a similar image feature of an object into the same image segment group, and
the sound segment classification step comprises
extracting a sound feature of an object from each sound segment in the input video and classifying a plurality of sound segments each having a similar sound feature of an object into the same sound segment group.
4. The image-sound segment corresponding method according to claim 1, wherein the image segment classification step comprises
classifying the image segments so that each image segment in the input video belongs to one image segment group but does not belong multiply to a plurality of image segment groups, and
the sound segment classification step comprises
classifying the sound segments so that each sound segment in the input video belongs to one sound segment group but does not belong multiply to a plurality of sound segment groups.
5. The image-sound segment corresponding method according to claim 1, wherein a number of image segment groups generated by classifying the image segments in the input video in the image segment classification step and a number of sound segment groups generated by classifying the sound segments in the input video in the sound segment classification step are equal.
6. The image-sound segment corresponding method according to claim 1, wherein the inter-segment group score calculation step comprises
calculating the similarity score between each image segment group and each sound segment group, based on a time duration on which only one of the image segments belonging to the image segment group and the sound segments belonging to the sound segment group is present.
7. The image-sound segment corresponding method according to claim 1, wherein the inter-segment group score calculation step comprises
calculating the similarity score between each image segment group and each sound segment group, using the similarity score between each image segment group and each sound segment group and a weight which is calculated based on one of the image feature and the sound feature extracted from the segments or a combination thereof.
8. The image-sound segment corresponding method according to claim 1, wherein the segment group correspondence decision step comprises
making the decision whether or not an object included in the image segment groups and an object included in the sound segment groups are the same, based on the scores to make correspondence between the image segment groups and the sound segment groups, the correspondence being at least one of one-to-one, one-to-many, and many-to-many correspondence.
9. The image-sound segment corresponding method according to claim 1, wherein the segment group correspondence decision step comprises:
making the decision sequentially whether or not an object included in the image segment groups and an object included in the sound segment groups are the same, for combination of image segment groups and sound segment groups, beginning with a set of an image segment group and a sound segment group for which the highest of the scores is calculated, or for combination of image segment groups and sound segment groups each having a score greater than or equal to a predetermined threshold and beginning with a set of an image segment group and a sound segment group for which the highest of the scores is calculated; and
making one-to one correspondence between the image segment groups and the sound segment groups.
10. The image-sound segment corresponding method according to claim 1, wherein the segment group correspondence decision step comprises
making correspondence between each image segment group and a sound segment group which has the highest score with the image segment group and which does not made to correspond to other image segment groups, or which has the highest score with the image segment group and whose score with other image segment groups is less than or equal to a predetermined threshold, in descending order of time length, or numbers of segments, of the image segments constituting the image segment groups.
11. The image-sound segment corresponding method according to claim 1, wherein the segment group correspondence decision step comprises
making correspondence between each sound segment group and an image segment group which has the highest score with the sound segment group and which does not correspond to other sound segment groups, or which has the highest score with the sound segment group and whose score with other sound segment groups is less than or equal to a predetermined threshold, in descending order of time length, or numbers of segments, of the sound segments belonging to the sound segment groups.
12. The image-sound segment corresponding method according to claim 1, wherein the segment group correspondence decision step comprises
making correspondence between an image segment group and a sound segment group, each of which the highest score is calculated, with regard to the scores of the plurality of image segment groups and the scores of the plurality of sound segment groups.
13. The image-sound segment corresponding method according to claim 1, wherein the segment group correspondence decision step comprises
deciding that the image segment group and the sound segment group do not include an identical object and making no correspondence between image segment group and sound segment group, when the image segment group or the sound segment group has a score, less than or equal to a predetermined threshold, with any sound segment group or image segment group.
14. The image-sound segment corresponding method according to claim 1, wherein the image segment includes a segment generated by dividing a video based on a time position of an appearance or a disappearance of a human face, and
the sound segment includes a segment generated by dividing a video based on a time position of an appearance or a disappearance of a human voice.
15. The image-sound segment corresponding method according to claim 1, wherein when a plurality of faces appear in the same frame, the image segment includes a segment generated by dividing a video based on a time position of an appearance or a disappearance of each face, and
when voices of a plurality of persons are uttered at the same time, the sound segment includes a segment generated by dividing a video based on a time position of an appearance or a disappearance of each voice.
16. The image-sound segment corresponding method according to claim 1, wherein the image segment classification step comprises
analyzing the input video to generate a plurality of facial segment groups, each of facial segment groups including a plurality of image segments which include a face of the same person, as the image segment groups,
the sound segment classification step comprises
analyzing the input video to generate voice segment groups, each of the voice segment groups including sound segments which include a voice of the same person,
the inter-segment group score calculation step comprises
calculating a similarity score between each facial segment group and each voice segment group based on a time duration on which the facial segment group and the voice segment group are present at the same time, and
the segment group correspondence decision step comprises
making the decision, using the scores, whether or not a person in the facial segment groups and a person in the voice segment groups are the same.
17. The image-sound segment corresponding method according to claim 16, wherein the inter-segment group score calculation step comprises
calculating the similarity score between each facial segment group and each voice segment group using a time duration on which the facial segment group and the voice segment group are present at the same time,
at least one of a face position, a face size, and a face direction extracted from each image segment, and
at least one of a voice volume and a voice clarity extracted from each sound segment.
18. The image-sound segment corresponding method according to claim 1, wherein the image segment classification step comprises
classifying the image segments, generated by dividing the video based on a time position of an appearance or a disappearance of a face to generate a plurality of facial segment groups, each including image segments including faces of the same person, as the image segment groups,
the sound segment classification step comprises
classifying the sound segments, generated by dividing the video based on a time position of an occurrence or an extinction of a voice to generate voice segment groups each including sound segments including voices of the same person,
the inter-segment group score calculation step comprises
calculating the similarity score between each facial segment group and each voice segment group based on a time duration on which the facial segment group and the voice segment group are present at the same time, and
the segment group correspondence decision step comprises
making the decision whether or not a person in the facial segment groups and a person in the voice segment groups are the same, beginning with a set of a facial segment group and a voice segment group for which the highest of the scores is calculated, to make one-to one correspondence between the facial segment groups and the voice segment groups.
19. (canceled)
20. An image-sound segment corresponding apparatus comprising:
an image segment classification unit that analyzes an input video to generate a plurality of image segment groups, each of the image segment groups including a plurality of image segments which include an identical object;
a sound segment classification unit that analyzes the input video to generate a plurality of sound segment groups, each of the sound segment groups including a plurality of sound segments which include an identical object;
an inter-segment group score calculation unit that calculates a similarity score between each image segment group and each sound segment group; and
a segment group correspondence decision unit that decides, using the scores, whether or not an object in the image segment groups and an object in the sound segment groups are the same.
21. The image-sound segment corresponding apparatus according to claim 20, further comprising:
an image segment input unit that inputs information on the image segments in the input video;
a sound segment input unit that inputs information on the sound segments in the input video; and
a decision result output unit that outputs a decision result of correspondence between the image segment groups and the sound segment groups, wherein
the image segment input unit supplies image data of the image segments in the input video, and time positions of start points and end points of the image segments in the input video, to the image segment classification unit,
the sound segment input unit supplies sound data on the sound segments in the input video, and time positions of start points and end points of the sound segments in the input video, to the sound segment classification unit,
the image segment classification unit classifies the image segments, based on the image data on the plurality of image segments received from the image segment input unit, generates the plurality of image segment groups, each including a plurality of image segments, and supplies to the inter-segment group score calculation unit the time positions of start points and end points of the image segments, classified into each image segment group, and the image data of the image segments,
the sound segment classification unit classifies the sound segments, based on the sound data on the plurality of sound segments received from the sound segment input unit, generates the plurality of sound segment groups each including a plurality of sound segments, and supplies to the inter-segment group score calculation unit the time positions of start points and end points of the sound segments, classified into each sound segment group, and the sound data on the sound segments,
the inter-segment group score calculation unit uses the time positions of the start points and end points of the image segments, which are received from the image segment classification unit and classified into each image segment group, and the time positions of the start points and end points of the sound segments, which are received from the sound segment classification unit and classified into each sound segment group, calculates the similarity score of each combination of the image segment group and the sound segment group, and supplies to the segment group correspondence decision unit the time positions of the image segments belonging to the image segment groups and the image data, the time positions of the sound segments belonging to the sound segment groups and the sound data, and the scores calculated for each combination of the image segment group and the sound segment group,
the segment group correspondence decision unit decides whether or not an object in the image segment groups and an object in the sound segment groups are the same based on the scores received from the inter-segment group score calculation unit and calculated among the plurality of image segment groups and the plurality of sound segment groups,
the time positions of the image segments belonging to the image segment groups and the image data, the time positions of the sound segments belonging to the sound segment groups and the sound data, and a correspondence decision result between the image segment groups and the sound segment groups are supplied to the decision result output unit, and
the decision result output unit receives and outputs the correspondence decision result.
22. The image-sound segment corresponding apparatus according to claim 20, further comprising
a decision result storage unit that stores therein a decision result of correspondence relation, wherein the segment group correspondence decision unit decides a correspondence relation between the image segment groups and the sound segment groups, based on the scores received from the inter-segment group score calculation unit and the decision result of correspondence between the image segments and the sound segments received from the decision result storage unit, and supplies the decision result to the decision result storage unit.
23. The image-sound segment corresponding apparatus according to claim 20, wherein the image segment classification unit analyzes the input video and generates a plurality of facial segment groups, each of the facial segment groups including image segments which include a face of the same person, as the image segment groups,
the sound segment classification unit analyzes the input video and generates a plurality of sound segment groups, each of the sound segment groups including sound segments which include a voice of the same person,
the inter-segment group score calculation unit calculates a similarity score between each facial segment group and each voice segment group based on a time duration on which the facial segment group and the voice segment group are present at the same time, and
the segment group correspondence decision unit decides, using the scores, whether or not a person in the facial segment groups and a person in the voice segment groups are the same.
24. The image-sound segment corresponding apparatus according to claim 23, wherein the inter-segment group score calculation unit calculates the similarity score between the each facial segment group and the each voice segment group using
a time duration on which the facial segment group and the voice group are present at the same time,
at least one of a face position, a face size, and a face direction extracted from each image segment, and
at least one of a voice volume and a voice clarity extracted from each sound segment.
25. The image-sound segment corresponding apparatus according to claim 20, wherein the image segment classification unit classifies the image segments, generated by dividing the video based on a time position of an appearance or a disappearance of a face, generates facial segment groups, each of the facial segment groups including image segments which include the face of the same person, as the image segment groups,
the sound segment classification unit classifies the sound segments, generated by dividing the video based on a time position of an occurrence or an extinction of a voice to generate voice segment groups, each of the voice segment groups including sound segments which include the voice of the same person,
the inter-segment group score calculation unit calculates the similarity score between each facial segment group and each voice segment group, based on a time duration on which the facial segment group and the voice segment group are present at the same time, and
the segment group correspondence decision unit decides whether or not a person in the facial segment groups and a person in the voice segment groups is the same, beginning with a set of a facial segment group and a voice segment group for which the highest of the scores is calculated, and makes one-to-one correspondence between the facial segment groups and the voice segment groups.
26. A computer-readable storage medium storing a program causing a data processing apparatus to perform the steps of the method according to claim 1.
27. A computer program causing a data processing apparatus to execute:
an image segment classification processing that analyzes an input video to generate a plurality of image segment groups, each of the image segment groups including a plurality of image segments which include an identical object;
a sound segment classification processing that analyzes the input video to generate a plurality of sound segment groups, each of the sound segment groups including a plurality of sound segments which include an identical object;
an inter-segment group score calculation processing that calculates a similarity score between each image segment group and each sound segment group; and
a segment group correspondence decision processing that decides, using the scores, whether or not an object in the image segment groups and an object in the sound segment groups are the same.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A computer-implemented method for providing uniform access to a gateway in an extended virtualized layer-2 network, comprising:
identifying a first media access control (MAC) address in a layer-2 header of a data frame, wherein the first MAC address is associated with a respective gateway in the extended virtualized layer-2 network;
modifying the layer-2 header by swapping the first MAC address with a second MAC address in the layer-2 header, wherein the second MAC address uniquely identifies a gateway in the extended virtualized layer-2 network; and
forwarding the frame with the modified header to the gateway based on the second MAC address.
2. The computer-implemented method of claim 1, wherein the first MAC address is specific to the extended virtualized layer-2 network; and
wherein the first MAC address is not associated with a second extended virtualized layer-2 network.
3. The computer-implemented method of claim 1, wherein the first MAC address corresponds to an Internet Protocol (IP) address, and wherein the IP address is associated with a respective gateway and is uniform in the extended virtualized layer-2 network.
4. The computer-implemented method of claim 3, further comprising maintaining a mapping between the first MAC address and the IP address.
5. The computer-implemented method of claim 4, wherein the mapping is maintained by a virtual machine; and
wherein the method further comprises retaining the mapping during migration of the virtual machine.
6. The computer-implemented method of claim 5, in response to the migration of the virtual machine, further comprising:
modifying the layer-2 header by swapping the first MAC address with a third MAC address in the layer-2 header, wherein the third MAC address uniquely identifies a second gateway in the extended virtualized layer-2 network; and
forwarding the frame with the modified header to the second gateway based on the third MAC address.
7. The computer-implemented method of claim 3, further comprising:
identifying an address resolution query from a virtual machine for the IP address, wherein the virtual machine is associated with a virtualization software;
generating by the virtualization software an address resolution response indicating a correspondence between the first MAC address and the IP address; and
providing the generated address resolution response to the virtual machine.
8. The computer-implemented method of claim 7, further comprising:
identifying an address resolution query from a second virtual machine for the IP address, wherein the second virtual machine is associated with a second extended virtualized layer-2 network;
generating by the virtualization software an address resolution response indicating a correspondence between a third MAC address and the IP address, wherein the third MAC address is associated with a respective gateway in the second extended virtualized layer-2 network; and
providing the generated address resolution response to the second virtual machine.
9. The computer-implemented method of claim 7, further comprising precluding the virtualization software from forwarding the address resolution query to a gateway associated with the IP address
10. The computer-implemented method of claim 1, wherein the extended virtualized layer-2 network is implemented based on one or more of:
a Virtual Extensible Local Area Network (VXLAN);
a Stateless Transport Tunnels (STT);
a Multi-Protocol Label Switching (MPLS) protocol; and
a Generic Routing Encapsulation (GRE) protocol.
11. A non-transitory storage medium storing instructions which when executed by a processor cause the processor to perform a method for providing uniform access to a gateway in an extended virtualized layer-2 network, the method comprising:
identifying a first media access control (MAC) address in a layer-2 header of a data frame, wherein the first MAC address is associated with a respective gateway in the extended virtualized layer-2 network;
modifying the layer-2 header by swapping the first MAC address with a second MAC address in the layer-2 header, wherein the second MAC address uniquely identifies a gateway in the extended virtualized layer-2 network; and
forwarding the frame with the modified header to the gateway based on the second MAC address.
12. The non-transitory storage medium of claim 11, wherein the first MAC address is specific to the extended virtualized layer-2 network; and
wherein the first MAC address is not associated with a second extended virtualized layer2 network.
13. The non-transitory storage medium of claim 11, wherein the first MAC address corresponds to an Internet Protocol (IP) address, and wherein the IP address is associated with a respective gateway and is uniform in the extended virtualized layer-2 network.
14. The non-transitory storage medium of claim 13, wherein the method further comprises maintaining a mapping between the first MAC address and the IP address.
15. The non-transitory storage medium of claim 14, wherein the mapping is maintained by a virtual machine; and
wherein the method further comprises retaining the mapping during migration of the virtual machine.
16. The non-transitory storage medium of claim 15, wherein, in response to the migration of the virtual machine, the method further comprises:
modifying the layer-2 header by swapping the first MAC address with a third MAC address in the layer-2 header, wherein the third MAC address uniquely identifies a second gateway in the extended virtualized layer-2 network; and
forwarding the frame with the modified header to the second gateway based on the third MAC address.
17. The non-transitory storage medium of claim 13, the method further comprises:
identifying an address resolution query from a virtual machine for the IP address, wherein the virtual machine is associated with a virtualization software;
generating by the virtualization software an address resolution response indicating a correspondence between the first MAC address and the IP address; and
providing the generated address resolution response to the virtual machine.
18. The non-transitory storage medium of claim 17, the method further comprises:
identifying an address resolution query from a second virtual machine for the IP address, wherein the second virtual machine is associated with a second extended virtualized layer-2 network;
generating by the virtualization software an address resolution response indicating a correspondence between a third MAC address and the IP address, wherein the third MAC address is associated with a respective gateway in the second extended virtualized layer-2 network; and
providing the generated address resolution response to the second virtual machine.
19. The non-transitory storage medium of claim 17, the method further comprises precluding the virtualization software from forwarding the address resolution query to a gateway associated with the IP address
20. The non-transitory storage medium of claim 11, wherein the extended virtualized layer-2 network is implemented based on one or more of:
a Virtual Extensible Local Area Network (VXLAN);
a Stateless Transport Tunnels (STT);
a Multi-Protocol Label Switching (MPLS) protocol; and
a Generic Routing Encapsulation (GRE) protocol.
21. A computing system for providing uniform access to a gateway in an extended virtualized layer-2 network, the computing system comprising:
a processor; and
a storage device coupled to the processor and storing instructions which when executed by the processor cause the processor to perform a method, the method comprising:
identifying a first media access control (MAC) address in a layer-2 header of a data frame, wherein the first MAC address is associated with a respective gateway in the extended virtualized layer-2 network;
modifying the layer-2 header by swapping the first MAC address with a second MAC address in the layer-2 header, wherein the second MAC address uniquely identifies a gateway in the extended virtualized layer-2 network; and
forwarding the frame with the modified header to the gateway based on the second MAC address.