1. An image recording device which records an image on a recording medium, said image recording device comprising:
an image processing unit configured to sequentially read, at a second frame rate, images generated by an imaging unit, and to output, from among the images sequentially read, images read at a first frame rate at first resolution and other images at second resolution, the first frame rate being lower than the second frame rate, and the second resolution being lower than the first resolution;
a coding unit configured to generate a first coded stream by coding an image sequence made up of the images having the first resolution, and to generate a second coded stream by coding an image sequence made up of the other images having the second resolution, the images and the other images being outputted by said image processing unit, and the second coded stream being reproducible at the second frame rate when reproduced in combination with the first coded stream; and
a recording unit configured to record, on the recording medium, the first and the second coded streams generated by said coding unit.
2. The image recording device according to claim 1,
wherein said image processing unit is configured to generate the other images having the second resolution by reading only pixels that constitute a part of pixels making up the image generated by the imaging unit.
3. The image recording device according to claim 2,
wherein said image processing unit is configured to read, per line as a unit of reading, at least one remaining unit of reading which is other than at least one unit of reading among units of reading included in the image, the line being made up of horizontally-arranged pixels among the pixels making up the image.
4. The image recording device according to claim 2,
wherein said image processing unit is configured to read, per column as a unit of reading, at least one remaining unit of reading which is other than at least one unit of reading among units of reading included in the image, the column being made up of vertically-arranged pixels among the pixels making up the image.
5. The image recording device according to claim 2,
wherein said image processing unit is configured to specify, from among the pixels making up the image, lines each made up of horizontally-arranged pixels and columns each made up of vertically-arranged pixels, and to read a pixel located at an intersection of each of the specified lines and each of the specified columns.
6. The image recording device according to claim 2,
wherein, when reading the other images while sequentially reading two images at the first frame rate, said image processing unit is configured to read pixels each located at a spatial position that differs from one image to another among the other images.
7. The image recording device according to claim 1,
wherein said image processing unit is configured to output, at the first resolution, an image which is, among the images sequentially read at the second frame rate, other than the images read at the first frame rate and the other images.
8. The image recording device according to claim 1,
wherein said image processing unit is configured to read, at the first resolution, each of the images generated by the imaging unit, and to generate each of the other images having the second resolution by decimating pixels that constitute a part of pixels included in the read image.
9. The image recording device according to claim 1,
wherein said coding unit includes:
a first coding unit configured to code the image sequence made up of the images having the first resolution; and
a second coding unit configured to code the image sequence made up of the other images having the second resolution.
10. An image reproduction device which reads, from a recording medium, a coded stream of an image sequence made up of images captured at a predetermined frame rate, and reproduces the image sequence,
wherein a first coded stream and a second coded stream are recorded on the recording medium, the first coded stream including first images coded at first resolution, and the second coded stream including second images coded at second resolution lower than the first resolution,
said image reproduction device comprising:
a read mechanism unit configured to read the first and the second coded streams from the recording medium;
a decoding unit configured to decode the first images from the first coded stream, and to decode the second images from the second coded stream, the first and the second coded streams being read by said read mechanism unit;
a resolution conversion unit configured to convert a resolution of the second images decoded by said decoding unit to the first resolution; and
a reproduction unit configured to cause a display unit provided outside said image reproduction device to display the first and the second images in image capturing order, by processing, as a single video signal, a video signal indicating the first images decoded by said decoding unit and a video signal indicating the second images having the resolution converted to the first resolution.
11. The image reproduction device according to claim 10,
wherein said resolution conversion unit is further configured to convert a resolution of the first images decoded by said decoding unit to the second resolution, and
said reproduction unit is further configured to cause said display unit to display, in the image capturing order, the second images decoded by said decoding unit and the first images having the resolution converted to the second resolution.
12. The image reproduction device according to claim 10,
wherein said resolution conversion unit is configured to interpolate a pixel included in one of the first images that are decoded, into a same position as a spatial position of the pixel in one of the second images that are decoded, so as to convert the resolution of the one of the second images to the first resolution.
13. The image reproduction device according to claim 10,
wherein each of the second images coded and included in the second coded stream includes a pixel at a spatial position different from a spatial position of a pixel in another one of the second images, and
when assuming that one of the second images that are decoded and sequential in the image capturing order is to be converted, said resolution conversion unit is configured to convert the resolution of the one of the second images to the first resolution by interpolating a pixel included in another one of the second images that is other than the one of the second images that is to be converted, into a same position as the spatial position of the pixel in the one of the second images that is to be converted.
14. The image reproduction device according to claim 10,
wherein said resolution conversion unit is configured to estimate motion between one of the first images that are decoded and one of the second images that are decoded, to compensate the one of the first images with the estimated motion, and to interpolate a pixel included in the one of the first images that is compensated with the estimated motion, into a same position as the spatial position of the pixel in the one of the second images, so as to convert the resolution of the one of the second images to the first resolution.
15. The image reproduction device according to claim 10,
wherein said resolution conversion unit is configured to estimate motion between one of the second images that are decoded and another one of the second images, to compensate the one of the second images with the estimated motion, and to interpolate a pixel included in the one of the second images that is compensated with the estimated motion, into a same position as the spatial position of the pixel in the another one of the second images that is decoded, so as to convert a resolution of the another one of the second images to the first resolution.
16. The image reproduction device according to claim 10,
wherein said decoding unit includes:
a first decoding unit configured to decode the first images from the first coded stream; and
a second decoding unit configured to decode the second images from the second coded stream.
17. An image recording method for recording an image on a recording medium, said image recording method comprising:
sequentially reading, at a second frame rate, images generated by an imaging unit, and outputting, from among the images sequentially read, images read at a first frame rate at first resolution and other images at second resolution, the first frame rate being lower than the second frame rate, and the second resolution being lower than the first resolution;
generating a first coded stream by coding an image sequence made up of the images having the first resolution, and generating a second coded stream by coding an image sequence made up of the other images having the second resolution, the images and the other images being outputted in said sequentially reading, and the second coded stream being reproducible at the second frame rate when reproduced in combination with the first coded stream; and
recording, on the recording medium, the first and the second coded streams generated in said generating.
18. An image reproduction method for reading a coded stream of an image sequence made up of images captured at a predetermined frame rate and reproducing the image sequence,
wherein a first coded stream and a second coded stream are recorded on the recording medium, the first coded stream including first images coded at first resolution and the second coded stream including second images coded at second resolution lower than the first resolution,
said image reproduction device comprising:
reading the first and the second coded streams from the recording medium;
decoding the first images from the first coded stream, and decoding the second images from the second coded stream, the first and the second coded streams being read in said reading;
converting a resolution of the second images decoded in said decoding to the first resolution; and
displaying the first and the second images in image capturing order, by processing, as a single video signal, a video signal indicating the first images decoded in said decoding and a video signal indicating the second images having the resolution converted to the first resolution.
19. A program for recording an image on a recording medium, said program causing a computer to execute:
sequentially reading, at a second frame rate, images generated by an imaging unit, and outputting, from among the images sequentially read, images read at a first frame rate at first resolution and other images at second resolution, the first frame rate being lower than the second frame rate, and the second resolution being lower than the first resolution;
generating a first coded stream by coding an image sequence made up of the images having the first resolution, and generating a second coded stream by coding an image sequence made up of the other images having the second resolution, the images and the other images being outputted in the sequentially reading, and the second coded stream being reproducible at the second frame rate when reproduced in combination with the first coded stream; and
recording, on the recording medium, the first and the second coded streams generated in the generating.
20. A program for reading a coded stream of an image sequence made up of images captured at a predetermined frame rate and reproducing the image sequence,
wherein a first coded stream and a second coded stream are recorded on the recording medium, the first coded stream including first images coded at first resolution and the second coded stream including second images coded at second resolution lower than the first resolution,
said program causing a computer to execute:
causing a read mechanism unit to read the first and the second coded streams from the recording medium;
decoding the first images from the first coded stream, and decoding the second images from the second coded stream, the first and the second coded streams being read by the read mechanism unit;
converting a resolution of the second images decoded in the decoding to the first resolution; and
causing a display unit to display the first and the second images in image capturing order, by processing, as a single video signal, a video signal indicating the first images decoded in the decoding and a video signal indicating the second images having the resolution converted to the first resolution.
The claims below are in addition to those above.
All refrences to claim(s) which appear below refer to the numbering after this setence.
1. A thread arbitration system for use in a processor, the system comprising:
an array comprising a plurality of first entries each corresponding to an element of the array, wherein the array is shared by a plurality of threads, and wherein each entry of the plurality of first entries is configured for use by any thread of the plurality of threads;
control circuitry, wherein in response to determining a given thread has not deallocated any entry of the plurality of first entries for a predetermined first amount of time, the circuitry is further configured to store an indication that the given thread has been identified as a thread hog;
a thread arbitration table comprising a plurality of second entries, wherein each entry of the second entries is configured to indicate a high threshold value and a low threshold value; and
a thread history shift register (HSR) corresponding to the given thread, wherein the thread HSR is used for indexing into the thread arbitration table to select a corresponding second entry, wherein each bit of the thread HSR indicates whether the given thread was identified as a thread hog over a previous predetermined second amount of time.
2. The system as recited in claim 1, wherein in response to determining a number of said plurality of first entries allocated by a given thread is greater than a corresponding high threshold value, the control circuitry is configured to indicate the given thread is stalled from allocating further entries of the first entries, and wherein in response to determining the given thread is stalled, the circuitry is further configured to wait until a number of first entries allocated by the given thread is less than a corresponding low threshold value before releasing the stall of the given thread.
3. The system as recited in claim 2, wherein the circuitry is further configured to indicate the given thread continues a current state of being stalled from allocating, or continues its current state of being permitted to allocate, entries of the first entries in response to determining a number of first entries allocated by the given thread is between the corresponding high threshold value and the corresponding low threshold value.
4. The system as recited in claim 1, further comprising a sample counter, wherein the sample counter is configured to increment each clock cycle that one or more entries of the plurality of first entries are allocated to the given thread, and wherein the circuitry is further configured to shift a thread hog indication value into the thread HSR responsive to detecting the sample counter has reached a predetermined value corresponding to the predetermined second amount of time.
5. The system as recited in claim 4, wherein the circuitry is further configured to:
store an indication in a found thread hog register which indicates the given thread has been identified as a thread hog;
responsive to determining one or more first entries are allocated by the given thread during the predetermined second amount of time:
shift a value from the found thread hog register to the thread HSR; and
reset the found thread hog register.
6. The system as recited in claim 4, wherein the circuitry is further configured to detect the predetermined first amount of time in response to an idle counter reaching a predetermined first count, wherein the idle counter increments each clock cycle that the given thread has entries of allocated first entries and resets responsive to the given thread deallocating an entry of the first entries.
7. The system as recited in claim 5, wherein the circuitry is further configured to select the second entry based at least in part on a number of active threads of the plurality of threads.
8. The system as recited in claim 1, wherein in response to determining each bit of the thread HSR indicates a given thread was a thread hog over each recent predetermined second amount of time, the given thread is identified as a true thread hog, and wherein in response to determining the given thread is a true thread hog, only one group of instructions of the given thread is allowed into a pipeline of the processor at a time.
9. A method for use in a processor, the method comprising:
maintaining an array comprising a plurality of first entries each corresponding to an element of the array, wherein the array is shared by a plurality of threads, and wherein each entry of the plurality of first entries is configured for use by any thread of the plurality of threads;
storing an indication in a found thread hog register which indicates a given thread has been identified as a thread hog in response to determining the given thread has not deallocated any entry of the plurality of first entries for a predetermined first amount of time;
storing a high threshold value and a low threshold value in each entry of a thread arbitration table, wherein the thread arbitration table comprises a second plurality of entries;
shifting a value of the found thread hog register into a corresponding thread history shift register (HSR) responsive to detecting a predetermined second amount of time has elapsed, wherein each bit of the thread HSR indicates whether the given thread was identified as a thread hog over a previous predetermined second amount of time; and
indexing into the thread arbitration table and selecting a second entry with the corresponding thread.
10. The method as recited in claim 9, the method further comprising:
indicating a given thread is stalled from allocating further entries of the first entries, in response to determining a number of said plurality of first entries allocated by the given thread is greater than a corresponding high threshold value; and
in response to determining the given thread is stalled, waiting until a number of first entries allocated by the given thread is less than a corresponding low threshold value before releasing the stall of the given thread.
11. The method as recited in claim 10, wherein the method further comprises indicating the given thread continues its current state of being stalled from allocating, or continues its current state of being permitted to allocate, entries of the first entries in response to determining a number of first entries allocated by the given thread is between the corresponding high threshold value and the corresponding low threshold value.
12. The method as recited in claim 9, further comprising:
incrementing a sample counter each clock cycle that one or more entries of the plurality of first entries are allocated to a given thread; and
shifting a value from the found thread hog register into the thread HSR responsive to detecting the sample counter has reached a predetermined value corresponding to the predetermined second amount of time.
13. The method as recited in claim 12, wherein in response to determining one or more first entries are allocated by the given thread during the predetermined second amount of time, the method further comprises:
shifting a value from the found thread hog register into the thread HSR; and
resetting the found thread hog register.
14. The method as recited in claim 12, further comprising detecting the predetermined first amount of time, in response to an idle counter reaching a predetermined first count, wherein the idle counter increments each clock cycle that the given thread has entries of allocated first entries, and resets in response to the given thread deallocating an entry of the plurality of first entries.
15. The method as recited in claim 13, further comprising selecting the second entry based at least in part on a number of active threads of the plurality of threads.
16. The method as recited in claim 9, further comprising:
identifying a given thread as a true thread hog in response to determining each bit of the thread HSR indicates the given thread was a thread hog over each recent predetermined second amount of time;
allowing only a single group of instructions of the given thread into a pipeline of the processor, in response to determining the given thread is a true thread hog.
17. A non-transitory computer readable storage medium storing program instructions operable to efficiently arbitrate threads in a multi-threaded resource, wherein the program instructions are executable by a processor to:
maintain an array comprising a plurality of first entries each corresponding to an element of the array, wherein the array is shared by a plurality of threads, and wherein each entry of the plurality of first entries is configured for use by any thread of the plurality of threads;
store an indication in a found thread hog register which indicates a given thread has been identified as a thread hog in response to determining the given thread has not deallocated any entry of the plurality of first entries for a predetermined amount of time;
store a high threshold value and a low threshold value in each entry of a thread arbitration table, wherein the thread arbitration table comprises a second plurality of entries;
shift a value of the found thread hog register into a corresponding thread history shift register (HSR) responsive to detecting a predetermined second amount of time has elapsed, wherein each bit of the thread HSR indicates whether the given thread was identified as a thread hog over a previous predetermined second amount of time; and
index into the thread arbitration table and selecting a second entry with the corresponding thread.
18. The non-transitory storage medium as recited in claim 17, wherein the program instructions are further executable to:
indicate a given thread is stalled from allocating further entries of the first entries, in response to determining a number of said plurality of first entries allocated by the given thread is greater than a corresponding high threshold value; and
in response to determining the given thread is stalled, waiting until a number of first entries allocated by the given thread is less than a corresponding low threshold value before releasing the stall of the given thread.