A VLSI sorting image sensor: Global massively parallel intensity-to-time processing for low-latency adaptive vision

This paper presents a new intensity-to-time processing paradigm suitable for very large scale integration (VLSI) computational sensor implementation of global operations over sensed images. Global image quantities usually describe images with fewer data. When computed at the point of sensing, global quantities result in a low-latency performance due to the reduced data transfer requirements between an image sensor and a processor. The global quantities also help global top-down adaptation: the quantities are continuously computed on-chip, and are readily available to sensing for adaptation. As an example, we have developed a sorting image computational sensor-a VLSI chip which senses an image and sorts all pixel by their intensities. The first sorting sensor prototype is a 21 x 26 array of cells. It receives an image optically, senses it, and computes the image's cumulative histogram-a global quantity which can be quickly routed off chip via one pin. In addition, the global cumulative histogram is used internally on-chip in a top-down fashion to adapt the values in individual pixel so as to reflect the index of the incoming light, thus computing an "image of indices." The image of indices never saturates and has a uniform histogram.