La demanda de sensores de imagen es imparable y su cuota de mercado ha crecido de forma exponencial durante la última década. En este entorno de continua demanda, las tecnologías de integración vertical se perfilan como el vehículo tecnológico a través del cual los sensores de imagen evolucionarán en el futuro.
Las tecnologías de integración vertical permiten que dentro de un mismo píxel pueda incorporarse mayor capacidad de procesamiento y memoria, sin afectar su tamaño ni el ratio entre el área dedicada al sensado de la luz y el área total del píxel (‘fill factor ’). Además, se prevé que varias tecnologías distintas o variantes de una misma tecnología puedan coexistir y aplicarse al diseño de los píxeles.
El proyecto comprende, en su fase inicial, un estudio de viabilidad de las tecnologías de integración vertical modernas con la empresa TELEDYNE-ANAFOCUS como socio preferente. En una fase posterior, se abordará el diseño de dos sensores de imagen que permitan acometer estos objetivos y demostrar la viabilidad de la tecnología mediante la búsqueda de arquitecturas de píxel novedosas que puedan explotar su potencial. La empresa, aportará un estudio detallado sobre las características eléctricas de una tecnología de integración vertical moderna, en base a su experiencia previa. En concreto, detallará las características eléctricas de las interconexiones verticales disponibles. En paralelo, la empresa proporcionará especificaciones de diseño para poder transferir los chips que se diseñen al mercado. Para el diseño de los sensores, se instalará el kit de diseño de la tecnología recomendada por la empresa.
Los sensores a integrar tendrán dos arquitecturas distintas de sensores de imagen. El primero de ellos es un sensor basado en SPADs, con capacidad de medición del tiempo de vuelo. El segundo, es un sensor de alto rango dinámico que combina operación asíncrona basada en eventos con un formato de datos de salida idéntico al de los sensores APS.
Publications in international journals
A Mobile Platform for Movement Tracking based on a Fast-Execution-Time Optical-Flow Algorithm
R. de la Rosa-Vidal, J.A. Leñero-Bardallo, J.M. Guerrero-Rodríguez and A. Rodríguez-Vázquez
Journal Paper · IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, no.4, pp 1715-1727, 2021
abstract
doi
A multi-purpose mechanical platform to track moving objects in three-dimensional space has been developed. It is composed of one main microcontroller board that processes all system data, two cameras, three motors, and one secondary microcontroller board to position a platform with three degrees of freedom. The system computes the optical flow and moves the cameras accordingly, tracking motion within the visual scene. The platform operates autonomously. To the best of our knowledge, there are no similar systems reported with low-resolution image sensors and low-cost microcontrollers. Existing solutions rely on personal computers and advanced FPGAs to process image data. This article concludes that the optical flow operation is efficient even using an image sensor with very low resolution. Thus, the system complexity and image data processing are alleviated significantly. The platform can be easily adapted to different application scenarios by adding new peripherals, sensors, or image processing algorithms. A detailed description of the system design and experimental results are provided.
Uso de la termografía para el análisis y seguimiento de los hemangiomas infantiles
J.A. Leñero, J.A. Pérez-Carrasco, C. Serrano, B. Acha, J.O. Álvárez and J. Bernabéu-Wittel
Conference · Congreso Anual de la Sociedad Española de Ingeniería Biomédica CASEIB 2022
abstract
Los hemangiomas infantiles se encuentran entre los tumores más comunes en los bebés. Para una correcta evolución es fundamental determinar su extensión, volumen y ubicación. Sin embargo, esto puede representar un desafío, ya que los hemangiomas pueden presentar patrones irregulares, estar cubiertos por cabello y su profundidad no se puede estimar fácilmente. Habitualmente, su diagnóstico se basa en estimaciones de área, textura y color. En este trabajo presentamos una técnica alternativa de diagnóstico, la termografía, que es una herramienta útil e inmediata para un examen preciso de los hemangiomas. Los hemangiomas alteran localmente la temperatura corporal, debido a que el flujo sanguíneo en su interior es mayor debido a la proliferación de células endoteliales de los vasos sanguíneos. Por tanto, midiendo las variaciones de temperatura en la piel, se puede estimar la extensión del hemangioma. Se analizaron 30 pacientes con hemangiomas infantiles con una cámara térmica durante varias sesiones en dos fases. A raíz de los resultados obtenidos concluimos que un aumento de la temperatura en el interior del tumor durante el seguimiento era indicativo de una evolución no deseable.
A self-powered asynchronous image sensor with independent in-pixel harvesting and sensing operations
R. Gomez-Merchan, J.A. Leñero-Bardallo and A. Rodríguez-Vázquez
Conference · IS&T International Symposium on Electronic Imaging 2023
abstract
A self-powered asynchronous sensor with a novel pixel architecture is presented. Pixels are autonomous and can harvest or sense energy independently. During the image acquisition, pixels toggle to a harvesting operation mode once they have sensed their local illumination level. With the proposed pixel architecture, most illuminated pixels provide an early contribution to power the sensor, while low-illuminated ones spend more time sensing their local illumination. Thus, the equivalent frame rate is higher than the one offered by conventional self-powered sensors that harvest and sense illumination in independent phases. The proposed sensor uses a Time-to-First-Spike readout that allows trading between image quality and data and bandwidth consumption. The device has HDR operation with a dynamic range of 80 dB. Pixel power consumption is only 70 pW. The article describes the sensors and pixel’s architectures in detail. Experimental results are provided and discussed. Sensor specifications are benchmarked against the art.
A Low-Latency, Low-Power CMOS Sun Sensor for Attitude Calculation Using Photovoltaic Regime and On-Chip Centroid Computation
R. Gomez-Merchan, J. A. Leñero-Bardallo, M. López-Carmona and Á. Rodríguez-Vázquez
Journal Paper · IEEE Transactions on Instrumentation and Measurement, 2023
abstract
doi
The demand for sun sensors has skyrocketed in the last years due to the huge expected deployment of satellites associated with the New Space concept. Sun sensors compute the position of the sun relative to the observer and play a crucial role in navigation systems. However, the sensor itself and the associated electronics must be able to operate in harsh environments. Thus, reducing hardware and post-processing resources improves the robustness of the system. Furthermore, reducing power consumption increases the lifetime of microsatellites with a limited power budget. This work describes the design, implementation, and characterization of a proof-of-concept prototype of a low-power, high-speed sun sensor architecture. The proposed sensor uses photodiodes working in the photovoltaic regime and event-driven vision concepts to overcome the limitations of conventional digital sun sensors in terms of latency, data throughput, and power consumption. The temporal resolution of the prototype is in the microsecond range with an average power consumption lower than 100μW . Experimental results are discussed and compared with the state-of-the-art.
TEMAS: A Flexible non-AI Algorithm for Metrology of Single-core and Core-shell Nanoparticles from TEM images
Jorge J. Saenz Noval, Ruben Gomez-Merchan, Juan A. Leñero-Bardallo and Lionel C. Gontard
Journal Paper · Particle and Particle Systems Characterization, vol. 40, issue: 2, 2023
abstract
doi
An essential application of electron microscopy is to provide feedback to tune the fabrication of nanoparticles (NPs). Real samples tend to follow a size distribution commonly linked to the synthesis process used and in turn to their functional properties. This study presents an algorithm for measuring particle size distributions in electron microscopy images. State-of-the-art methods based on Artificial Intelligence (e.g., Deep Learning) require extensive datasets of labeled images similar to those expected to be analyzed, and extensive supervised re-training is often required for cross-domain application. In contrast, the non-AI algorithm described in this study is accurate and can be quickly set up for measuring new experimental images in different domains. The accuracy of the method is validated quantitatively and comparing graphical and descriptive statistics. Different size distributions are measured on images of platinum and gold nanocatalysts supported on carbon black, amorphous carbon, and titanium dioxide crystals. Also, images of platinum-iron core-shell NPs supported on thin amorphous carbon film are successfully analyzed. The limitation of evaluating different algorithms for NPs metrology is the lack of standards that different researchers can use as ground truth. In order to overcome this limitation, the images and the ground truth measurements presented here are shared as an open dataset.
Other scientific-technical publications directly related to the results of the project
On the Application of Quanta Imaging Acquisition to Spiking Luminance Sensors
R.J. Méndez-Romero, J.A. Leñero-Bardallo and A. Rodríguez-Vázquez
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2022
abstract
Spiking luminance asynchronous sensors are image sensors whose pixels spike asynchronously and autonomously with a frequency proportional to illumination. They can render images with low latency, high dynamic range, and low power consumption. However, one of their drawbacks is the operation under high illumination or with large pixel arrays that can easily saturate their asynchronous readout logic. In this article, the novel quanta imaging acquisition technique is evaluated to alleviate this limitation. Simulation and experimental results demonstrate that the technique is easy to implement. It also reduces the data throughput, keeping acceptable image quality.
Centroid Estimation Method with Sub-pixel Resolution For Event-based Sun Sensors
L. Farian, P. Fernandez-Peramo, P. Hafliger and J.A. Lenero-Bardallo
Conference · Conference on Ph.D Research in Microelectronics and Electronics PRIME 2022
abstract
A new method to obtain sub-pixel measurement resolution for sun sensors based on spiking pixels is presented. The procedure is intended to increase the resolution of the estimated angle. The method uses the profile of incident light to estimate the angle of the vector towards the sun with sub-pixel resolution. Read-out time, data bandwidth, and spatial resolution are improved. Experimental results are provided. The proposed method can be implemented in any asynchronous sun sensor operating in Time-to-First-Spike (TFS) mode.
On the implementation of in-pixel controlled diodes with sensing and energy harvesting capabilities
R. Gomez-Merchan, R. de la Rosa-Vidal, J.A. Leñero-Bardallo and A. Rodríguez-Vázquez
Conference · Conference on Ph.D Research in Microelectronics and Electronics PRIME 2022
abstract
Energy harvesting plays a crucial role in low-power systems and Internet-of-Things (IoT) sensing nodes. Measuring the illumination level of the scene is desired in such applications. Few studies have explored the possibility of designing image sensors that use photodiodes to harvest energy from the scene to reduce consumption or even achieve a self-powered operation using frame-based approaches. This work aims to validate the switching capabilities of photodiodes independently within a photodiode array. While most studies focus on alternating the harvesting and sensing operation in two different phases, in this approach a fraction of the photodiodes are connected to a global node to harvest energy, while the rest are sensing. This configuration qualifies to design asynchronous imagers and optimize the harvesting operation. The preliminary experimental results reported in this publication emphasize the validity of this asynchronous switching in photodiode arrays.
PixiStamp: A tool to acquire, process, and sequence AER data from event-driven systems
R. de la Rosa-Vidal, R. Gomez-Merchan, J.A. Leñero-Bardallo and A. Rodríguez-Vázquez
Conference · Conference on Ph.D Research in Microelectronics and Electronics PRIME 2022
abstract
We present a new tool, PixiStamp, to readout, process, and sequence data of event-driven systems that exchange data using the Address Event Representation (AER) protocol. PixiStamp is a compact acquisition board that can be easily attached to other devices. Over other existing solutions, it has enhanced hardware processing capabilities to process AER data and generate control signals after data processing, making possible a closed-loop device control. The article describes in detail the system architecture, its mechanical design, and its main features.
Making Event-driven and SPAD-based Imagers Converge: A 64x64 110nm CMOS Multi-mode Vision Sensor
R. Gomez-Merchan, J. A. Leñero-Bardallo and Á. Rodríguez-Vázquez
3rd International SPAD Sensor Workshop, June 13-15, 2022
Sensor de imagen con resolución energética para microscopía electrónica
J.J. Sáenz-Noval, J.A. Leñero-Bardallo and L.C. Gontard
JIPII 2022 (II Jornadas de Investigación Predoctoral en Ingeniería informática), June 21 2022, Puerto Real, Cádiz, Spain
A self-powered asynchronous image sensor with independent in-pixel harvesting and sensing operations
R. Gomez-Merchan, J. A. Leñero-Bardallo and Á. Rodríguez-Vázquez
Electronic Imaging 2023, 15-19 January, 2023. San Francisco (USA) Arnaud Darmont Award for Best Paper
Patents directly derived from the results of the project.
Pixel for DVS vision sensors with one or more photodiodes operating in photovoltaic regime
P. Fernández-Peramo and J.A. Leñero-Bardallo
National patent application 202230929, 2022
