Found results matching for:
Author: Bernabé Linares Barranco
Year: Since 2002
Journal Papers
A Review of Ising Machines Implemented in Conventional and Emerging Technologies
T. Zhang, Q. Tao, B. Liu, A. Grimaldi, E. Raimondo, M. Jiménez, M.J. Avedillo, J. Núñez, B. Linares-Barranco, T. Serrano-Gotarredona, G. Finocchio and Jie Han
Journal Paper · IEEE Transactions on Nanotechnology (Early Access), 2024
abstract
doi
Ising machines have received growing interest as efficient and hardware-friendly solvers for combinatorial optimization problems (COPs). They search for the absolute or approximate ground states of the Ising model with a proper annealing process. In contrast to Ising machines built with superconductive or optical circuits, complementary metal-oxide-semiconductor (CMOS) Ising machines offer inexpensive fabrication, high scalability, and easy integration with mainstream semiconductor chips. As low-energy and CMOS-compatible emerging technologies, spintronics and pase-transition devices offer functionalities that can enhance the scalability and sampling performance of Ising machines. In this article, we survey various approaches in the process flow for solving COPs using CMOS, hybrid CMOSspintronic, and phase-transition devices. First, the methods for formulating COPs as Ising problems and embedding Ising formulations to the topology of the Ising machine are reviewed. Then, Ising machines are classified by their underlying operational principles and reviewed from a perspective of hardware implementation. CMOS solutions are advantageous with denser connectivity, whereas hybrid CMOS-spintronic and phase-transition device-based solutions show great potential in energy efficiency and high performance. Finally, the challenges and prospects are discussed for the Ising formulation, embedding process, and implementation of Ising machines.
A CMOS-compatible oscillation-based VO2 Ising machine solver
O. Maher, M. Jiménez, C. Delacour, N. Harnack, J. Núñez, M.J. Avedillo, B. Linares-Barranco, A. Todri-Sanial, G. Indiveri and S. Karg
Journal Paper · Nature Communications vol. 15, article 3334, 2024
abstract
doi
Phase-encoded oscillating neural networks offer compelling advantages over metal-oxide-semiconductor-based technology for tackling complex optimization problems, with promising potential for ultralow power consumption and exceptionally rapid computational performance. In this work, we investigate the ability of these networks to solve optimization problems belonging to the nondeterministic polynomial time complexity class using nanoscale vanadium-dioxide-based oscillators integrated onto a Silicon platform. Specifically, we demonstrate how the dynamic behavior of coupled vanadium dioxide devices can effectively solve combinatorial optimization problems, including Graph Coloring, Max-cut, and Max-3SAT problems. The electrical mappings of these problems are derived from the equivalent Ising Hamiltonian formulation to design circuits with up to nine crossbar vanadium dioxide oscillators. Using sub-harmonic injection locking techniques, we binarize the solution space provided by the oscillators and demonstrate that graphs with high connection density (μ > 0.4) converge more easily towards the optimal solution due to the small spectral radius of the problema’s equivalent adjacency matrix. Our findings indicate that these systems achieve stability within 25 oscillation cycles and exhibit power efficiency and potential for scaling that surpasses available commercial options and other technologies under study. These results pave the way for accelerated parallel computing enabled by large-scale networks of interconnected oscillators.
Experimental Demonstration of Coupled Differential Oscillator Networks for Versatile Applications
M. Jiménez, J. Núñez, J. Shamsi, B. Linares-Barranco and M.J. Avedillo
Journal Paper · Frontiers in Neuroscience, Neuromorphic Engineering, vol. 17, 2023
abstract
doi
Oscillatory Neural Networks (ONNs) exhibit a high potential for energy-efficient computing. In ONNs, neurons are implemented with oscillators and synapses with resistive and/or capacitive coupling between pairs of oscillators. Computing is carried out on the basis of the rich, complex, nonlinear synchronization dynamics of a system of coupled oscillators. The exploited synchronization phenomena in ONNs are an example of fully parallel collective computing.A fast system´s convergence to stable states, which correspond to the desired processed information, enables an energy-efficient solution if small area and low-power oscillators are used, specifically, when they are built on the basis of the hysteresis exhibited by phase-transition materials such as VO2. In recent years, there have been numerous studies on ONNs using VO2. Most of them report simulation results. Although in some cases experimental results are also shown, they don´t implement the design techniques that other works on electrical simulations report that allow to improve the behavior of the ONNs.Experimental validation of these approaches is necessary. Therefore, in this work, we describe an ONN realized in a commercial CMOS technology in which the oscillators are built using a circuit that we have developed to emulate the VO2 device. The purpose is to be able to study in depth the synchronization dynamics of relaxation oscillators similar to those that can be performed with VO2 devices. The fabricated circuit is very flexible. It allows programming the synapses to implement different ONNs, calibrating the frequency of the oscillators or controlling their initialization. It uses differential oscillators and resistive synapses equivalent to the use of memristors. In this article, the designed and fabricated circuit is described in detail and experimental results are shown. Specifically, its satisfactory operation as an associative memory is demonstrated. The experiments carried out allow us to conclude that the ONN must be operated according to the type of computational task to be solved, and guidelines are extracted in this regard.
Learning Algorithms for Oscillatory Neural Networks as Associative Memory for Pattern Recognition
M. Jiménez, M.J. Avedillo, B. Linares-Barranco and J. Núñez
Journal Paper · Frontiers in Neuroscience, Neuromorphic Engineering, vol. 17, 2023
abstract
doi
Alternative paradigms to the von Neumann computing scheme are currently arousing huge interest. Oscillatory neural networks (ONNs) using emerging phase-change materials like VO2 constitute an energy-efficient, massively parallel, brain-inspired, in-memory computing approach. The encoding of information in the phase pattern of frequency-locked, weakly coupled oscillators makes it possible to exploit their rich nonlinear dynamics and their synchronization phenomena for computing. A single fully connected ONN layer can implement an auto-associative memory comparable to that of a Hopfield network, hence Hebbian learning rule is the most widely adopted method for configuring ONNs for such applications, despite its well-known limitations. An extensive amount of literature is available about learning in Hopfield networks, with information regarding many different learning algorithms that perform better than the Hebbian rule. However, not all of these algorithms are useful for ONN training due to the constraints imposed by their physical implementation. This paper evaluates different learning methods with respect to their suitability for ONNs. It proposes a new approach, which is compared against previous works. The proposed method has been shown to produce competitive results in terms of pattern recognition accuracy with reduced precision in synaptic weights, and to be suitable for online learning.
Operating Coupled VO2-based Oscillators for Solving Ising Models
M.J. Avedillo, M. Jiménez, C. Delacour, A. Todri-Sanial, B. Linares-Barranco and J. Núñez
Journal Paper · IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2023
abstract
doi
Coupled nano-oscillators are attracting increasing interest because of their potential to perform computation efficiently, enabling new applications in computing and information processing. The potential of phase transition devices for such dynamical systems has recently been recognized. This paper investigates the implementation of coupled VO2-based oscillator networks to solve combinatorial optimization problems. The target problem is mapped to an Ising model, which is solved by the synchronization dynamics of the system. Different factors that impact the probability of the system reaching the ground state of the Ising Hamiltonian and, therefore, the optimum solution to the corresponding optimization problem, are analyzed. The simulation-based analysis has led to the proposal of a novel Second-Harmonic Injection Locking (SHIL) schedule. Its main feature is that SHIL signal amplitude is repeatedly smoothly increased and decreased. Reducing SHIL strength is the mechanism that enables escaping from local minimum energy states. Our experiments show better results in terms of success probability than previously reported approaches. An experimental Oscillatory Ising Machine (OIM) has been built to validate our proposal.
CMOS Front End for Interfacing Spin-Hall Nano-Oscillators for Neuromorphic Computing in the GHz Range
R. Fiorelli, E. Peralias, R. Mendez-Romero, M. Rajabali, A. Kumar, M. Zahedinejad, J. Akerman, F. Moradi, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Electronics, vol. 12, no. 1, article 230, 2023
abstract
doi
Spin-Hall-effect nano-oscillators are promising beyond the CMOS devices currently available, and can potentially be used to emulate the functioning of neurons in computational neuromorphic systems. As they oscillate in the 4-20 GHz range, they could potentially be used for building highly accelerated neural hardware platforms. However, due to their extremely low signal level and high impedance at their output, as well as their microwave-range operating frequency, discerning whether the SHNO is oscillating or not carries a great challenge when its state read-out circuit is implemented using CMOS technologies. This paper presents the first CMOS front-end read-out circuitry, implemented in 180 nm, working at a SHNO oscillation frequency up to 4.7 GHz, managing to discern SHNO amplitudes of 100 mu V even for an impedance as large as 300 ohm and a noise figure of 5.3 dB(300 ohm). A design flow of this front end is presented, as well as the architecture of each of its blocks. The study of the low-noise amplifier is deepened for its intrinsic difficulties in the design, satisfying the characteristics of SHNOs.
Effect of Device Mismatches in Differential Oscillatory Neural Networks
J. Shamsi, M.J. Avedillo, B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 70, no. 2, pp 872-883, 2023
abstract
doi
Analog implementation of Oscillatory Neural Networks (ONNs) has the potential to implement fast and ultra-low-power computing capabilities. One of the drawbacks of analog implementation is component mismatches which cause desynchronization and instability in ONNs. Emerging devices like memristors and VO2are particularly prone to variations. In this paper, we study the effect of component mismatches on the performance of differential ONNs (DONNs). Mismatches were considered in two main blocks: differential oscillatory neurons and synaptic circuits. To measure DONN tolerance to mismatches in each block, performance was evaluated with mismatches being present separately in each block. Memristor-bridge circuits with four memristors were used as the synaptic circuits. The differential oscillatory neurons were based on VO2-devices. The simulation results showed that DONN performance was more vulnerable to mismatches in the components of the differential oscillatory neurons than to mismatches in the synaptic circuits. DONNs were found to tolerate up to 20% mismatches in the memristance of the synaptic circuits. However, mismatches in the differential oscillatory neurons resulted in non-uniformity of the natural frequencies, causing desynchronization and instability. Simulations showed that 0.5% relative standard deviation (RSD) in natural frequencies can reduce DONN performance dramatically. In addition, sensitivity analyses showed that the high threshold voltage of VO2-devices is the most sensitive parameter for frequency non-uniformity and desynchronization.
A CMOS-memristor hybrid system for implementing stochastic binary spike timing-dependent plasticity
J. Ahmadi-Farsani, S. Ricci, S. Hashemkhani, D. Ielmini, B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, vol. 380, no. 2228, article 20210018, 2022
abstract
doi
This paper describes a fully experimental hybrid system in which a 4x4 memristive crossbar spiking neural network (SNN) was assembled using custom high-resistance state memristors with analogue CMOS neurons fabricated in 180 nm CMOS technology. The custom memristors used NMOS selector transistors, made available on a second 180 nm CMOS chip. One drawback is that memristors operate with currents in the micro-amperes range, while analogue CMOS neurons may need to operate with currents in the pico-amperes range. One possible solution was to use a compact circuit to scale the memristor-domain currents down to the analogue CMOS neuron domain currents by at least 5-6 orders of magnitude. Here, we proposed using an on-chip compact current splitter circuit based on MOS ladders to aggressively attenuate the currents by over 5 orders of magnitude. This circuit was added before each neuron. This paper describes the proper experimental operation of an SNN circuit using a 4x4 1T1R synaptic crossbar together with four post-synaptic CMOS circuits, each with a 5-decade current attenuator and an integrate-and-fire neuron. It also demonstrates one-shot winner-takes-all training and stochastic binary spike-timing-dependent-plasticity learning using this small system.
MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems
C. Lammie, W. Xiang, B. Linares-Barranco and M.R. Azghadi
Journal Paper · Neurocomputing, vol. 485, pp.124-133, 2022
abstract
doi
Memristive devices have shown great promise to facilitate the acceleration and improve the power efficiency of Deep Learning (DL) systems. Crossbar architectures constructed using these Resistive Random Access Memory (RRAM) devices can be used to efficiently implement various in-memory computing operations, such as Multiply Accumulate (MAC) and unrolled-convolutions, which are used extensively in Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs). However, memristive devices face concerns of aging and non-idealities, which limit the accuracy, reliability, and robustness of Memristive Deep Learning Systems (MDLSs), that should be considered prior to circuit-level realization. This Original Software Publication(OSP) presents MemTorch, an open-source1 framework for customized large-scale memristive Deep Learning (DL) simulations, with a refined focus on the co simulation of device non-idealities. MemTorch also facilitates co-modelling of key crossbar peripheral circuitry. MemTorch adopts a modernized software engineering methodology and integrates directly with the well-known PyTorch Machine Learning (ML) library.
Liquid State Machine on SpiNNaker for Spatio-Temporal Classification Tasks
A. Patino-Saucedo, H. Rostro-Gonzalez, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 16, article 819063, 2022
abstract
doi
Liquid State Machines (LSMs) are computing reservoirs composed of recurrently connected Spiking Neural Networks which have attracted research interest for their modeling capacity of biological structures and as promising pattern recognition tools suitable for their implementation in neuromorphic processors, benefited from the modest use of computing resources in their training process. However, it has been difficult to optimize LSMs for solving complex tasks such as event-based computer vision and few implementations in large-scale neuromorphic processors have been attempted. In this work, we show that offline-trained LSMs implemented in the SpiNNaker neuromorphic processor are able to classify visual events, achieving state-of-the-art performance in the event-based N-MNIST dataset. The training of the readout layer is performed using a recent adaptation of back-propagation-through-time (BPTT) for SNNs, while the internal weights of the reservoir are kept static. Results show that mapping our LSM from a Deep Learning framework to SpiNNaker does not affect the performance of the classification task. Additionally, we show that weight quantization, which substantially reduces the memory footprint of the LSM, has a small impact on its performance.
A Neuromorphic CMOS Circuit with Self-Repairing Capability
E. Rahiminejad, F. Azad, A. Parvizi-Fard, M. Amiri and B. Linares-Barranco
Journal Paper · IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 5, pp 2246-2258, 2021
abstract
doi
Neurophysiological observations confirm that the brain not only is able to detect the impaired synapses (in brain damage) but also it is relatively capable of repairing faulty synapses. It has been shown that retrograde signaling by astrocytes leads to the modulation of synaptic transmission and thus bidirectional collaboration of astrocyte with nearby neurons is an important aspect of self-repairing mechanism. Specifically, the retrograde signaling via astrocyte can increase the transmission probability of the healthy synapses linked to the neuron. Motivated by these findings, in the present research, a CMOS neuromorphic circuit with self-repairing capabilities is proposed based on astrocyte signaling. In this way, the computational model of self-repairing process is hired as a basis for designing a novel analog integrated circuit in the 180-nm CMOS technology. It is illustrated that the proposed analog circuit is able to successfully recompense the damaged synapses by appropriately modifying the voltage signals of the remaining healthy synapses in the wide range of frequency. The proposed circuit occupies 7500- µm² silicon area and its power consumption is about 65.4 µW. This neuromorphic fault-tolerant circuit can be considered as a key candidate for future silicon neuronal systems and implementation of neurorobotic and neuro-inspired circuits.
How Frequency Injection Locking Can Train Oscillatory Neural Networks to Compute in Phase
A. Todri-Sanial, S. Carapezzi, C. Delacour, M. Abernot, T. Gil, Elisabetta Corti, S.F. Karg, J. Nüñez, M. Jiménez, M.J. Avedillo and B. Linares-Barranco
Journal Paper · IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 5, pp 1996-2009, 2021
abstract
doi
Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model--information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition.
Digital Implementation of Oscillatory Neural Network for Image Recognition Applications
M. Abernot, T. Gil, M. Jiménez, J. Núñez, M.J. Avellido, B. Linares-Barranco, T. Gonos, T. Hardelin and A. Todri-Sanial
Journal Paper · Frontiers in Neuroscience, vol. 15, article 713054, 2021
abstract
doi
Computing paradigm based on von Neuman architectures cannot keep up with the ever-increasing data growth (also called ‘data deluge gap ’). This has resulted in investigating novel computing paradigms and design approaches at all levels from materials to system-level implementations and applications. An alternative computing approach based on artificial neural networks uses oscillators to compute or Oscillatory Neural Networks (ONNs). ONNs can perform computations efficiently and can be used to build a more extensive neuromorphic system. Here, we address a fundamental problem: can we efficiently perform artificial intelligence applications with ONNs? We present a digital ONN implementation to show a proof-of-concept of the ONN approach of ‘computing-in-phase’ for pattern recognition applications. To the best of our knowledge, this is the first attempt to implement an FPGA-based fully-digital ONN. We report ONN accuracy, training, inference, memory capacity, operating frequency, hardware resources based on simulations and implementations of 5 × 3 and 10 × 6 ONNs. We present the digital ONN implementation on FPGA for pattern recognition applications such as performing digits recognition from a camera stream. We discuss practical challenges and future directions in implementing digital ONN.
Neutron-Induced, Single-Event Effects on Neuromorphic Event-Based Vision Sensor: A First Step and Tools to Space Applications
S. Roffe, H. Akolkar, A.D. George, B. Linares-Barranco and R.B. Benosman
Journal Paper · IEEE Access, vol. 9, pp 85748-85763, 2021
abstract
doi
This paper studies the suitability of neuromorphic event-based vision cameras for spaceflight and the effects of neutron radiation on their performance. Neuromorphic event-based vision cameras are novel sensors that implement asynchronous, clockless data acquisition, providing information about the change in illuminance ≥120dB with sub-millisecond temporal precision. These sensors have huge potential for space applications as they provide an extremely sparse representation of visual dynamics while removing redundant information, thereby conforming to low-resource requirements. An event-based sensor was irradiated under wide-spectrum neutrons at Los Alamos Neutron Science Center and its effects were classified. Radiation-induced damage of the sensor under wide-spectrum neutrons was tested, as was the radiative effect on the signal-to-noise ratio of the output at different angles of incidence from the beam source. We found that the sensor had very fast recovery during radiation, showing high correlation of noise event bursts with respect to source macro-pulses. No statistically significant differences were observed between the number of events induced at different angles of incidence but significant differences were found in the spatial structure of noise events at different angles. The results show that event-based cameras are capable of functioning in a space-like, radiative environment with a signal-to-noise ratio of 3.355. They also show that radiation-induced noise does not affect event-level computation. Finally, we introduce the Event-based Radiation-Induced Noise Simulation Environment (Event-RINSE), a simulation environment based on the noise-modelling we conducted and capable of injecting the effects of radiation-induced noise from the collected data to any stream of events in order to ensure that developed code can operate in a radiative environment. To the best of our knowledge, this is the first time such analysis of neutron-induced noise has been performed on a neuromorphic vision sensor, and this study shows the advantage of using such sensors for space applications.
SL-Animals-DVS: event-driven sign language animals dataset
A. Vasudevan, P. Negri, C. di Ielsi, B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · Pattern Analysis and Applications, vol. 24, no. 2, 2021
abstract
doi
Non-intrusive visual-based applications supporting the communication of people employing sign language for communication are always an open and attractive research field for the human action recognition community. Automatic sign language interpretation is a complex visual recognition task where motion across time distinguishes the sign being performed. In recent years, the development of robust and successful deep-learning techniques has been accompanied by the creation of a large number of databases. The availability of challenging datasets of Sign Language (SL) terms and phrases helps to push the research to develop new algorithms and methods to tackle their automatic recognition. This paper presents ‘SL-Animals-DV’, an event-based action dataset captured by a Dynamic Vision Sensor (DVS). The DVS records non-fluent signers performing a small set of isolated words derived from SL signs of various animals as a continuous spike flow at very low latency. This is especially suited for SL signs which are usually made at very high speeds. We benchmark the recognition performance on this data using three state-of-the-art Spiking Neural Networks (SNN) recognition systems. SNNs are naturally compatible to make use of the temporal information that is provided by the DVS where the information is encoded in the spike times. The dataset has about 1100 samples of 59 subjects performing 19 sign language signs in isolation at different scenarios, providing a challenging evaluation platform for this emerging technology.
Hardware Implementation of Differential Oscillatory Neural Networks using VO2-based Oscillators and Memristor-Bridge Circuits
J. Shamsi, M.J. Avedillo, B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · Frontiers in Neuroscience, vol. 15, article 674567, 2021
abstract
doi
Oscillatory Neural Networks (ONNs) are currently arousing interest in the research community for their potential to implement very fast, ultra-low-power computing tasks by exploiting specific emerging technologies. From the architectural point of view, ONNs are based on the synchronization of oscillatory neurons in cognitive processing, as occurs in the human brain. As emerging technologies, VO2 and memristive devices show promising potential for the efficient implementation of ONNs. Abundant literature is now becoming available pertaining to the study and building of ONNs based on VO2 devices and resistive coupling, such as memristors. One drawback of direct resistive coupling is that physical resistances cannot be negative, but from the architectural and computational perspective this would be a powerful advantage when interconnecting weights in ONNs. Here we solve the problem by proposing a hardware implementation technique based on differential oscillatory neurons for ONNs (DONNs) with VO2-based oscillators and memristor-bridge circuits. Each differential oscillatory neuron is made of a pair of VO2 oscillators operating in anti-phase. This way, the neurons provide a pair of differential output signals in opposite phase. The memristor-bridge circuit is used as an adjustable coupling function that is compatible with differential structures and capable of providing both positive and negative weights. By combining differential oscillatory neurons and memristor-bridge circuits, we propose the hardware implementation of a fully connected differential ONN (DONN) and use it as an associative memory. The standard Hebbian rule is used for training, and the weights are then mapped to the memristor-bridge circuit through a proposed mapping rule. The paper also introduces some functional and hardware specifications to evaluate the design. Evaluation is performed by circuit-level electrical simulations and shows that the retrieval accuracy of the proposed design is comparable to that of classic Hopfield Neural Networks.
Insights into the Dynamics of Coupled VO2 Oscillators for ONNs
J. Núñez, J.M. Quintana, M.J. Avedillo, M. Jiménez, A. Todri-Sanial, E. Corti, S. Karg and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 68, no. 10, pp 3356-3360, 2021
abstract
doi
The collective behavior of many coupled oscillator systems is currently being explored for the implementation of different non-conventional computing paradigms. In particular, VO2 based nano-oscillators have been proposed to implement oscillatory neural networks that can serve as associative memories, useful in pattern recognition applications. Although the dynamics of a pair of coupled oscillators have already been extensively analyzed, in this paper, the topic is addressed more practically. Firstly, for the application mentioned above, each oscillator needs to be initialized in a given phase to represent the input pattern. We demonstrate the impact of this initialization mechanism on the final phase relationship of the oscillators. Secondly, such oscillatory networks are based on frequency synchronization, in which the impact of variability is critical. We carried out a comprehensive mathematical analysis of a pair of coupled oscillators taking into account both issues, which is a first step towards the design of the oscillatory neural networks for associative memory applications.
Oscillatory Neural Networks using VO2 based Phase Encoded Logic
J. Núñez, M.J. Avedillo, M. Jiménez, J.M. Quintana, A. Todri-Sanial, E. Corti, S. Karg and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 15, article 655823, 2021
abstract
doi pdf
Nano-oscillators based on phase-transition materials are being explored for the implementation of different non-conventional computing paradigms. In particular, vanadium dioxide (VO 2) devices are used to design autonomous non-linear oscillators from which oscillatory neural networks (ONNs) can be developed. In this work, we propose a new architecture for ONNs in which sub-harmonic injection locking (SHIL) is exploited to ensure that the phase information encoded in each neuron can only take two values. In this sense, the implementation of ONNs from neurons that inherently encode information with two-phase values has advantages in terms of robustness and tolerance to variability present in VO2 devices. Unlike conventional interconnection schemes, in which the sign of the weights is coded in the value of the resistances, in our proposal the negative (positive) weights are coded using static inverting (non-inverting) logic at the output of the oscillator. The operation of the proposed architecture is shown for pattern recognition applications.
Efficient Spike-Driven Learning With Dendritic Event-Based Processing
S. Yang, T. Gao, J. Wang, B. Deng, B. Lansdell and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 15, article 601109, 2021
abstract
doi pdf
A critical challenge in neuromorphic computing is to present computationally efficient algorithms of learning. When implementing gradient-based learning, error information must be routed through the network, such that each neuron knows its contribution to output, and thus how to adjust its weight. This is known as the credit assignment problem. Exactly implementing a solution like backpropagation involves weight sharing, which requires additional bandwidth and computations in a neuromorphic system. Instead, models of learning from neuroscience can provide inspiration for how to communicate error information efficiently, without weight sharing. Here we present a novel dendritic event-based processing (DEP) algorithm, using a two-compartment leaky integrate-and-fire neuron with partially segregated dendrites that effectively solves the credit assignment problem. In order to optimize the proposed algorithm, a dynamic fixed-point representation method and piecewise linear approximation approach are presented, while the synaptic events are binarized during learning. The presented optimization makes the proposed DEP algorithm very suitable for implementation in digital or mixed-signal neuromorphic hardware. The experimental results show that spiking representations can rapidly learn, achieving high performance by using the proposed DEP algorithm. We find the learning capability is affected by the degree of dendritic segregation, and the form of synaptic feedback connections. This study provides a bridge between the biological learning and neuromorphic learning, and is meaningful for the real-time applications in the field of artificial intelligence.
Neuromorphic Low-power Inference on Memristive Crossbars with On-chip Offset Calibration
C. Mohan, L.A. Camuñas-Mesa, J.M. de la Rosa, E. Vianello, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Access, vol. 9, pp 38043-38061, 2021
abstract
doi pdf
Monolithic integration of silicon with nano-sized Redox-based resistive Random-Access Memory (ReRAM) devices opened the door to the creation of dense synaptic connections for bio-inspired neuromorphic circuits. One drawback of OxRAM based neuromorphic systems is the relatively low ON resistance of OxRAM synapses (in the range of just a few kilo-ohms). This requires relatively large currents (many micro amperes per synapse), and therefore imposes strong driving capability demands on peripheral circuitry, limiting scalability and low power operation. After learning, however, a read inference can be made low-power by applying very small amplitude read pulses, which require much smaller driving currents per synapse. Here we propose and experimentally demonstrate a technique to reduce the amplitude of read inference pulses in monolithic neuromorphic CMOS OxRAM-synaptic crossbar systems. Unfortunately, applying tiny read pulses is non-trivial due to the presence of random DC offset voltages. To overcome this, we propose finely calibrating DC offset voltages using a bulk-based three-stage on-chip calibration technique. In this work, we demonstrate spiking pattern recognition using STDP learning on a small 4x4 proof-of-concept memristive crossbar, where on-chip offset calibration is implemented and inference pulse amplitude could be made as small as 2mV. A chip with pre-synaptic calibrated input neuron drivers and a 4x4 1T1R synapse crossbar was designed and fabricated in the CEA-LETI MAD200 technology, which uses monolithic integration of OxRAMs above ST130nm CMOS. Custom-made PCBs hosting the post-synaptic circuits and control FPGAs were used to test the chip in different experiments, including synapse characterization, template matching, and pattern recognition using STDP learning, and to demonstrate the use of on-chip offset-calibrated low-power amplifiers. According to our experiments, the minimum possible inference pulse amplitude is limited by offset voltage drifts and noise. We conclude the paper with some suggestions for future work in this direction.
Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
M. Rahimiazghadi, C. Lammie, J.K. Eshraghian, M. Payvand, E. Donati, B. Linares-Barranco and G. Indiveri
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 14, no. 6, pp 1138-1159, 2020
abstract
doi
With the advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors, new opportunities are emerging for applying deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of the medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies ranging from emerging memristive devices, to established Field Programmable Gate Arrays (FPGAs), and mature Complementary Metal Oxide Semiconductor (CMOS) technology can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. After providing the required background, we unify the sparsely distributed research on neural network and neuromorphic hardware implementations as applied to the healthcare domain. In addition, we benchmark various hardware platforms by performing a biomedical electromyography (EMG) signal processing task and drawing comparisons among them in terms of inference delay and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that different accelerators and neuromorphic processors introduce to healthcare and biomedical domains. This paper can serve a large audience, ranging from nanoelectronics researchers to biomedical and healthcare practitioners in grasping the fundamental interplay between hardware, algorithms, and clinical adoption of these tools, as we shed light on the future of deep networks and spiking neuromorphic processing systems.
Enhanced Linearity in FD-SOI CMOS Body-Input Analog Circuits - Application to Voltage- Controlled Ring Oscillators and Frequency-Based ΣΔ ADCs
J. Ahmadi-Farsani, V. Zúñiga-González, T. Serrano-Gotarredona, B. Linares-Barranco and J.M. de la Rosa
Journal Paper · IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 67 ,no. 10, pp 3297-3308, 2020
abstract
doi
This paper investigates the use of the body terminal of MOS transistors to improve the linearity of some key circuits used to implement analog and mixed-signal circuits integrated in Fully Depleted Silicon on Insulator (FD-SOI) CMOS. This technology allows to increase the body factor with respect to conventional (bulk) CMOS processes. This effect is analyzed in basic analog building blocks - such as switches, simple-stage transconductors and Voltage-Controlled Ring Oscillators (VCROs). Approximated expressions are derived for the nonlinear characteristics and harmonic distortion of some of these circuits. As an application, transistor-level simulations of two VCRO-based ΣΔ modulators designed in a 28-nm FD-SOI CMOS technology are shown in order to demonstrate the benefits of the presented techniques.
Sound Source Localization in Wide-Range Outdoor Environment using Distributed Sensor Network
M. Faraji, S.B. Shouraki, E. Iranmehr and B. Linares-Barranco
Journal Paper · IEEE Sensors Journal, vol. 20, no. 4, pp 2234-2246, 2020
abstract
doi pdf
Sound source localization has always been one of the most challenging subjects in different fields of engineering, one of the most important of which being tracking of flying objects. This article focuses on sound source localization using fuzzy fusion and a beamforming method. It proposes a new fuzzy-based algorithm for localizing a sound source using distributed sensor nodes. Eight low-cost sensor nodes have been constructed in this study each of which consists of a microphone array to capture sound waves. Each node is able to record audio signals synchronously on an SD card to evaluate different algorithms offline. However, the sensor nodes are designed to be able to estimate the location of the sound source in real-time. In the proposed algorithm, every node estimates the direction of the sound source. Moreover, a calibration algorithm is used for extracting the orientation of sensor nodes to calibrate the estimated directions. The calibrated directions are fuzzified and then used for localizing the sound source by fuzzy fusion. An experiment was designed based on localizing a flying quadcopter as a moving sound source to evaluate the performance of the proposed algorithm. The flying trajectory was then estimated and compared with the target trajectory extracted from the GPS module mounted on the quadcopter. Comparing the estimated sound source with the target location, a mean distance error of ${6.03}{m}$ was achieved in a wide-range outdoor environment with the size of ${240}\times {160}\times {80} \,\,{m}^{{3}}$ . The achieved mean distance error is reasonable regarding the mean precision of the GPS module. The practical results illustrate the effectiveness of the proposed approach in localizing a sound source in a wide-range outdoor environment.
Event-driven implementation of deep spiking convolutional neural networks for supervised classification using the SpiNNaker neuromorphic platform
A. Patino-Saucedo, H. Rostro-Gonzalez, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Neural Networks, vol. 121, pp 319-328, 2020
abstract
doi pdf
Neural networks have enabled great advances in recent times due mainly to improved parallel computing capabilities in accordance to Moore′s Law, which allowed reducing the time needed for the parameter learning of complex, multi-layered neural architectures. However, with silicon technology reaching its physical limits, new types of computing paradigms are needed to increase the power efficiency of learning algorithms, especially for dealing with deep spatio-temporal knowledge on embedded applications. With the goal of mimicking the brain′s power efficiency, new hardware architectures such as the SpiNNaker board have been built. Furthermore, recent works have shown that networks using spiking neurons as learning units can match classical neural networks in supervised tasks. In this paper, we show that the implementation of state-of-the-art models on both the MNIST and the event-based NMNIST digit recognition datasets is possible on neuromorphic hardware. We use two approaches, by directly converting a classical neural network to its spiking version and by training a spiking network from scratch. For both cases, software simulations and implementations into a SpiNNaker 103 machine were performed. Numerical results approaching the state of the art on digit recognition are presented, and a new method to decrease the spike rate needed for the task is proposed, which allows a significant reduction of the spikes (up to 34 times for a fully connected architecture) while preserving the accuracy of the system. With this method, we provide new insights on the capabilities offered by networks of spiking neurons to efficiently encode spatio-temporal information.
Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity
A. Yousefzadeh, M.A. Khoei, S. Hoseini, P. Holanda, S. Leroux, O. Moreira, J. Tapson, B. Dhoedt, P. Simoens, T. Serrano-Gotarredona, B. Linares-Barranco
Journal Paper · IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 4, pp 668-678, 2019
abstract
doi
Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms.
Bio-Inspired Evolutionary Model of Spiking Neural Networks in Ionic Liquid Space
E. Iranmehr, S.B. Shouraki, M.M. Faraji, N. Bagheri and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 13, article 1085, 2019
abstract
doi pdf
One of the biggest struggles while working with artificial neural networks is being able to come up with models which closely match biological observations. Biological neural networks seem to capable of creating and pruning dendritic spines, leading to synapses being changed, which results in higher learning capability. The latter forms the basis of the present study in which a new ionic model for reservoir-like networks, consisting of spiking neurons, is introduced. High plasticity of this model makes learning possible with a fewer number of neurons. In order to study the effect of the applied stimulus in an ionic liquid space through time, a diffusion operator is used which somehow compensates for the separation between spatial and temporal coding in spiking neural networks and therefore, makes the mentioned model suitable for spatiotemporal patterns. Inspired by partial structural changes in the human brain over the years, the proposed model evolves during the learning process. The effect of topological evolution on the proposed model's performance for some classification problems is studied in this paper. Several datasets have been used to evaluate the performance of the proposed model compared to the original LSM. Classification results via separation and accuracy values have shown that the proposed ionic liquid outperforms the original LSM.
A Neuromorphic Digital Circuit for Neuronal Information Encoding using Astrocytic Calcium Oscillations
F. Faramarzi, F. Azad, M. Amiri and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 13, article 998, 2019
abstract
doi pdf
Neurophysiological observations are clarifying how astrocytes can actively participate in information processing and how they can encode information through frequency and amplitude modulation of intracellular Ca2+ signals. Consequently, hardware realization of astrocytes is important for developing the next generation of bio-inspired computing systems. In this paper, astrocytic calcium oscillations and neuronal firing dynamics are presented by De Pittà and IF (Integrated & Fire) models, respectively. Considering highly nonlinear equations of the astrocyte model, linear approximation and single constant multiplication (SCM) techniques are employed for efficient hardware execution while maintaining the dynamic of the original models. This low-cost hardware architecture for the astrocyte model is able to show the essential features of different types of Ca2+ modulation such as amplitude modulation (AM), frequency modulation (FM), or both modes (AFM). To show good agreement between the results of original models simulated in MATLAB and the proposed digital circuits executed on FPGA, quantitative, and qualitative analyses including phase plane are done. This new neuromorphic circuit of astrocyte is able to successfully demonstrate AM/FM/AFM calcium signaling in its real operation on FPGA and has applications in self-repairing systems. It also can be employed as a subsystem for linking biological cells to artificial neuronal networks using astrocytic calcium oscillations in future research.
Neuromorphic spiking neural networks and their memristor-CMOS hardware implementations
L.A. Camuñas-Mesa, B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · Materials, vol. 12, no. 7, article 2745, 2019
abstract
doi pdf
Inspired by biology, neuromorphic systems have been trying to emulate the human brain for decades, taking advantage of its massive parallelism and sparse information coding. Recently, several large-scale hardware projects have demonstrated the outstanding capabilities of this paradigm for applications related to sensory information processing. These systems allow for the implementation of massive neural networks with millions of neurons and billions of synapses. However, the realization of learning strategies in these systems consumes an important proportion of resources in terms of area and power. The recent development of nanoscale memristors that can be integrated with Complementary Metal-Oxide-Semiconductor (CMOS) technology opens a very promising solution to emulate the behavior of biological synapses. Therefore, hybrid memristor-CMOS approaches have been proposed to implement large-scale neural networks with learning capabilities, offering a scalable and lower-cost alternative to existing CMOS systems.
Digital hardware realization of a novel adaptive ink drop spread operator and its application in modeling and classification and on-chip training
S. Haghzad Klidbary, S. Bagheri Shouraki and B. Linares-Barranco
Journal Paper · International Journal of Machine Learning and Cybernetics, vol. 10, no. 9, pp 2541-2561, 2019
abstract
doi pdf
In artificial intelligence (AI), proposing an efficient algorithm with an appropriate hardware implementation has always been a challenge because of the well-accepted fact that AI hardware implementations should ideally be comparable to biological systems in terms of hardware area. Active learning method (ALM) is a fuzzy learning algorithm inspired by human brain computations. Unlike traditional algorithms, which employ complicated computations, ALM tries to model human brain computations using qualitative and behavioral descriptions of the problem. The main computational engine in ALM is the ink drop spread (IDS) operator, but this operator imposes high memory requirements and computational costs, making the ALM algorithm and its hardware implementation unsuitable for some of the applications. This paper proposes an adaptive alternative method for implementing the IDS operator; a method which results in a marked reduction in the algorithm´s computational complexity and in the amount of memory required and hardware. To check its validity and performance, the method was used to carry out modeling and pattern classification tasks. This paper used challenging and real-world datasets and compared with well-known algorithms (adaptive neuro-fuzzy inference system and multi-layer perceptron) in software simulation and hardware implementation. Compared to traditional implementations of the ALM algorithm and other learning algorithms, the proposed FPGA implementation offers higher speed, less hardware, and improved performance, thus facilitating real-time application. Our ultimate goal in this paper was to present a hardware implementation with an on-chip training that allows it to adapt to its environment without dependency on the host system (on-chip learning).
On practical issues for stochastic STDP hardware with 1-bit synaptic weights
A. Yousefzadeh, E. Stromatias, M. Soto, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 12, article 665, 2018
abstract
doi pdf
In computational neuroscience, synaptic plasticity learning rules are typically studied using the full 64-bit floating point precision computers provide. However, for dedicated hardware implementations, the precision used not only penalizes directly the required memory resources, but also the computing, communication, and energy resources. When it comes to hardware engineering, a key question is always to find the minimum number of necessary bits to keep the neurocomputational system working satisfactorily. Here we present some techniques and results obtained when limiting synaptic weights to 1-bit precision, applied to a Spike-Timing-Dependent-Plasticity (STDP) learning rule in Spiking Neural Networks (SNN). We first illustrate the 1-bit synapses STDP operation by replicating a classical biological experiment on visual orientation tuning, using a simple four neuron setup. After this, we apply 1-bit STDP learning to the hidden feature extraction layer of a 2-layer system, where for the second (and output) layer we use already reported SNN classifiers. The systems are tested on two spiking datasets: a Dynamic Vision Sensor (DVS) recorded poker card symbols dataset and a Poisson-distributed spike representation MNIST dataset version. Tests are performed using the in-house MegaSim event-driven behavioral simulator and by implementing the systems on FPGA (Field Programmable Gate Array) hardware.
Calibration of offset via bulk for low-power HfO2 based 1T1R memristive crossbar read-out system
C. Mohan, L.A. Camuñas-Mesa, E. Vianello, L. Periniolla, C. Reita, J.M. de la Rosa, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Microelectronic Engineering, vol. 198, pp 35-47, 2018
abstract
doi
Neuromorphic RRAM circuits typically need currents of several mA when many binary memristive devices are activated at the same time. This is due to the low resistance state of these devices, which increases the power consumption and limits the scalability. To overcome this limitation, it is vital to investigate how to minimize the amplitude of the read-out inference pulses sent through the crossbar lines. However, the amplitude of such inference voltage pulses will become limited by the offset voltage of read-out circuits. This paper presents a three-stage calibration circuit to compensate for offset voltage in the wordlines of a memristor-array read-out system. The proposed calibration scheme is based on adjusting the bulk voltage of one of the input differential pair MOSFETs by means of a switchable cascade of resistor ladders. This renders the possibility to obtain calibration voltage steps less than 0.1mV by cascading a few number of stages, whose results are only limited by mismatch, temperature, electrical noise and other fabrication defects. The system is built using HfO2-based binary memristive synaptic devices on top of a 130-nm CMOS technology. Layout-extracted simulations considering technology corners, PVT variations and electrical noise are shown to validate the presented calibration scheme.
Active Perception with Dynamic Vision Sensors. Minimum Saccades with Optimum Recognition
A. Yousefzadeh, G. Orchard, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 12, no. 4, pp 927-939, 2018
abstract
doi
Vision processing with dynamic vision sensors (DVSs) is becoming increasingly popular. This type of a bio-inspired vision sensor does not record static images. The DVS pixel activity relies on the changes in light intensity. In this paper, we introduce a platform for the object recognition with a DVS in which the sensor is installed on a moving pan-tilt unit in a closed loop with a recognition neural network. This neural network is trained to recognize objects observed by a DVS, while the pan-tilt unit is moved to emulate micro-saccades. We show that performing more saccades in different directions can result in having more information about the object, and therefore, more accurate object recognition is possible. However, in high-performance and low-latency platforms, performing additional saccades adds latency and power consumption. Here, we show that the number of saccades can be reduced while keeping the same recognition accuracy by performing intelligent saccadic movements, in a closed action-perception smart loop. We propose an algorithm for smart saccadic movement decisions that can reduce the number of necessary saccades to half, on average, for a predefined accuracy on the N-MNIST dataset. Additionally, we show that by replacing this control algorithm with an artificial neural network that learns to control the saccades, we can also reduce to half the average number of saccades needed for the N-MNIST recognition.
A Configurable Event-Driven Convolutional Node with Rate Saturation Mechanism for Modular ConvNet Systems Implementation
L.A. Camuñas-Mesa, Y.L. Domínguez-Cordero, A. Linares-Barranco, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 12, article 63, 2018
abstract
doi pdf
Convolutional Neural Networks (ConvNets) are a particular type of neural network often used for many applications like image recognition, video analysis or natural language processing. They are inspired by the human brain, following a specific organization of the connectivity pattern between layers of neurons known as receptive field. These networks have been traditionally implemented in software, but they are becoming more computationally expensive as they scale up, having limitations for real-time processing of high-speed stimuli. On the other hand, hardware implementations show difficulties to be used for different applications, due to their reduced flexibility. In this paper, we propose a fully configurable event-driven convolutional node with rate saturation mechanism that can be used to implement arbitrary ConvNets on FPGAs. This node includes a convolutional processing unit and a routing element which allows to build large 2D arrays where any multilayer structure can be implemented. The rate saturation mechanism emulates the refractory behavior in biological neurons, guaranteeing a minimum separation in time between consecutive events. A 4-layer ConvNet with 22 convolutional nodes trained for poker card symbol recognition has been implemented in a Spartan6 FPGA. This network has been tested with a stimulus where 40 poker cards were observed by a Dynamic Vision Sensor (DVS) in 1 s time. Different slow-down factors were applied to characterize the behavior of the system for high speed processing. For slow stimulus play-back, a 96% recognition rate is obtained with a power consumption of 0.85 mW. At maximum play-back speed, a traffic control mechanism downsamples the input stimulus, obtaining a recognition rate above 63% when less than 20% of the input events are processed, demonstrating the robustness of the network.
Memristors fire away
B. Linares-Barranco
Journal Paper · Nature Electronics, vol. 1, no. 2, pp 100-101, 2018
abstract
doi
Neuromorphic computing based on fully memristive neural networks could offer a scalable and lower-cost alternative to existing neural spiking chips based solely on CMOS technology.
Digital Implementation of the Two-Compartmental Pinsky-Rinzel Pyramidal Neuron Model
E. Rahimian, S. Zabihi, M. Amiri and B. Linares-Barranco
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 2, no. 1, pp 47-57, 2018
abstract
doi pdf
It is believed that brain-like computing system can be achieved by the fusion of electronics and neuroscience. In this way, the optimized digital hardware implementation of neurons, primary units of nervous system, play a vital role in neuromorphic applications. Moreover, one of the main features of pyramidal neurons in cortical areas is bursting activities that has a critical role in synaptic plasticity. The Pinsky-Rinzel model is a nonlinear two-compartmental model for CA3 pyramidal cell that is widely used in neuroscience. In this paper, a modified Pinsky-Rinzel pyramidal model is proposed by replacing its complex nonlinear equations with piecewise linear approximation. Next, a digital circuit is designed for the simplified model to be able to implement on a low-cost digital hardware, such as field-programmable gate array (FPGA). Both original and proposed models are simulated in MATLAB and next digital circuit simulated in Vivado is compared to show that obtained results are in good agreement. Finally, the results of physical implementation on FPGA are also illustrated. The presented circuit advances preceding designs with regards to the ability to replicate essential characteristics of different firing responses including bursting and spiking in the compartmental model. This new circuit has various applications in neuromorphic engineering, such as developing new neuroinspired chips.
Event-Driven Stereo Visual Tracking Algorithm to Solve Object Occlusion
L.A. Camunas-Mesa, T. Serrano-Gotarredona, S. Ieng, R. Benosman and B. Linares-Barranco
Journal Paper · IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 9, pp 4223-4237, 2017
abstract
doi
Object tracking is a major problem for many computer vision applications, but it continues to be computationally expensive. The use of bio-inspired neuromorphic event-driven dynamic vision sensors (DVSs) has heralded new methods for vision processing, exploiting reduced amount of data and very precise timing resolutions. Previous studies have shown these neural spiking sensors to be well suited to implementing single-sensor object tracking systems, although they experience difficulties when solving ambiguities caused by object occlusion. DVSs have also performed well in 3-D reconstruction in which event matching techniques are applied in stereo setups. In this paper, we propose a new event-driven stereo object tracking algorithm that simultaneously integrates 3-D reconstruction and cluster tracking, introducing feedback information in both tasks to improve their respective performances. This algorithm, inspired by human vision, identifies objects and learns their position and size in order to solve ambiguities. This strategy has been validated in four different experiments where the 3-D positions of two objects were tracked in a stereo setup even when occlusion occurred. The objects studied in the experiments were: 1) two swinging pens, the distance between which during movement was measured with an error of less than 0.5%; 2) a pen and a box, to confirm the correctness of the results obtained with a more complex object; 3) two straws attached to a fan and rotating at 6 revolutions per second, to demonstrate the high-speed capabilities of this approach; and 4) two people walking in a real-world environment.
On Multiple AER Handshaking Channels over High-Speed Bit-Serial Bidirectional LVDS Links with Flow-Control and Clock-Correction on Commercial FPGAs for Scalable Neuromorphic Systems
A. Yousefzadeh, M. Jablonski, T. Iakymchuk, A. Linares-Barranco, A. Rosado, L.A. Plana, S. Temple, T. Serrano-Gotarredona, S.B. Furber and B. Linares-Barranco
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol 11, no. 5, pp 1133-1147, 2017
abstract
doi pdf
Address event representation (AER) is a widely employed asynchronous technique for interchanging "neural spikes" between different hardware elements in neuromorphic systems. Each neuron or cell in a chip or a system is assigned an address (or ID), which is typically communicated through a high-speed digital bus, thus time-multiplexing a high number of neural connections. Conventional AER links use parallel physical wires together with a pair of handshaking signals (request and acknowledge). In this paper, we present a fully serial implementation using bidirectional SATA connectors with a pair of low-voltage differential signaling (LVDS) wires for each direction. The proposed implementation can multiplex a number of conventional parallel AER links for each physical LVDS connection. It uses flow control, clock correction, and byte alignment techniques to transmit 32-bit address events reliably over multiplexed serial connections. The setup has been tested using commercial Spartan6 FPGAs attaining a maximum event transmission speed of 75 Meps (Mega events per second) for 32-bit events at a line rate of 3.0 Gbps. Full HDL codes (vhdl/verilog) and example demonstration codes for the SpiNNaker platform will be made available.
An Event-Driven Classifier for Spiking Neural Networks Fed with Synthetic or Dynamic Vision Sensor Data
E. Stromatias, M. Soto, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 11, article 350, 2017
abstract
doi pdf
This paper introduces a novel methodology for training an event-driven classifier within a Spiking Neural Network (SNN) System capable of yielding good classification results when using both synthetic input data and real data captured from Dynamic Vision Sensor (DVS) chips. The proposed supervised method uses the spiking activity provided by an arbitrary topology of prior SNN layers to build histograms and train the classifier in the frame domain using the stochastic gradient descent algorithm. In addition, this approach can cope with leaky integrate-and-fire neuron models within the SNN, a desirable feature for real-world SNN applications, where neural activation must fade away after some time in the absence of inputs. Consequently, this way of building histograms captures the dynamics of spikes immediately before the classifier. We tested our method on the MNIST data set using different synthetic encodings and real DVS sensory data sets such as N-MNIST, MNIST-DVS, and Poker-DVS using the same network topology and feature maps. We demonstrate the effectiveness of our approach by achieving the highest classification accuracy reported on the N-MNIST (97.77%) and Poker-DVS (100%) real DVS data sets to date with a spiking convolutional network. Moreover, by using the proposed method we were able to retrain the output layer of a previously reported spiking neural network and increase its performance by 2%, suggesting that the proposed classifier can be used as the output layer in works where features are extracted using unsupervised spike-based learning methods. In addition, we also analyze SNN performance figures such as total event activity and network latencies, which are relevant for eventual hardware implementations. In summary, the paper aggregates unsupervised-trained SNNs with a supervised-trained SNN classifier, combining and applying them to heterogeneous sets of benchmarks, both synthetic and from real DVS chips.
A Hybrid CMOS-Memristor Neuromorphic Synapse
M.R. Azghadi, B. Linares-Barranco, D. Abbott and P.H.W. Leong
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 11, no. 2, pp 434-445, 2017
abstract
doi pdf
Although data processing technology continues to advance at an astonishing rate, computers with brain-like processing capabilities still elude us. It is envisioned that such computers may be achieved by the fusion of neuroscience and nano-electronics to realize a brain-inspired platform. This paper proposes a high-performance nano-scale Complementary Metal Oxide Semiconductor (CMOS)-memristive circuit, which mimics a number of essential learning properties of biological synapses. The proposed synaptic circuit that is composed of memristors and CMOS transistors, alters its memristance in response to timing differences among its pre- and post-synaptic action potentials, giving rise to a family of Spike Timing Dependent Plasticity (STDP). The presented design advances preceding memristive synapse designs with regards to the ability to replicate essential behaviours characterised in a number of electrophysiological experiments performed in the animal brain, which involve higher order spike interactions. Furthermore, the proposed hybrid device CMOS area is estimated as 600 μm2 in a 0.35 μm process -this represents a factor of ten reduction in area with respect to prior CMOS art. The new design is integrated with silicon neurons in a crossbar array structure amenable to large-scale neuromorphic architectures and may pave the way for future neuromorphic systems with spike timing-dependent learning features. These systems are emerging for deployment in various applications ranging from basic neuroscience research, to pattern recognition, to Brain-Machine-Interfaces.
An address event representation-based processing system for a biped robot
U. Jaramillo-Avila, H. Rostro-Gonzalez, L.A. Camuñas-Mesa, R.J. Romero-Troncoso and B. Linares-Barranco
Journal Paper · International Journal of Advanced Robotic Systems, vol. 13, no. 1, 2016
abstract
doi pdf
In recent years, several important advances have been made in the fields of both biologically inspired sensorial processing and locomotion systems, such as Address Event Representation-based cameras (or Dynamic Vision Sensors) and in human-like robot locomotion, e.g,. the walking of a biped robot. However, making these fields merge properly is not an easy task. In this regard, Neuromorphic Engineering is a fast-growing research field, the main goal of which is the biologically inspired design of hybrid hardware systems in order to mimic neural architectures and to process information in the manner of the brain. However, few robotic applications exist to illustrate them. The main goal of this work is to demonstrate, by creating a closed-loop system using only bio-inspired techniques, how such applications can work properly. We present an algorithm using Spiking Neural Networks (SNN) for a biped robot equipped with a Dynamic Vision Sensor, which is designed to follow a line drawn on the floor. This is a commonly used method for demonstrating control techniques. Most of them are fairly simple to implement without very sophisticated components; however, it can still serve as a good test in more elaborate circumstances. In addition, the locomotion system proposed is able to coordinately control the six DOFs of a biped robot in switching between basic forms of movement. The latter has been implemented as a FPGA-based neuromorphic system. Numerical tests and hardware validation are presented.
Fast Predictive Handshaking in Synchronous FPGAs for Fully Asynchronous Multisymbol Chip Links: Application to SpiNNaker 2-of-7 Links
A. Yousefzadeh, L.A. Plana, S. Temple, T. Serrano-Gotarredona, S.B. Furber and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 63, no. 8, pp 763-767, 2016
abstract
doi pdf
Asynchronous handshaken interchip links are very popular among neuromorphic full-custom chips due to their delay-insensitive and high-speed properties. Of special interest are those links that minimize bit-line transitions for power saving, such as the two-phase handshaken non-return-to-zero (NRZ) 2-of-7 protocol used in the SpiNNaker chips. Interfacing such custom chip links to field-programmable gate arrays (FPGAs) is always of great interest, so that additional functionalities can be experimented and exploited for producing more versatile systems. Present-day commercial FPGAs operate typically in synchronous mode, thus making it necessary to incorporate synchronizers when interfacing with asynchronous chips. This introduces extra latencies and precludes pipelining, deteriorating transmission speed, particularly when sending multisymbols per unit communication packet. In this brief, we present a technique that learns to estimate the delay of a symbol transaction, thus allowing a fast pipelining from symbol to symbol. The technique has been tested on links between FPGAs and SpiNNaker chips, achieving the same throughput as fully asynchronous synchronizerless links between SpiNNaker chips. The links have been tested for periods of over one week without any transaction failure. Verilog codes of FPGA circuits are available as additional material for download.
Generalized reconfigurable memristive dynamical system (MDS) for neuromorphic applications
M. Bavandpour, H. Soleimani, B. Linares-Barranco, D. Abbott and L.O. Chua
Journal Paper · Frontiers in Neuroscience, vol. 9, article 409, 2015
abstract
doi
This study firstly presents (i) a novel general cellular mapping scheme for two dimensional neuromorphic dynamical systems such as bio-inspired neuron models, and (ii) an efficient mixed analog-digital circuit, which can be conveniently implemented on a hybrid memristor-crossbar/CMOS platform, for hardware implementation of the scheme. This approach employs 4n memristors and no switch for implementing an n-cell system in comparison with 2(n2) memristors and 2n switches of a Cellular Memristive Dynamical System (CMDS). Moreover, this approach allows for dynamical variables with both analog and one-hot digital values opening a wide range of choices for interconnections and networking schemes. Dynamical response analyses show that this circuit exhibits various responses based on the underlying bifurcation scenarios which determine the main characteristics of the neuromorphic dynamical systems. Due to high programmability of the circuit, it can be applied to a variety of learning systems, real-time applications, and analytically indescribable dynamical systems. We simulate the FitzHugh-Nagumo (FHN), Adaptive Exponential (AdEx) integrate and fire, and Izhikevich neuron models on our platform, and investigate the dynamical behaviors of these circuits as case studies. Moreover, error analysis shows that our approach is suitably accurate. We also develop a simple hardware prototype for experimental demonstration of our approach.
Poker-DVS and MNIST-DVS. Their history, how they were made, and other details
T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 9, article 481, 2015
abstract
doi pdf
This article reports on two databases for event-driven object recognition using a Dynamic Vision Sensor (DVS). The first, which we call Poker-DVS and is being released together with this article, was obtained by browsing specially made poker card decks in front of a DVS camera for 2-4 s. Each card appeared on the screen for about 20-30 ms. The poker pips were tracked and isolated off-line to constitute the 131-recording Poker-DVS database. The second database, which we call MNIST-DVS and which was released in December 2013, consists of a set of 30,000 DVS camera recordings obtained by displaying 10,000 moving symbols from the standard MNIST 70,000-picture database on an LCD monitor for about 2-3 s each. Each of the 10,000 symbols was displayed at three different scales, so that event-driven object recognition algorithms could easily be tested for different object sizes. This article tells the story behind both databases, covering, among other aspects, details of how they work and the reasons for their creation. We provide not only the databases with corresponding scripts, but also the scripts and data used to generate the figures shown in this article (as Supplementary Material).
Modeling and experimental demonstration of a hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits
X. Guo, F. Merrikh-Bayat, L. Gao, B.D. Hoskins, F. Alibart, B. Linares-Barranco, L. Theogarajan, C. Teuscher and D.B. Strukov
Journal Paper · Frontiers in Neuroscience, vol. 9, article 488, 2015
abstract
doi pdf
The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2-x/Pt memristors and CMOS integrated circuit components.
Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network
B. Zhao, R. Ding, S. Chen, B. Linares-Barranco and H. Tang
Journal Paper · IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 9, pp 1963-1978, 2015
abstract
doi
This paper introduces an event-driven feedforward categorization system, which takes data from a temporal contrast address event representation (AER) sensor. The proposed system extracts bio-inspired cortex-like features and discriminates different patterns using an AER based tempotron classifier (a network of leaky integrate-and-fire spiking neurons). One of the system's most appealing characteristics is its event-driven processing, with both input and features taking the form of address events (spikes). The system was evaluated on an AER posture dataset and compared with two recently developed bio-inspired models. Experimental results have shown that it consumes much less simulation time while still maintaining comparable performance. In addition, experiments on the Mixed National Institute of Standards and Technology (MNIST) image dataset have demonstrated that the proposed system can work not only on raw AER data but also on images (with a preprocessing step to convert images into AER events) and that it can maintain competitive accuracy even when noise is added. The system was further evaluated on the MNIST dynamic vision sensor dataset (in which data is recorded using an AER dynamic vision sensor), with testing accuracy of 88.14%.
Plasticity in memristive devices for spiking neural networks
S. Saïghi, C.G. Mayr, T. Serrano-Gotarredona, H. Schmidt, G. Lecerf, J. Tomas, J. Grollier, S. Boyn, A.F. Vincent, D. Querlioz, S. La Barbera, F. Alibart, D. Vuillaume, O. Bichler, C. Gamrat and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 9, article 51, 2015
abstract
doi pdf
Memristive devices present a new device technology allowing for the realization of compact non-volatile memories. Some of them are already in the process of industrialization. Additionally, they exhibit complex multilevel and plastic behaviors, which make them good candidates for the implementation of artificial synapses in neuromorphic engineering. However, memristive effects rely on diverse physical mechanisms, and their plastic behaviors differ strongly from one technology to another. Here, we present measurements performed on different memristive devices and the opportunities that they provide. We show that they can be used to implement different learning rules whose properties emerge directly from device physics: real time or accelerated operation, deterministic or stochastic behavior, long term or short term plasticity. We then discuss how such devices might be integrated into a complete architecture. These results highlight that there is no unique way to exploit memristive devices in neuromorphic systems. Understanding and embracing device physics is the key for their optimal use.
Spiking neuro-fuzzy clustering system and its memristor crossbar based implementation
M. Bavandpour, S. Bagheri-Shouraki, H. Soleimani, A. Ahmadi and B. Linares-Barranco
Journal Paper · Microelectronics Journal, vol. 45, no. 11, pp 1450-1462, 2014
abstract
doi
This study proposes a spiking neuro-fuzzy clustering system based on a novel spike encoding scheme and a compatible learning algorithm. In this system, we utilize an analog to binary encoding scheme that properly maps the concept of 'distance' in multi-dimensional analog spaces to the concept of 'dissimilarity' of binary bits in the equivalent binary spaces. When this scheme is combined with a novel binary to spike encoding scheme and a proper learning algorithm is applied, a powerful clustering algorithm is produced. This algorithm creates flexible fuzzy clusters in its analog input space and modifies their shapes to different convex shapes during the learning process. This system has plausible biological support due to its spike-based learning mechanism, its Quasi Spike Time Dependent Plasticity learning policy and its brain-like fuzzy clustering performance. Moreover, this neuro-fuzzy system is fully implementable on the hybrid memristor-crossbar/CMOS platform. The resultant circuit was simulated on one clustering task carried out in the binary input space on the Simon Lucas handwritten dataset and another clustering task carried out in the analog input space on Fisher's Iris standard dataset. The results show that it attained a higher clustering rate in comparison with other algorithms such as the Self Organizing Map, K-mean and the Spiking Radial Basis Function. The circuit was also successfully simulated on an image segmentation task and some clustering tasks performed in noisy spaces with various cluster sizes. Furthermore, the circuit variability analysis shows that device and signal variations up to 20% had no significant impact on the circuit's clustering performance, so the system is sufficiently immune to different variations due to its fuzzy nature.
Retinomorphic event-based vision sensors: Bioinspired cameras with spiking output
C. Posch, T. Serrano-Gotarredona, B. Linares-Barranco and T. Delbruck
Journal Paper · Proceedings of the IEEE, vol. 102, no. 10, pp 1470-1484, 2014
abstract
doi
State-of-the-art image sensors suffer from significant limitations imposed by their very principle of operation. These sensors acquire the visual information as a series of 'snapshot' images, recorded at discrete points in time. Visual information gets time quantized at a predetermined frame rate which has no relation to the dynamics present in the scene. Furthermore, each recorded frame conveys the information from all pixels, regardless of whether this information, or a part of it, has changed since the last frame had been acquired. This acquisition method limits the temporal resolution, potentially missing important information, and leads to redundancy in the recorded image data, unnecessarily inflating data rate and volume. Biology is leading the way to a more efficient style of image acquisition. Biological vision systems are driven by events happening within the scene in view, and not, like image sensors, by artificially created timing and control signals. Translating the frameless paradigm of biological vision to artificial imaging systems implies that control over the acquisition of visual information is no longer being imposed externally to an array of pixels but the decision making is transferred to the single pixel that handles its own information individually. In this paper, recent developments in bioinspired, neuromorphic optical sensing and artificial vision are presented and discussed. It is suggested that bioinspired vision systems have the potential to outperform conventional, frame-based vision systems in many application fields and to establish new benchmarks in terms of redundancy suppression and data compression, dynamic range, temporal resolution, and power efficiency. Demanding vision tasks such as real-time 3-D mapping, complex multiobject tracking, or fast visual feedback loops for sensory-motor action, tasks that often pose severe, sometimes insurmountable, challenges to conventional artificial vision systems, are in reach using bioinspired vision sensing and processing techniques.
On the use of orientation filters for 3D reconstruction in event-driven stereo vision
L.A. Camuñas-Mesa, T. Serrano-Gotarredona, S.H. Ieng, R.B. Benosman and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 8, article 48, 2014
abstract
doi pdf
The recently developed Dynamic Vision Sensors (DVS) sense visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, therefore increasing the number of constraints applied to the matching algorithm. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.
A 1.5 ns off/on switching-time voltage-mode lvds driver/receiver pair for asynchronous aer bit-serial chip grid links with up to 40 times event-rate dependent power savings
C. Zamarreno-Ramos, R. Kulkarni, J. Silva-Martinez, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 7, no. 5, pp 722-731, 2013
abstract
doi pdf
This paper presents a low power fast ON/OFF switchable voltage mode implementation of a driver/receiver pair intended to be used in high speed bit-serial Low Voltage Differential Signaling (LVDS) Address Event Representation (AER) chip grids, where short (like 32-bit) sparse data packages are transmitted. Voltage-Mode drivers require intrinsically half the power of their Current-Mode counterparts and do not require Common-Mode Voltage Control. However, for fast ON/OFF switching a special high-speed voltage regulator is required which needs to be kept ON during data pauses, and hence its power consumption must be minimized, resulting in tight design constraints. A proof-of-concept chip test prototype has been designed and fabricated in low-cost standard 0.35 μ m CMOS. At 500 mV voltage swing with 500 Mbps serial bit rate and 32 bit events, current consumption scales from 15.9 mA (7.7 mA for the driver and 8.2 mA for the receiver) at 10 Mevent/s rate to 406 μ A ( 343 μ A for the driver and 62.5 μA for the receiver) for an event rate below 10 Kevent/s, therefore achieving a rate dependent power saving of up to 40 times, while keeping switching times at 1.5 ns. Maximum achievable event rate was 13.7 Meps at 638 Mbps serial bit rate. Additionally, differential voltage swing is tunable, thus allowing further power reductions.
Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets
J.A. Pérez-Carrasco, B. Zhao, C. Serrano, B. Acha, T. Serrano-Gotarredona, S. Chen and B. Linares-Barranco
Journal Paper · IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 35, no. 11, pp 2706-2719, 2013
abstract
doi pdf
Event-driven visual sensors have attracted interest from a number of different research communities. They provide visual information in quite a different way from conventional video systems consisting of sequences of still images rendered at a given 'frame rate'. Event-driven vision sensors take inspiration from biology. Each pixel sends out an event (spike) when it senses something meaningful is happening, without any notion of a frame. A special type of event-driven sensor is the so-called dynamic vision sensor (DVS) where each pixel computes relative changes of light or 'temporal contrast.' The sensor output consists of a continuous flow of pixel events that represent the moving objects in the scene. Pixel events become available with microsecond delays with respect to 'reality.' These events can be processed 'as they flow' by a cascade of event (convolution) processors. As a result, input and output event flows are practically coincident in time, and objects can be recognized as soon as the sensor provides enough meaningful events. In this paper, we present a methodology for mapping from a properly trained neural network in a conventional frame-driven representation to an event-driven representation. The method is illustrated by studying event-driven convolutional neural networks (ConvNet) trained to recognize rotating human silhouettes or high speed poker card symbols. The event-driven ConvNet is fed with recordings obtained from a real DVS camera. The event-driven ConvNet is simulated with a dedicated event-driven simulator and consists of a number of event-driven processing modules, the characteristics of which are obtained from individually manufactured hardware modules.
Integration of nanoscale memristor synapses in neuromorphic computing architectures
G. Indiveri, B. Linares-Barranco, R. Legenstein, G. Deligeorgisand and T. Prodromakis
Journal Paper · Nanotechnology, vol. 24, no. 38, article number 384010, 2013
abstract
doi pdf
Conventional neuro-computing architectures and artificial neural networks have often been developed with no or loose connections to neuroscience. As a consequence, they have largely ignored key features of biological neural processing systems, such as their extremely low-power consumption features or their ability to carry out robust and efficient computation using massively parallel arrays of limited precision, highly variable, and unreliable components. Recent developments in nano-technologies are making available extremely compact and low power, but also variable and unreliable solid-state devices that can potentially extend the offerings of availing CMOS technologies. In particular, memristors are regarded as a promising solution for modeling key features of biological synapses due to their nanoscale dimensions, their capacity to store multiple bits of information per element and the low energy required to write distinct states. In this paper, we first review the neuro- and neuromorphic computing approaches that can best exploit the properties of memristor and scale devices, and then propose a novel hybrid memristor-CMOS neuromorphic circuit which represents a radical departure from conventional neuro-computing approaches, as it uses memristors to directly emulate the biophysics and temporal dynamics of real synapses. We point out the differences between the use of memristors in conventional neuro-computing architectures and the hybrid memristor-CMOS circuit proposed, and argue how this circuit represents an ideal building block for implementing brain-inspired probabilistic computing paradigms that are robust to variability and fault tolerant by design.
STDP and sTDP variations with memristors for spiking neuromorphic learning systems
T. Serrano-Gotarredona, T. Masquelier, T. Prodromakis, G. Indiveri and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 7, article 2, 2013
abstract
doi pdf
In this paper we review several ways of realizing asynchronous Spike-Timing-Dependent-Plasticity (STDP) using memristors as synapses. Our focus is on how to use individual memristors to implement synaptic weight multiplications, in a way such that it is not necessary to (a) introduce global synchronization and (b) to separate memristor learning phases from memristor performing phases. In the approaches described, neurons fire spikes asynchronously when they wish and memristive synapses perform computation and learn at their own pace, as it happens in biological neural systems. We distinguish between two different memristor physics, depending on whether they respond to the original 'moving wall' or to the 'filament creation and annihilation' models. Independent of the memristor physics, we discuss two different types of STDP rules that can be implemented with memristors: either the pure timing-based rule that takes into account the arrival time of the spikes from the pre- and the post-synaptic neurons, or a hybrid rule that takes into account only the timing of pre-synaptic spikes and the membrane potential and other state variables of the post-synaptic neuron. We show how to implement these rules in cross-bar architectures that comprise massive arrays of memristors, and we discuss applications for artificial vision.
A Proposal for Hybrid Memristor-CMOS Spiking Neuromorphic Learning Systems
T. Serrano-Gotarredona, T. Prodromakis and B. Linares Barranco
Journal Paper · IEEE Circuits and Systems Magazine, vol. 13, no. 2, pp 74-88, 2013
abstract
doi
Recent research in nanotechnology has led to the practical realization of nanoscale devices that behave as memristors, a device that was postulated in the seventies by Chua based on circuit theoretical reasonings. On the other hand, neuromorphic engineering, a discipline that implements physical artifacts based on neuroscience knowledge, has related neural learning mechanisms to the operation of memristors. As a result, neuro-inspired learning architectures can be proposed that exploit nanoscale memristors for building very large scale systems with very dense synaptic-like memory elements. At present, the deep understanding of the internal mechanisms governing memristor operation is still an open issue, and the practical realization of very large scale and reliable memristive fabric for neural learning applications is not a reality yet. However, in the meantime, researchers are proposing and analyzing potential circuit architectures that would combine a standard CMOS substrate with a memristive nanoscale fabric on top to realize hybrid memristor-CMOS neural learning systems. The focus of this paper is on one such architecture for implementing the very well established Spike-Timing-Dependent-Plasticity (STDP) learning mechanism found in biology. In this paper we quickly review spiking neural systems, STDP learning, and memristors, and propose a hybrid memristor-CMOS system architecture with the potential of implementing a large scale STDP learning spiking neural system. Such architecture would eventually allow to implement real-time brain-like processing learning systems with about neurons and synapses on one single Printed Circuit Board (PCB).
A 128x128 1.5% Contrast Sensitivity 0.9% FPN 3μs Latency 4mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers
T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Journal of Solid-State Circuits, vol. 48, no. 3, pp 827-838, 2013
abstract
doi pdf
Dynamic Vision Sensors (DVS) have recently appeared as a new paradigm for vision sensing and processing. They feature unique characteristics such as contrast coding under wide illumination variation, micro-second latency response to fast stimuli, and low output data rates (which greatly improves the efficiency of post-processing stages). They can track extremely fast objects (e.g., time resolution is better than 100 kFrames/s video) without special lighting conditions. Their availability has triggered a new range of vision applications in the fields of surveillance, motion analyses, robotics, and microscopic dynamic observations. One key DVS feature is contrast sensitivity, which has so far been reported to be in the 10-15% range. In this paper, a novel pixel photo sensing and transimpedance pre-amplification stage makes it possible to improve by one order of magnitude contrast sensitivity (down to 1.5%) and power (down to 4mW), reduce the best reported FPN (Fixed Pattern Noise) by a factor of 2 (down to 0.9%), while maintaining the shortest reported latency (3μs) and good Dynamic Range (120 dB), and further reducing overall area (down to 30×31μm per pixel). The only penalty is the limitation of intrascene Dynamic Range to 3 decades. A 128×128 DVS test prototype has been fabricated in standard 0.35μm CMOS and extensive experimental characterization results are provided.
Multicasting Mesh AER: A Scalable Assembly Approach for Reconfigurable Neuromorphic Structured AER Systems. Application to ConvNets
C. Zamarreño-Ramos, A. Linares-Barranco, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 7, no. 1, pp 82-102, 2013
abstract
doi pdf
This paper presents a modular, scalable approach to assembling hierarchically structured neuromorphic Address Event Representation (AER) systems. The method consists of arranging modules in a 2D mesh, each communicating bidirectionally with all four neighbors. Address events include a module label. Each module includes an AER router which decides how to route address events. Two routing approaches have been proposed, analyzed and tested, using either destination or source module labels. Our analyses reveal that depending on traffic conditions and network topologies either one or the other approach may result in better performance. Experimental results are given after testing the approach using high-end Virtex-6 FPGAs. The approach is proposed for both single and multiple FPGAs, in which case a special bidirectional parallel-serial AER link with flow control is exploited, using the FPGA Rocket-I/O interfaces. Extensive test results are provided exploiting convolution modules of 64 $,times,$64 pixels with kernels with sizes up to 11$,times,$ 11, which process real sensory data from a Dynamic Vision Sensor (DVS) retina. One single Virtex-6 FPGA can hold up to 64 of these convolution modules, which is equivalent to a neural network with $262 times 10^{3}~{rm neurons}$ and almost 32 million synapses.
A 0.35μm Sub-ns Wake-up Time ON-OFF Switchable LVDS Driver-Receiver Chip I/O Pad Pair for Rate-Dependent Power Saving in AER Bit-Serial Links
C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Biomedical Circuits and Systems, vol. 6, no. 5, pp 486-497, 2012
abstract
doi pdf
This paper presents a low power switchable current mode driver/receiver I/O pair for high speed serial transmission of asynchronous address event representation (AER) information. The sparse nature of AER packets (also called events) allows driver/receiver bias currents to be switched off to save power. The on/off times must be lower than the bit time to minimize the latency introduced by the switching mechanism. Using this technique, the link power consumption can be scaled down with the event rate without compromising the maximum system throughput. The proposed technique has been implemented on a typical push/pull low voltage differential signaling (LVDS) circuit, but it can easily be extended to other widely used current mode standards, such as current mode logic (CML) or low-voltage positive emitter-coupled logic (LVPECL). A proof of concept prototype has been fabricated in $0.35~mu{rm m}$ CMOS incorporating the proposed driver/receiver pair along with a previously reported switchable serializer/deserializer scheme. At a 500 Mbps bit rate, the maximum event rate is 11 Mevent/s for 32-bit events. In this situation, current consumption is 7.5 mA and 9.6 mA for the driver and receiver, respectively, while differential voltage amplitude is $pm 300~{rm mV}$. However, if event rate is lower than 20–30 Kevent/s, current consumption has a floor of $270~mu{rm A}$ for the driver and $570~mu{rm A}$ for the receiver. The measured ON/OFF switching times are in the order of 1 ns. The serial link could be operated at up to 710 Mbps bit rate, resulting in a maximum 32-bit event rate of 15 Mevent/s . This is the same peak event rate as that obtained with the same SerDes circuits and a non-switched drive- /receiver pair.
Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing
C. Farabet, R. Paz, J. Pérez-Carrasco, C. Zamarreño-Ramos, A. Linares-Barranco, Y. LeCun, E. Culurciello, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 6, article 32, 2012
abstract
doi pdf
Most scene segmentation and categorization architectures for the extraction of features in images and patches make exhaustive use of 2D convolution operations for template matching, template search, and denoising. Convolutional Neural Networks (ConvNets) are one example of such architectures that can implement general-purpose bio-inspired vision systems. In standard digital computers 2D convolutions are usually expensive in terms of resource consumption and impose severe limitations for efficient real-time applications. Nevertheless, neuro-cortex inspired solutions, like dedicated Frame-Based or Frame-Free Spiking ConvNet Convolution Processors, are advancing real-time visual processing. These two approaches share the neural inspiration, but each of them solves the problem in different ways. Frame-Based ConvNets process frame by frame video information in a very robust and fast way that requires to use and share the available hardware resources (such as: multipliers, adders). Hardware resources are fixed- and time-multiplexed by fetching data in and out. Thus memory bandwidth and size is important for good performance. On the other hand, spike-based convolution processors are a frame-free alternative that is able to perform convolution of a spike-based source of visual information with very low latency, which makes ideal for very high-speed applications. However, hardware resources need to be available all the time and cannot be time-multiplexed. Thus, hardware should be modular, reconfigurable, and expansible. Hardware implementations in both VLSI custom integrated circuits (digital and analog) and FPGA have been already used to demonstrate the performance of these systems. In this paper we present a comparison study of these two neuro-inspired solutions. A brief description of both systems is presented and also discussions about their differences, pros and cons.
A memristive nanoparticle/organic hybrid synapstor for neuroinspired computing
F. Alibart, S. Pleutin, O. Bichler, C. Gamrat, T. Serrano-Gotarredona, B. Linares-Barranco and D. Vuillaume
Journal Paper · Advanced Functional Materials, vol. 22, no. 3, pp 609-616, 2012
abstract
doi pdf
A large effort is devoted to the research of new computing paradigms associated with innovative nanotechnologies that should complement and/or propose alternative solutions to the classical Von Neumann/CMOS (complementary metal oxide semiconductor) association. Among various propositions, spiking neural network (SNN) seems a valid candidate. i) In terms of functions, SNN using relative spike timing for information coding are deemed to be the most effective at taking inspiration from the brain to allow fast and efficient processing of information for complex tasks in recognition or classification. ii) In terms of technology, SNN may be able to benefit the most from nanodevices because SNN architectures are intrinsically tolerant to defective devices and performance variability. Here, spike-timing-dependent plasticity (STDP), a basic and primordial learning function in the brain, is demonstrated with a new class of synapstor (synapse-transistor), called nanoparticle organic memory field-effect transistor (NOMFET). This learning function is obtained with a simple hybrid material made of the self-assembly of gold nanoparticles and organic semiconductor thin films. Beyond mimicking biological synapses, it is also demonstrated how the shape of the applied spikes can tailor the STDP learning function. Moreover, the experiments and modeling show that this synapstor is a memristive device. Finally, these synapstors are successfully coupled with a CMOS platform emulating the pre- and postsynaptic neurons, and a behavioral macromodel is developed on usual device simulator.
An event-driven multi-kernel convolution processor module for event-driven vision sensors
L. Camuñas-Mesa, C. Zamarreño-Ramos, A. Linares-Barranco, A.J. Acosta-Jiménez, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Journal of Solid-State Circuits, vol. 47, no. 2, pp 504-517, 2012
abstract
doi pdf
Event-Driven vision sensing is a new way of sensing visual reality in a frame-free manner. This is, the vision sensor (camera) is not capturing a sequence of still frames, as in conventional video and computer vision systems. In Event-Driven sensors each pixel autonomously and asynchronously decides when to send its address out. This way, the sensor output is a continuous stream of address events representing reality dynamically continuously and without constraining to frames. In this paper we present an Event-Driven Convolution Module for computing 2D convolutions on such event streams. The Convolution Module has been designed to assemble many of them for building modular and hierarchical Convolutional Neural Networks for robust shape and pose invariant object recognition. The Convolution Module has multi-kernel capability. This is, it will select the convolution kernel depending on the origin of the event. A proof-of-concept test prototype has been fabricated in a 0.35 mu m CMOS process and extensive experimental results are provided. The Convolution Processor has also been combined with an Event-Driven Dynamic Vision Sensor (DVS) for high-speed recognition examples. The chip can discriminate propellers rotating at 2 k revolutions per second, detect symbols on a 52 card deck when browsing all cards in 410 ms, or detect and follow the center of a phosphor oscilloscope trace rotating at 5 KHz.
Efficient feedforward categorization of objects and human postures with address-event image sensors
S.S. Chen, P. Akselrod, B. Zhao, J.A. Pérez-Carrasco, B. Linares-Barranco and E. Culurciello
Journal Paper · IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 2, pp 302-314, 2012
abstract
doi pdf
This paper proposes an algorithm for feedforward categorization of objects and, in particular, human postures in real-time video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event-based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified line segment Hausdorff-distance classifier combined with on-the-fly cluster-based size and position invariant categorization. The system can achieve about 90 percent average success rate in the categorization of human postures, while using only a small number of training samples. Compared to state-of-the-art bio-inspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least five times, and is an ideal candidate for hardware implementation with event-based circuits.
Neuromorphic silicon neuron circuits
G. Indiveri, B. Linares-Barranco, T.J. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, Shih-Chii Liu, P. Dudek, P. Häfliger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna11, F. Folowosele, S. Saighi, T. Serrano-Gotarredona, J. Wijekoon, Y. Wang1 and K. Boahen
Journal Paper · Frontiers in Neuroscience, vol. 5,article 73, 2011
abstract
doi pdf
Hardware implementations of spiking neurons can be extremely useful for a large variety of applications, ranging from high-speed modeling of large-scale neural systems to real-time behaving systems, to bidirectional brain-machine interfaces. The specific circuit solutions used to implement silicon neurons depend on the application requirements. In this paper we describe the most common building blocks and techniques used to implement these circuits, and present an overview of a wide range of neuromorphic silicon neurons, which implement different computational models, ranging from biophysically realistic and conductance-based Hodgkin-Huxley models to bi-dimensional generalized adaptive integrate and fire models. We compare the different design methodologies used for each silicon neuron design described, and demonstrate their features with experimental results, measured from a wide range of fabricated VLSI chips.
An instant-startup jitter-tolerant manchester-encoding serializer/deserializer scheme for event-driven bit-serial LVDS interchip AER links
C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems I, Regular Papers, vol. 58, no. 11, pp 2647-2660, 2011
abstract
doi pdf
This paper presents a serializer/deserializer scheme for asynchronous address event representation (AER) bit-serial interchip communications. Each serial AER (sAER) link uses four wires: a micro strip pair for low voltage differential signaling (LVDS) and two handshaking lines. Each event is represented by a 32-bit word. Two extra preamble bits are used for alignment. Transmission clock is embedded in the data using Manchester encoding. As opposed to conventional LVDS links, the presented approach allows to stop physical communication between data events, so that no "comma" characters need to be transmitted during these pauses. As soon as a new event needs to be transmitted, the link recovers immediately thanks to a built-in control voltage memorization circuit. As a result, power consumption of the serializer and deserializer circuits is proportional to data event rate. The approach is also highly tolerant to clock jitter, due to the asynchronous nature and the Manchester encoding. A chip test prototype has been fabricated in standard 0.35 mu m CMOS including a pair of Serializer and Deserializer circuits. Maximum measured event transmission rate is 15 Meps (mega events per second) for 32-bit events, with a maximum bit transmission speed of 670 Mbps (mega bits per second).
On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex
C. Zamarreño-Ramos, L.A. Camuñas-Mesa, J.A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Frontiers in Neuroscience, vol. 5, article 26, 2011
abstract
doi pdf
In this paper we present a very exciting overlap between emergent nanotechnology and neuroscience, which has been discovered by neuromorphic engineers. Specifically, we are linking one type of memristor nanotechnology devices to the biological synaptic update rule known as spike-time-dependent-plasticity (STDP) found in real biological synapses. Understanding this link allows neuromorphic engineers to develop circuit architectures that use this type of memristors to artificially emulate parts of the visual cortex. We focus on the type of memristors referred to as voltage or flux driven memristors and focus our discussions on a behavioral macro-model for such devices. The implementations result in fully asynchronous architectures with neurons sending their action potentials not only forward but also backward. One critical aspect is to use neurons that generate spikes of specific shapes. We will see how by changing the shapes of the neuron action potential spikes we can tune and manipulate the STDP learning rules for both excitatory and inhibitory synapses. We will see how neurons and memristors can be interconnected to achieve large scale spiking learning systems, that follow a type of multiplicative STDP learning rule. We will briefly extend the architectures to use three-terminal transistors with similar memristive behavior. We will illustrate how a V1 visual cortex layer can assembled and how it is capable of learning to extract orientations from visual data coming from a real artificial CMOS spiking retina observing real life scenes. Finally, we will discuss limitations of currently available memristors. The results presented are based on behavioral simulations and do not take into account non-idealities of devices and interconnects. The aim of this paper is to present, in a tutorial manner, an initial framework for the possible development of fully asynchronous STDP learning neuromorphic architectures exploiting two or three-terminal memristive type devices. All files used for the simulations are made available through the journal web site.
A 3.6 μs latency asynchronous frame-free event-driven dynamic-vision-sensor
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Journal of Solid-State Circuits, vol. 46, no. 6, pp 1443-1455 2011
abstract
doi pdf
This paper presents a 128 x 128 dynamic vision sensor. Each pixel detects temporal changes in the local illumination. A minimum illumination temporal contrast of 10% can be detected. A compact preamplification stage has been introduced that allows to improve the minimum detectable contrast over previous designs, while at the same time reducing the pixel area by 1/3. The pixel responds to illumination changes in less than 3.6 mu s. The ability of the sensor to capture very fast moving objects, rotating at 10 K revolutions per second, has been verified experimentally. A frame-based sensor capable to achieve this, would require at least 100 K frames per second.
A 32x32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 Meps throughput
L. Camuñas-Mesa, A. Acosta-Jiménez, C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems I-Regular Papers, vol. 58, no. 4, pp 777-790, 2011
abstract
doi pdf
This paper describes a convolution chip for event-driven vision sensing and processing systems. As opposed to conventional frame-constraint vision systems, in event-driven vision there is no need for frames. In frame-free event-based vision, information is represented by a continuous flow of self-timed asynchronous events. Such events can be processed on the fly by event-based convolution chips, providing at their output a continuous event flow representing the 2-D filtered version of the input flow. In this paper we present a 32 x 32 pixel 2-D convolution event processor whose kernel can have arbitrary shape and size up to 32 x 32. Arrays of such chips can be assembled to process larger pixel arrays. Event latency between input and output event flows can be as low as 155 ns. Input event throughput can reach 20 Meps (mega events per second), and output peak event rate can reach 45 Meps. The chip can be configured to discriminate between two simulated propeller-like shapes rotating simultaneously in the field of view at a speed as high as 9400 rps (revolutions per second). Achieving this with a frame-constraint system would require a sensing and processing capability of about 100 K frames per second. The prototype chip has been built in 0.35 mu m CMOS technology, occupies 4.3 x 5.4 mm(2) and consumes a peak power of 200 mW at maximum kernel size at maximum input event rate.
A five-decade dynamic-range ambient-light-independent calibrated signed-spatial-contrast AER retina with 0.1-ms latency and optional time-to-first-spike mode
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems I-Regular Papers, vol. 57, no. 10, pp 2632-2643, 2010
abstract
doi pdf
Address Event Representation (AER) is an emergent technology for assembling modular multiblock bio-inspired sensory and processing systems. Visual sensors (retinae) are among the first AER modules to be reported since the introduction of the technology. Spatial-contrast AER retinae are of special interest since they provide highly compressed data flow without reducing the relevant information required for performing recognition. The reported AER contrast retinae perform a contrast computation based on the ratio between a pixel's local light intensity and a spatially weighted average of its neighborhood. This resulted in compact circuits but with the penalty of all pixels generating output signals even if they sensed no contrast. In this paper, we present a spatial-contrast retina with a signed output: Contrast is computed as the relative difference (not the ratio) between a pixel's local light and its surrounding spatial average and normalized with respect to ambient light. As a result, contrast is ambient light independent, includes a sign, and the output will be zero if there is no contrast. Furthermore, an adjustable thresholding mechanism has been included, such that pixels remain silent until they sense an absolute contrast above the adjustable threshold. The pixel contrast-computation circuit is based on Boahen's biharmonic operator contrast circuit, which has been improved to include mismatch calibration and adaptive-current-based biasing. As a result, the contrast-computation circuit shows much less mismatch, is almost insensitive to ambient light illumination, and biasing is much less critical than in the original voltage biasing scheme. The retina includes an optional global reset mechanism for operation in ambient-light-independent Time-to-First-Spike contrast-computation mode. A 32 32 pixel test prototype has been fabricated in 0.35-mu m CMOS. Experimental results are provided.
Fast vision through frameless event-based sensing and convolutional processing: Application to texture recognition
J.A. Pérez-Carrasco, B. Acha, C. Serrano, L. Camuñas-Mesa, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Neural Networks, vol. 21, no. 4, pp 609-620, 2010
abstract
doi pdf
Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted.
CAVIAR: A 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking
R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, L. Camuñas-Mesa, R. Berner, M. Rivas-Pérez, T. Delbrueck, S.C. Liu, R. Douglas, P. Hafliger, G. Jiménez-Moreno, A. Civit-Ballcels, T. Serrano-Gotarredona, A.J. Acosta-Jiménez and B. Linares-Barranco
Journal Paper · IEEE Transactions on Neural Networks, vol. 20, no. 9, pp 1417-1438, 2009
abstract
doi pdf
This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.
A weak-to-strong inversion mismatch model for analog circuit design
G. Vicente-Sánchez, J. Velarde-Ramírez, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Analog Integrated Circuits and Signal Processing, vol. 59, no. 3, pp 325-340, 2009
abstract
doi
Abstract not available
On real-time AER 2-D convolutions hardware for neuromorphic spike-based cortical processing
R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jiménez, C. Serrano-Gotarredona, J.A. Pérez-Carrasco, B. Linares-Barranco, A. Linares-Barranco, G. Jiménez-Moreno and A. Civit-Ballcels
Journal Paper · IEEE Transactions on Neural Networks, vol. 19, no. 7, pp 1196-1219, 2008
abstract
doi pdf
In this paper, a chip that performs real-time image convolutions with programmable kernels of arbitrary shape is presented. The chip is a first experimental prototype of reduced size to validate the implemented circuits and system level techniques. The convolution processing is based on the address-event-representation (AER) technique, which is a spike-based biologically inspired image and video representation technique that favors communication bandwidth for pixels with more information. As a first test prototype, a pixel array of 16 x 16 has been implemented with programmable kernel size of up to 16 x 16. The chip has been fabricated in a standard 0.35-mu m complimentary metal-oxide-semiconductor (CMOS) process. The technique also allows to process larger size images by assembling 2-D arrays of such chips. Pixel operation exploits low-power mixed analog-digital circuit techniques. Because of the low currents involved (down to nanoamperes or even picoamperes), an important amount of pixel area is devoted to mismatch calibration. The rest of the chip uses digital circuit techniques, both synchronous and asynchronous. The fabricated chip has been thoroughly tested, both at the pixel level and at the system level. Specific computer interfaces have been developed for generating AER streams from conventional computers and feeding them as inputs to the convolution chip, and for grabbing AER streams coming out of the convolution chip and storing and analyzing them on computers. Extensive experimental results are provided. At the end of this paper, we provide discussions and results on scaling up the approach for larger pixel arrays and multilayer cortical AER systems.
A calibration technique for very low current and compact tunable neuromorphic cells: Application to 5-bit 20-nA DACs
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 55, no. 6, pp 522-526, 2008
abstract
doi pdf
Low current applications, like neuromorphic circuits, where operating currents can be as low as a few nanoamperes or less, suffer from huge transistor mismatches, resulting in around or less than 1-bit precisions. Recently, a neuromorphic programmable-kernel 2-D convolution chip has been reported where each pixel included two compact calibrated digital-to-analog converters (DACs) of 5-bit resolution, for currents down to picoamperes. Those DACs were based on MOS ladder structures, which although compact require 3N + 1 unit transistors (N is the number of calibration bits). Here, we present a new calibration approach not based on ladders, but on individually calibratable current sources made with MOS transistors of digitally adjustable length, which require only N-sized transistors. The scheme includes a translinear circuit-based tuning scheme, which allows us to expand the operating range of the calibrated circuits with graceful precision degradation, over four decades of operating currents. Experimental results are provided for 5-bit resolution DACs operating at 20 nA using two different translinear tuning schemes. Maximum measured precision is 5.05 and 7.15 b, respectively, for the two DAC schemes.
Inter-spike-intervals analysis of AER Poisson-like generator hardware
A. Linares-Barranco, M. Oster, D. Cascado, G. Jiménez, A. Civit and B. Linares Barranco
Journal Paper · Neurocomputing, vol. 70, no. 16-18, pp 2692-2700, 2007
abstract
doi
Address-Event-Representation (AER) is a communication protocol for transferring images between chips, originally developed for bio-inspired image-processing systems. Such systems may consist of a complicated hierarchical structure with many chips that transmit images among them in real time, while performing some processing (for example, convolutions). In developing AER-based systems it is very convenient to have available some means of generating AER streams from on-computer stored images. Rank order coding (ROC) and Poisson rate coding are the extremes of spikes coding. In this paper, we present a pseudo-random hardware method for generating AER streams in real time from a sequence of images stored in a computer's memory. The Kolmogorov-Smirnov test has been applied to quantify that this method follows a Poisson distribution of the spikes. A USB-AER board, developed by our RTCAR group, have been used for the measurements. An example scenaRío of use under the EU CAVIAR project is presented. (c) 2007 Elsevier B.V. All rights reserved.
The stochastic I-Pot: A circuit block for programming bias currents
R. Serrano-Gotarredona, L. Camuñas-Mesa, T. Serrano-Gotarredona, J.A. Leñero-Bardallo and B. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 54, no. 9, pp 760-764, 2007
abstract
doi pdf
In this brief, we present the "Stochastic I-Pot." It is a circuit element that allows for digitally programming a precise bias current ranging over many decades, from pico-amperes up to hundreds of micro-amperes. I-Pot blocks can be chained within a chip to allow for any arbitrary number of programmable bias currents. The approach only requires to provide the chip with three external pins, the use of an external current measuring instrument, and a computer. This way, once all internal I-Pots have been characterized, they can be programmed through a computer to provide any desired current bias value with very low error. The circuit block turns out to be very practical for experimenting with new circuits (specially when a large number of biases are required), testing wide ranges of biases, introducing means for current mismatch calibration, offsets compensations, etc. using a reduced number of chip pins. We show experimental results of generating bias currents with errors of 0.38% (8 bits) for currents varying from 176 mu A to 19.6 pA. Temperature effects are characterized.
On an efficient CAD implementation of the distance term in Pelgrom's mismatch model
B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 26, no. 8, pp 1534-1538, 2007
abstract
doi pdf
In 1989, Pelgrom et al published a mismatch model for MOS transistors, where the variation of parameter mismatch between two identical transistors is given by two independent terms: a size-dependent term and a distance-dependent term. Some CAD tools based on a nonphysical interpretation of Pelgrom's distance term result in excessive computationally expensive algorithms, which become nonviable even for circuits with a reduced number of transistors. Furthermore, some researchers are reporting new variations on the original nonphysically interpreted algorithms, which may render false results. The purpose of this paper is to clarify the physical interpretation of the distance term of Pelgrom et al. and indicate how to model it efficiently in prospective CAD tools.
A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems
J. Costas-Santos, T. Serrano-Gotarredona, R. Serrano-Gotarredona and B. Linares Barranco
Journal Paper · IEEE Transactions on Circuits and Systems I-Regular Papers, vol. 54, no. 7, pp 1444-1458, 2007
abstract
doi pdf
We present a 32 x 32 pixels contrast retina microchip that provides its output as an address event representation (AER) stream. Spatial contrast is computed as the ratio between pixel photocurrent and a local average between neighboring,pixels obtained with a diffuser network. This current-based computation produces an important amount of mismatch between neighboring pixels, because the currents can be as low as a few pico-amperes. Consequently, a compact calibration circuitry has been included to trimm each pixel. Measurements show a reduction in mismatch standard deviation from 57% to 6.6% (indoor light). The paper describes the design of the pixel with its spatial contrast computation and calibration sections. About one third of pixel area is used for a 5-bit calibration circuit. Area of pixel is 58 mu m x 56 mu m, while its current consumption is about 20 nA at 1-kHz event rate. Extensive experimental results are provided for a prototype fabricated in a standard 0.35-mu m CMOS process.
A neuromorphic cortical-layer microchip for spike-based event processing vision systems
R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jiménez and B. Linares Barranco
Journal Paper · IEEE Transactions on Circuits and Systems I-Regular Papers, vol. 53, no. 12, pp 2548-2566, 2006
abstract
doi pdf
We present a neuromorphic cortical-layer processing microchip for address event representation (AER) spike-based processing systems. The microchip computes 2-D convolutions of video information represented in AER format in real time. AER, as opposed to conventional frame-based video representation, describes visual information as a sequence of events or spikes in a way similar to biological brains. This format allows for fast information identification and processing, without waiting to process complete image frames. The neuromorphic cortical-layer processing microchip presented in this paper computes convolutions of programmable kernels over the AER visual input information flow. It not only computes convolutions but also allows for a programmable forgetting rate, which in turn allows for a bio-inspired coincidence detection processing. Kernels are programmable and can be of arbitrary shape and arbitrary size of up to 32 x 32 pixels. The convolution processor operates on a pixel array of size 32 x 32, but can process an input space of up to 128 x 128 pixels. Larger pixel arrays can be directly processed by tiling arrays of chips. The chip receives and generates data in AER format, which is asynchronous and digital. However, its internal operation is based on analog low-current circuit techniques. The paper describes the architecture of the chip and circuits used for the pixels, including calibration techniques to overcome mismatch. Extensive experimental results are provided, describing pixel operation and calibration, convolution processing with and without forgetting, and high-speed recognition experiments like discriminating rotating propellers of different shape rotating at speeds of up to 5000 revolutions per second.
A low-power current mode fuzzy-ART cell
T. Serrano-Gotarredona and B. Linares Barranco
Journal Paper · IEEE Transactions on Neural Networks, vol. 17, no. 6, pp 1666-1673, 2006
abstract
doi
This paper presents a very large scale integration (VLSI) implementation of a low-power current-mode fuzzy-adaptive resonance theory (ART) cell. The cell is based on a compact new current source multibit memory cell with online learning capability. A small prototype of the designed cell and its peripheral block has been fabricated in the AustriaMicroSystems (AMS)-0.35-mu m technology. The cell occupies a total area of 44 x 34 mu m(2) and consumes a maximum current of 22 nA.
On algorithmic rate-coded AER generation
A. Linares-Barranco, G. Jiménez-Moreno, B. Linares-Barranco and A. Civit-Balcells
Journal Paper · IEEE Transactions on Neural Networks, vol. 17, no. 3, pp 771-788, 2006
abstract
doi pdf
This paper addresses the problem of converting a conventional video stream based on sequences of frames into the spike event-based representation known as the address-event-representation (AER). In this paper we concentrate on rate-coded AER. The problem is addressed as an algorithmic problem, in which different methods are proposed, implemented and tested through software algorithms. The proposed algorithms are comparatively evaluated according to different criteria. Emphasis is put on the potential of such algorithms for a) doing the frame-based to event-based representation in real time, and b) that the resulting event streams ressemble as much as possible those generated naturally by rate-coded address-event VLSI chips, such as silicon AER retinae. It is found that simple and straightforward algorithms tend to have high potential for real time but produce event distributions that differ considerably from those obtained in AER VLSI chips. On the other hand, sophisticated algorithms that yield better event distributions are not efficient for real time operations. The methods based on linear-feedback-shift-register (LFSR) pseudorandom number generation is a good compromise, which is feasible for real time and yield reasonably well distributed events in time. Our software experiments, on a 1.6-GHz Pentium IV, show that at 50% AER bus load the proposed algorithms require between 0.011 and 1.14 ms per 8 bit-pixel per frame. One of the proposed LFSR methods is implemented in real time hardware using a prototyping board that includes a VirtexE 300 FPGA. The demonstration hardware is capable of transforming frames of 64, x 64 pixels of 8-bit depth at a frame rate of 25 frames per second, producing spike events at a peak rate of 10(7) events per second.
A precise 90 degrees quadrature OTA-C oscillator tunable in the 50-130-MHz range
B. Linares-Barranco, T. Serrano-Gotarredona, J. Ramos-Martos, J. Ceballos-Cáceres, J.M. Mora and A. Linares-Barranco
Journal Paper · IEEE Transactions on Circuits and Systems I-Regular Papers, vol. 51, no. 4, pp 649-663, 2004
abstract
doi pdf
We present a very-large-scale integration continuous-time sinusoidal operational transconductance amplifiers quadrature oscillator fabricated in a standard double-poly 0.8-mum CMOS process. The oscillator is tunable in the frequency range from 50 to 130 MHz. The two phases produced by the oscillator show a low-quadrature phase error. A novel current-mode amplitude control scheme is developed that allows for very small amplitudes. Stability of the amplitude control loop is studied as well as design considerations for its optimization. Experimental results are provided.
Current mode techniques for sub-pico-ampere circuit design
B. Linares-Barranco, T. Serrano-Gotarredona, R. Serrano-Gotarredona and C. Serrano-Gotarredona
Journal Paper · Analog Integrated Circuits and Signal Processing, vol. 38, no. 2-3, pp 103-119, 2004
abstract
doi
In this paper we explore the low current limit that standard CMOS technologies offer for current mode based VLSI designs. We show and validate a reliable circuit design technique for current mode signal processing down to fempto-amperes. We will take advantage of specific-current extractors and logarithmic current splitters to obtain on-chip sub-pA currents. Then we will use a special on-chip saw-tooth oscillator to monitor and measure currents down to a few fempto-amps. This way, sub-pA currents are characterized without driving them off-chip, nor requiring expensive instrumentation with complicated low leakage setups. A special current mirror is also introduced for reliably replicating such low currents. As an example, a simple log-domain first-order low-pass filter is implemented that uses a 100 fF capacitor and a 3.5 fA bias current to achieve a cut-off frequency of 0.5 Hz and using an area of 12 x 24.35 mum(2) in a standard 0.35 mum CMOS process. A technique for characterizing noise at these currents is described and verified. Also, temperature dependence of leakage currents is measured as well.
Guest editorial: Special issue on neural networks hardware implementations
B. Linares-Barranco, A.G. Andreou, G. Indiveri and T. Shibata
Journal Paper · IEEE Transactions on Neural Networks, vol. 14, no. 5, pp 976-979, 2003
abstract
doi pdf
Abstract not available
Compact low-power calibration mini-DACs for neural arrays with programmable weights
B. Linares-Barranco, T. Serrano-Gotarredona and R. Serrano-Gotarredona
Journal Paper · IEEE Transactions on Neural Networks, vol. 14, no. 5, pp 1207-1216, 2003
abstract
doi pdf
This paper considers the viability of compact low-resolution low-power mini digital-to-analog converters (mini-DACs) for use in large arrays of neural type cells, where programmable weights are required. Transistors are biased in weak inversion in order to yield small currents and low power consumptions, a necessity when building large size arrays. One important drawback of weak inversion operation is poor matching between transistors. The resulting effective precision of a fabricated array of 50 DACs turned out to be 47% (1.1,bits), due to transistor mismatch. However, it is possible to combine them two by two in order to build calibrated DACs, thus compensating for inter-DAC mismatch. It is shown experimentally that the precision can be improved easily by a factor of 10 (4.8% or 4.4 bits), which makes these DACs viable for low-resolution applications such as massive arrays of neural processing circuits. A design methodology is provided, and illustrated through examples, to obtain calibrated mini-DACs of a given target precision. As an example application, we show simulation results of using this technique to calibrate an array of digitally controlled integrate-and-fire neurons.
Log-domain implementation of complex dynamics reaction-diffusion neural networks
T. Serrano-Gotarredona and B. Linares Barranco
Journal Paper · IEEE Transactions on Neural Networks, vol. 14, no. 5, pp 1337-1355, 2003
abstract
doi pdf
In this paper, we have identified a second-order reaction-diffusion differential equation able to reproduce through parameter setting different complex spatio-temporal behaviors. We have designed a log-domain hardware that implements the spatially discretized version of the selected reaction-diffusion equation. The logarithmic compression of the state variables allows several decades of variation of these state variables within subthreshold operation of the MOS transistors. Furthermore, as all the equation parameters are implemented as currents, they can be adjusted several decades. As a demonstrator, we have designed a chip containing a linear array of ten second-order dynamics coupled cells. Using this hardware, We have experimentally reproduced two complex spatio-temporal phenomena: the propagation of travelling waves and of trigger waves, as well as isolated oscillatory cells.
On the design and characterization of femtoampere current-mode circuits
B. Linares-Barranco and T. Serrano-Gotarredona
Journal Paper · IEEE Journal of Solid-State Circuits, vol. 38, no. 8, pp 1353-1363, 2003
abstract
doi pdf
In this paper, we show and validate a reliable circuit design technique based on source voltage shifting for current-mode signal processing down to femtoamperes. The technique involves specific-current extractors and logarithmic current splitters for obtaining on-chip subpicoampere currents. It also uses a special on-chip sawtooth oscillator to monitor and measure currents down to a few femtoamperes. This way, subpicoampere currents are characterized without driving them off chip and requiring expensive instrumentation with complicated low leakage setups. A special current mirror is also introduced for reliably replicating such low currents. As an example, a simple log-domain first-order low-pass filter is implemented that uses a 100-fF capacitor and a 3.5-fA bias current to achieve a cutoff frequency of 0.5 Hz. A technique for characterizing noise at these currents is also described and verified. Finally, transistor mismatch measurements are provided and discussed. Experimental measurements are shown throughout the paper, obtained from prototypes fabricated in the AMS 0.35-mum three-metal two-poly standard CMOS process.
Precise 90 degrees quadrature current-controlled oscillator tunable between 50-130 MHz
B. Linares-Barranco, T. Serrano-Gotarredona, J. Ramos-Martos, J. Ceballos-Cáceres, J.M. Mora and A. Linares-Barranco
Journal Paper · Electronics Letters, vol. 39, no. 11, pp 823-825, 2003
abstract
doi pdf
A VLSI continuous time sinusoidal OTA-C quadrature oscillator fabricated in a standard double-poly 0.8 mum CMOS process is presented. The oscillator is tunable in the frequency range from 50-130 MHz. The two phases produced by the oscillator show an extremely low phase difference error (less than 2degrees over the whole frequency range). A novel current mode amplitude control scheme is developed that allows for very small amplitudes. Experimental results are provided.
Current-mode fully-programmable piece-wise-linear block for neuro-fuzzy applications
T. Serrano-Gotarredona and B. Linares-Barranco
Journal Paper · Electronics Letters, vol. 38, no. 20, pp 1165-1166, 2002
abstract
doi pdf
A new method to implement an arbitrary piece-wise-linear characteristic in current mode is presented. Each of the breaking points and each slope is separately controllable. As an example a block that implements an N-shaped piece-wise-linearity has been designed. The N-shaped block operates in the subthreshold region and uses only ten transistors. These characteristics make it especially suitable for large arrays of neuro-fuzzy systems where the number of transistors and power consumption per cell is an important concern. A prototype of this block has been fabricated in a 0.35μm CMOS technology. The functionality and programmability of this circuit has been verified through experimental results.
Conferences
Energy-efficient Brain-inspired Oscillatory Neural Networks using Phase-Transition Material
M. Jiménez, B. Linares-Barranco, M.J. Avedillo and J. Núñez
Conference · Workshop on Deep Learning meets Neuromorphic Hardware. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ECML PKDD 2023
abstract
Oscillatory Neural Network (ONN) is a promising neuromorphic computing approach which uses networks of frequency-locked coupled oscillators, and their inherent parallel synchronization to compute. Also, ONN can be im-plemented using phase-transition materials using nano-scale area, low voltage amplitude and reduced power consumption, being an efficient way to im-plement oscillator-based computing. In state-of-theart, ONN is built with a fully-connected architecture, with coupling configured depending on the tar-get application. Its most widespread use has been as associative memory, but recently it is gathering interest as a solver for non-deterministic polynomial time problem (NP-hard). This is performed on the basis of encoding the NP-problem in the Ising model, so ONN operates as an Ising machine. ONN state naturally evolves to minimum points in the Hamiltonian energy function re-sorting to its rich non-lineal dynamics, supposing a promising paradigm of fast, low-power, parallel computation.
Experimental Demonstration of Associative Memory in Coupled Differential Oscillator Networks
M. Jiménez, J. Núñez, J. Shamsi, B. Linares-Barranco and M.J. Avedillo
Conference · XXXVIII Conference on Design of Circuits and Integrated Systems DCIS 2023
abstract
The utilization of phase-transition materials-based nano-oscillators is being investigated to apply various non-traditional computing paradigms. Specifically, vanadium dioxide (VO2) devices are used to design self-sustained non-linear oscillators that can be employed for oscillatory neural networks (ONNs). In addition, in these ONN architectures sub-harmonic injection locking (SHIL) can be exploited to ensure that each neuron's phase information can only adopt one of two possible values. An integrated circuit demonstrator of an analog 9-neuron ONN using a deep-submicron commercial technology have been designed and fabricated. The oscillators forming the neurons closely resemble those designed using VO2 devices. The capability of the fabricated ONN to work as an associative memory has been tested. An example of two store patterns has been used to show that the ONN successfully stores the two patterns and exhibits the associative memory functionality.
Novel Iterative Hebbian Learning Rule for Oscillatory Associative Memory
M. Jiménez, M.J. Avedillo, B. Linares-Barranco and J. Núñez
Conference · XXXVIII Conference on Design of Circuits and Integrated Systems DCIS 2023
abstract
Alternative paradigms to the von Neumann computing scheme are currently arousing huge interest. Oscillatory neural networks (ONNs) using emerging phase-change materials constitute an energy-efficient, massively parallel, brain-inspired, in-memory computing approach. The encoding of information in the phase pattern of frequency-locked, weakly coupled oscillators makes it possible to exploit their rich nonlinear dynamics and their synchronization phenomena for computing. A single fully connected ONN layer can implement an auto-associative memory comparable to that of a Hopfield network. Hebbian learning rule is the most widely adopted method for configuring ONNs for such applications, despite its well-known limitations. Other approaches that perform better than the Hebbian rule are not useful for ONN training due to the constraints imposed by its physical implementation. This paper proposes a new approach and compares it with previous work. The proposed method has been shown to produce competitive results in terms of pattern recognition accuracy with reduced precision in synaptic weights, and to be suitable for online learning.
Enhancing Storage Capabilities of Oscillatory Neural Networks as Associative Memory
M. Jiménez, M.J. Avedillo, J. Núñez and B. Linares-Barranco
Conference · XXXVII Conference on Design of Circuits and Integrated Systems DCIS 2022
abstract
Abstract not available
Electronically Foveated Dynamic Vision Sensor
T. Serrano-Gotarredona, F. Faramarzi and B. Linares-Barranco
Conference · IEEE International Conference on Omni-Layer Intelligent Systems COINS 2022
abstract
This paper proposed a vision system which implements a foveal mechanism to concentrate the attention and dynamically control the center and size of region of interest. The core of the system is an electronically-foveated dynamic vision sensor. An architecture and implementation of an electronically-foveated dynamic vision sensor is proposed. Simulation results demonstrating its operation are provided.
Reliability Analysis of a Spiking Neural Network Hardware Accelerator
T. Spyrou, S.A. El-Sayed, E. Afacan, L.A. Camunas-Mesa, B. Linares-Barranco and H.G. Stratigopoulos
Conference · Conference · Design, Automation and Test in Europe DATE 2022
abstract
Despite the parallelism and sparsity in neural network models, their transfer into hardware unavoidably makes them susceptible to hardware-level faults. Hardware-level faults can occur either during manufacturing, such as physical defects and process-induced variations, or in the field due to environmental factors and aging. The performance under fault scenarios needs to be assessed so as to develop cost-effective fault-tolerance schemes. In this work, we assess the resilience characteristics of a hardware accelerator for Spiking Neural Networks (SNNs) designed in VHDL and implemented on an FPGA. The fault injection experiments pinpoint the parts of the design that need to be protected against faults, as well as the parts that are inherently fault-tolerant.
A Hybrid Memristor/CMOS SNN for Implementing One-Shot Winner-Takes-All Training
J. Ahmadi-Farsani, S. Ricci, S. Hashemkhani, D. Ielmini, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2022
abstract
This paper presents a spiking neural network for pattern recognition. The network synapses are realized byReRAM cells, which are a stack of AU/Ti/C/Ti/HfO2/Pt. These cells are connected to an array of NMOS transistors (fabricated in a CMOS 180nm technology) to form a 4by4 1T1R crossbar between pre and post-synaptic circuitries. The pre-synaptic part contains conditioning circuits to reshape the inputs before applying them to the memristive crossbar. The post-synaptic section includes current attenuators that allowed the memristor domain currents to be mapped to neuron domain currents, as well as physiologically realistic neuron circuits fabricated in a CMOS 180nm technology. As a demonstrator, the network is trained with a one-shot winner-takes-all method to differentiate four input patterns in its inference mode.
Event Data Downscaling for Embedded Computer Vision
A. Gruel, J. Martinet, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Computer Vision Theory and Applications VISAPP 2022
abstract
Event cameras (or silicon retinas) represent a new kind of sensor that measure pixel-wise changes in brightness and output asynchronous events accordingly. This novel technology allows for a sparse and energy-efficient recording and storage of visual information. While this type of data is sparse by definition, the event flow can be very high, up to 25M events per second, which requires significant processing resources to handle and therefore impedes embedded applications. Neuromorphic computer vision and event sensor based applications are receiving an increasing interest from the computer vision community (classification, detection, tracking, segmentation, etc.), especially for robotics or autonomous driving scenarios. Downscaling event data is an important feature in a system, especially if embedded, so as to be able to adjust the complexity of data to the available resources such as processing capability and power consumption. To the best of our knowledge, this works is the first attempt to formalize event data downscaling. In order to study the impact of spatial resolution downscaling, we compare several features of the resulting data, such as the total number of events, event density, information entropy, computation time and optical consistency as assessment criteria. Our code is available online at https://github.com/amygruel/EvVisu.
FeFETs for Phase Encoded Oscillatory based Computing
J. Núñez, M. Jiménez, B. Linares-Barranco and M.J. Avedillo
Conference · Design, Automation and Test in Europe DATE 2022
abstract
Coupled nano-oscillators are attracting increasing interest because of their potential to perform computation efficiently, enabling new applications in computing and information processing. The potential of Ferroelectric Field-Effect Transistor (FeFET) for such applications has recently been recognized, which is a step towards the physical realization given their ease of cointegration with commercial CMOS technologies. This paper investigates the design of oscillators using FeFETs and their potential for oscillator-based computing in which information is encoded in phase. As applications, we present the operation of FeFET coupled oscillators systems for graph coloring and Max-Cut problems, including subharmonic injection mechanism to discretize the phase in the second one.
Neuron Fault Tolerance in Spiking Neural Networks
T. Spyrou, S.A. El-Sayed, E. Afacan, L.A. Camunas-Mesa, B. Linares-Barranco and H.G. Stratigopoulos
Conference · Conference · Design, Automation and Test in Europe DATE 2021
abstract
The error-resiliency of Artificial Intelligence (AI) hardware accelerators is a major concern, especially when they are deployed in mission-critical and safety-critical applications. In this paper, we propose a neuron fault tolerance strategy for Spiking Neural Networks (SNNs). It is optimized for low area and power overhead by leveraging observations made from a large-scale fault injection experiment that pinpoints the critical fault types and locations. We describe the fault modeling approach, the fault injection framework, the results of the fault injection experiment, the fault-tolerance strategy, and the fault-tolerant SNN architecture. The idea is demonstrated on two SNNs that we designed for two SNN-oriented datasets, namely the N-MNIST and IBM's DVS128 gesture datasets.
Baseline Features Extraction from Microelectrode Array Recordings in an in vitro model of Acute Seizures using Digital Signal Processing for Electronic Implementation
G. Galeote-Checa, G. Panuccio, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Omni-Layer Intelligent Systems COINS 2021
abstract
Latest advances in CMOS technology, neuromorphic computing and flexible electronics are leading the way to a new era of brain implantable devices that promise to provide innovative and more effective treatments for brain disorders. Real-time signal detection and classification, as well as anticipation of brain electrical activity are yet under development. Microelectrode arrays (MEA) arise as a promising technology enabling detection of local fieldpotentials from multiple locations and permitting the acquisition of more information on brain network electrical activity than conventional electrophysiology techniques. However, whereas most of the electrophysiological studies addressing brain activity have focused on events/patterns analysis, no one has so far addressed the features that might be hidden within the signal baseline. Such features might be particularly relevant in the context of epilepsy, as the signal baseline may carry relevant information for seizure prediction. Here, we present a preliminary processing and analysis of signal baseline acquired through MEA in an in vitro model of limbic seizures. The signal conditioning was implemented using an infinite impulse response (IIR) digital filter. After signal preprocessing, we applied an averaging method to 16 baseline sections to find common patterns and to study the frequency spectrum of this type of signal. We have found signal components between 0.5-2 Hz and peaks at 350, 390 and 650 Hz. In addition, the reconstruction of the averaged signal may provide insights into the main baseline wave patterns. These results might be a preliminary step to study the influence of those components on a biological basis. Based on these results, we propose a possible electronic architecture implementation of the signal processing method.
Novel programmable single pulse generator for producing pulse widths in different time scales
H. Erfanijazi, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Content-Based Multimedia Indexing CBMI 2021
abstract
A novel programmable single pulse generator for producing a pulse in different time scales and amplitudes is presented in this paper to drive integrate-and-fire neuron circuits. The proposed circuit generates pulses with controlled variable width from 1.3ns up to several milliseconds that coincide with a wide range of binary and analog memristor applications. The designed pulse generator is intended as a tool to precisely control the amount of charge injected in memristors devices so that precise characterization of the memristors can be done without using external controlling circuitry. Furthermore, a much finer precision in the control of total injected charge can be achieved using the proposed technique. The proposed single pulse generator has been designed in XFAB 0.35µm technology to characterize the properties of memristors.
A Real-Time DSP-based Biohybrid MEA System for Seizure Detection In Vitro
J. Ahmadi-Farsani, D. Caron, G. Panuccio, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Symposium on Medical Measurements and Applications MeMeA 2021
abstract
This paper presents a biohybrid arrangement made of a commercial microelectrode array (MEA) system for seizure-like activity detection in brain slices. The set-up takes advantage of an embedded fixed-point digital signal processor (DSP) to implement a neuron model and a field-potential to spike converter (FP2SP). The neuron model is biologically plausible and capable of generating various firing modalities. Based on a three-step algorithm, FP2SP extracts spikes from the epileptiform activity generated by brain slices. The seizure detector system is developed by connecting the FP2SP to the model neuron and properly tuning the FP2SP parameters. The results show that all the blocks of this system can operate properly in real-time mode and recognize seizure-like activity.
Implementation of binary stochastic STDP learning using chalcogenide-based memristive devices
C. Mohan, L.A. Camuñas-Mesa, J.M. de la Rosa, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2020
abstract
The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spike-timing-dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation between pre- and post-synaptic spikes. In this work, we demonstrate experimentally that binary stochastic STDP learning can be obtained from a memristor when the appropriate pulses are applied at both sides of the device.
Spiking Neuron Hardware-Level Fault Modeling
S.A. El-Sayed, T. Spyrou, A. Pavlidis, E. Afacan, L.A. Camunas-Mesa, B. Linares-Barranco and H.G. Stratigopoulos
Conference · IEEE Int. Symposium on On-Line Testing and Robust System Design IOLTS 2020
abstract
The deployment of Artificial Intelligence (AI) hardware accelerators in a variety of applications, including safety-critical ones, requires assessing their inherent reliability to hardware-level faults and developing cost-effective fault tolerance techniques. This entails performing large-scale fault simulation experiments. However, transistor-level fault simulation is prohibitive and fault simulation should be carried out at a higher abstraction level. In this work, we focus on spiking neural networks (SNNs), and we follow a bottom-up approach starting from transistor-level simulations for developing a neuron behavioral-level fault model that can be readily employed for performing behavioral-level fault simulation of deep SNNs.
Experimental Body-Input Three-Stage DC Offset Calibration Scheme for Memristive Crossbar
C. Mohan, L.A. Camuñas-Mesa, E. Vianello, C. Reita, J.M. de la Rosa, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2020
abstract
Reading several ReRAMs simultaneously in a neuromorphic circuit increases power consumption and limits scalability. Applying small inference read pulses is a vain attempt when offset voltages of the read-out circuit are decisively more. This paper presents an experimental validation of a three-stage calibration scheme to calibrate the DC offset voltage across the rows of the memristive crossbar. The proposed method is based on biasing the body terminal of one of the differential pair MOSFETs of the buffer through a series of cascaded resistor banks arranged in three stages-coarse, fine and finer stages. The circuit is designed in a 130 nm CMOS technology, where the OxRAM-based binary memristors are built on top of it. A dedicated PCB and other auxiliary boards have been designed for testing the chip. Experimental results validate the presented approach, which is only limited by mismatch and electrical noise.
Auxiliary Pulse-Extender and Current-Attenuator Circuits for Flexible Interaction with Memristive Crossbars in SNNs
J. Ahmadi-Farsani, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Electronics Circuits and Systems ICECS 2020
abstract
This paper presents a pulse-extender, a delay-element, and a current-attenuator as auxiliary circuits that make it possible to have flexible interaction with memristor crossbars in spiking neural networks. In the presynaptic part, the pulse-extender makes the inputs compatible with the pulsed-characterization of memristors. In the post-synaptic part, the current attenuator relaxes the system in terms of requiring low-offset amplifiers and also makes it possible to design neurons with small membrane capacitors. The circuits are fabricated in a CMOS 180nm technology. The measurements verify that these blocks play an important role in reaching an SNN with real-time performance.
Oscillatory Hebbian Rule (OHR): An adaption of the Hebbian rule to Oscillatory Neural Networks
J. Shamsi, M.J. Avedillo and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2020
abstract
Hebbian rule plays an important role in training of artificial neural networks. According to this rule, a synaptic weight between two neurons is increased or decreased depending on the activity of the presynaptic and postsynaptic neurons. In this paper, an oscillatory version of the Hebbian rule is proposed for ONNs and is called Oscillatory Hebbian Rule (OHR). OHR simply expresses the weight change as a function of the phase difference between the presynaptic and postsynaptic neurons. Similar to STDP that weight change is an exponential function of the time difference between the presynaptic and postsynaptic spikes, OHR relates weight change to the phase difference between the presynaptic and postsynaptic neurons using exponential functions. Specifically, when two neurons are in-phase, the weight between them is increased while a weight between two anti-phase neurons is decreased. Simulation results show the capability of OHR for both supervised and unsupervised learning. In supervised learning, a basic block of feedforward architectures is trained as a classifier. When the basic block is used in unsupervised mode, it is capable to learn patterns while the output phase is converged to a specific phase.
A Current-Attenuator for Performing Read Operation in Memristor-Based Spiking Neural Networks
J. Ahmadi-Farsani, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2020
abstract
This paper presents a current attenuator fabricated in a CMOS 180nm technology, which works based on a CMOS ladder scheme. The attenuation factor is 104.14 dB, while it shows a non-linearity feature of less than 1.8 %. The circuit occupies an area of 2448 µm2 . Since the output current could be as low as tens of femtoamperes, an on-chip testing circuit is also proposed to make the lab-measurements as accurate as possible. The final results show that chip-measurements are following simulations. As a demonstrator, the current attenuator is internally connected to a compact CMOS neuron cell. The output membrane potential shows that the neuron is generating a real-time firing modality, and consequently approves that the current-attenuator is working robustly.
Implementation of a tunable spiking neuron for STDP with memristors in FDSOI 28nm
L.A. Camuñas-Mesa, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Artificial Intelligence Circuits and Systems AICAS 2020
abstract
Hybrid memristor-CMOS techniques have been recently proposed to build large-scale neural networks with learning capabilities. The intrinsic characteristics of memristors make them specially suited to implement synaptic connections between layers of spiking neurons, undergoing STDP learning (Spike-Timing-Dependent Plasticity) mechanisms when processing spikes with particular shapes. In a previous work, we proposed a tunable spiking neuron circuit which can generate spikes with controllable shape. In this work, the spike generator circuit has been implemented in FDSOI 28nm technology, and it has demonstrated its capability to produce spikes with pulse widths in the range between 8 µ s and 100ms.
Introduction and Analysis of an Event-Based Sign Language Dataset
A. Vasudevan, P. Negri, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Automatic Face and Gesture Recognition FG 2020
abstract
Human gestures recognition is a complex visual recognition task where motion across time distinguishes the type of action. Automatic systems tackle this problem using complex machine learning architectures and training datasets. In recent years, the use and success of robust deep learning techniques was compatible with the availability of a great number of these sets. This paper presents SL-Animals-DVS, an event-based action dataset captured by a Dynamic Vision Sensor (DVS). The DVS records humans performing sign language gestures of various animals as a continuous spike flow at very low latency. This is especially suited for sign language gestures which are usually made at very high speeds. We also benchmark the recognition performance on this data using two state-of-the-art Spiking Neural Networks (SNN) recognition systems. SNNs are naturally compatible to make use of the temporal information that is provided by the DVS where the information is encoded in the spike times. The dataset has about 1100 samples of 58 subjects performing 19 sign language gestures in isolation at different scenarios, providing a challenging evaluation platform for this emerging technology.
Using Neural Networks for Optimum band selection in Cognitive-Radio Systems
V. Zúñiga, L. Camuñas-Mesa, B. Linares-Barranco, T. Serrano-Gotarredona and J.M. de la Rosa
Conference · IEEE International Conference on Electronics Circuits and Systems ICECS 2020
abstract
The growing development of Internet of Things (IoT) devices is producing an increasing use of the electromagnetic spectrum for wireless communications. Cognitive Radio (CR) technology provides communication terminals with the capability to select arbitrary frequency bands dynamically in order to make a more efficient use of the frequency spectrum and bands occupied by different standards and communication protocols. In this work, we propose a system which uses Long Short-Term Memory (LSTM) networks to predict the future occupation of frequency bands and modifies the specifications of the analog and radio-frequency front-end, adapting dynamically to the best communication channel. System-level simulations of a band-pass filter are shown as a case study to validate the presented approach.
Self-Testing Analog Spiking Neuron Circuit
S.A. El-Sayed, L.A. Camunas-Mesa, B. Linares-Barranco and H.G. Stratigopoulos
Conference · Int. Conf. on Synthesis, Modeling, Analysis and Simulation Methods and Applications to Circuit Design SMACD 2019
abstract
Hardware-implemented neural networks are foreseen to play an increasing role in numerous applications. In this paper, we address the problem of post-manufacturing test and self-test of hardware-implemented neural networks. In particular, we propose a self-testable version of a spiking neuron circuit. The self-test wrapper is a compact circuit composed of a low-precision ramp generator and a small digital block. The self-test principle is demonstrated on a spiking neuron circuit design in 0.35μm CMOS technology.
Low-power hardware implementation of SNN with decision block for recognition tasks
L. A. Camuñas-Mesa, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Electronics Circuits and Systems ICECS 2019
abstract
We propose a fully configurable spiking convolutional node with rate saturation mechanism that can be used to implement arbitrary Convolutional Neural Networks (ConvNets) on FPGA. Using this node, a 4-layer ConvNet with 22 convolutional nodes and a decision block trained for poker card symbol recognition has been implemented in a Spartan6 FPGA, being tested with a stimulus where 40 poker cards where observed by a Dynamic Vision Sensor (DVS). In this paper, we study different strategies for the decision block to maximize the recognition rate with minimum power consumption.
Digital-Signal-Processor Realization of Izhikevich Neural Network for Real-Time Interaction with Electrophysiology Experiments
J. Ahmadi-Farsani, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Electronics Circuits and Systems ICECS 2019
abstract
The paper presents a realization on a digital signal processor of an Izhikevich Neural Network operating with biologically plausible real-time constants. The paper demonstrates the real-time realization of different neuron behavioral modes, i.e., regular spiking, chattering, bursting, and fast spiking under proper parametrization. Real-time spike-timing-dependentplasticity has also been embedded in the neural network realization. The paper studies the maximum array size that can be implemented on a TMS320C6455 microprocessor to be able to reproduce correctly the real-time dynamics of the different behaviors. The TMS320C6454, from the same DSP family as TMS320C6455, is embedded in a commercial microelectrode array system for real time interaction with biological neural cell cultures. As demonstrator, a simple classification of two binary patterns has been implemented. Upon learning activation, the system robustly unsupervisely learns to differentiate the two patterns.
Learning weights with STDP to build prototype images for classification
A. Vasudevan, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Design and Technology of Integrated Systems in Nanoscale Era DTIS 2019
abstract
The combination of Spike Timing Dependent Plasticity (STDP) and latency coding used in a spiking neural network has been shown to learn hierarchical features. In this paper we propose a new way to classify images using an SVM. Prototype images are built from the weights learned in an unsupervised manner using STDP. The prototype images are cross correlated with the input image and the peak of the cross correlation with each prototype image is used as additional features for an SVM. The network, demonstrated on the MNIST data set, achieves 99.15% testing accuracy which is the best reported accuracy for a SNN with unsupervised training.
A current attenuator for efficient memristive crossbars read-out
C. Mohan, J.M. de la Rosa, E. Vianello, L. Perniola, C. Reita, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2019
abstract
This paper presents a new current attenuator circuit to scale down the inference currents in memristor based crossbars that drive integrate-and-fire neurons, which subsequently allows to reduce the size of integrating capacitors by several orders of magnitude, making IC integration possible. The proposed circuit uses a linear switch to divide the inference current and scale it down by a factor of about 104. The proposed attenuator has been designed in 130nm CMOS technology. Simulation results considering noise, process and temperature variations are shown to validate the presented approach.
Conversion of Synchronous Artificial Neural Network to Asynchronous Spiking Neural Network using sigma-delta quantization
A. Yousefzadeh, S. Hosseini, P. Holanda, S. Leroux, T. Werner, T. Serrano-Gotarredona and B. Linares-Barranco, B. Dhoedt and P. Simoens
Conference · IEEE International Conference on Artificial Intelligence Circuits and Systems AICAS 2019
abstract
Artificial Neural Networks (ANNs) show great performance in several data analysis tasks including visual and auditory applications. However, direct implementation of these algorithms without considering the sparsity of data requires high processing power, consume vast amounts of energy and suffer from scalability issues. Inspired by biology, one of the methods which can reduce power consumption and allow scalability in the implementation of neural networks is asynchronous processing and communication by means of action potentials, so-called spikes. In this work, we use the wellknown sigma-delta quantization method and introduce an easy and straightforward solution to convert an Artificial Neural Network to a Spiking Neural Network which can be implemented asynchronously in a neuromorphic platform. Briefly, we used asynchronous spikes to communicate the quantized output activations of the neurons. Despite the fact that our proposed mechanism is simple and applicable to a wide range of different ANNs, it outperforms the state-of-the-art implementations from the accuracy and energy consumption point of view. All source code for this project is available upon request for the academic purpose.
On the Hardware Efficiency of 1-bit Homeostatic Stochastic STDP
A. Yousefzadeh, E. Stromatias, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Cognitive Computing Conference 2018
abstract
Here we propose and demonstrate on FPGA hardware a homeostatic stochastic 1-bit weight STDP rule (whose 19 neurons have separate thresholds for integrating spikes and for triggering STDP update) used in a self-learning 20 feature extraction layer, which when combined with a rudimentary hebbian spiking classifier is capable of classifying with up to 100% accuracy a DVS recorded poker card symbol benchmark.
Scene Context Classification with Event-Driven Spiking Deep Neural Networks
P. Negri, M. Soto, B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Conference on Electronics Circuits and Systems ICECS 2018
abstract
Event-Driven computation is attracting growing attention among researchers for several reasons. On one hand, the availability of new bio-inspired retina-like vision sensors that provide spiking outputs, like the Dynamic Vision Sensor (DVS) makes it possible to demonstrate energy efficient and high-speed complex vision tasks. On the other hand, the emergence of abundant new nanoscale devices that operate as tunable two-terminal resistive elements, which when operated through dynamic pulsing techniques emulate learning and processing in the brain, promise an explosion of highly compact energy efficient neuromorphic event-driven applications. In this paper, we focus for the first time on a high-level cognitive task, namely scene context classification, performed by event-driven computations and using real sensory data from a DVS camera.
Real-time temporal frequency detection in FPGA using event-based vision sensor
S. Hoseini and B. Linares-Barranco
Conference · IEEE International Conference on Intelligent Computer Communication and Processing ICCP 2018
abstract
A dynamic vision sensor (DVS) is a new type of vision sensor in which each pixel acts as a motion sensor and generates highly time-accurate events when it detects movement in the scene. The high temporal precision of these types of vision sensors allows the extraction of different low-level temporal features, which is not possible when using a frame-based camera. Hierarchical vision-processing systems use low-level features to recognize a higher level of abstraction. One of the lowlevel features that can be extracted with DVS is the temporal frequency. This feature can be used along with other visual features for more accurate object recognition when the object has rotating parts, such as a quadcopter. This work is an extension of our previous work, wherein we proposed an algorithm to extract this temporal low-level feature by using a DVS. In this work, we proposed a digital circuit with a small footprint to extract the frequency of rotating objects in real time with very low latency. We have synthesized the digital circuit in Spartan-6 field-programmable gate array (FPGA) and also in UMC 180-nm technology to measure the performance, power consumption, and occupied area. MATLAB and Verilog codes for this work are available for academic purposes upon request.
Hybrid Neural Network, an Efficient Low-Power Digital Hardware Implementation of Event-based Artificial Neural Network
A. Yousefzadeh, G. Orchard, E. Stromatias, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2018
abstract
Interest in event-based vision sensors has proliferated in recent years, with innovative technology becoming more accessible to new researchers and highlighting such sensors' potential to enable low-latency sensing at low computational cost. These sensors can outperform frame-based vision sensors regarding data compression, dynamic range, temporal resolution and power efficiency. However, available mature frame-based processing methods by using Artificial Neural Networks (ANNs) surpass Spiking Neural Networks (SNNs) in terms of accuracy of recognition. In this paper, we introduce a Hybrid Neural Network which is an intermediate solution to exploit advantages of both event-based and frame-based processing. We have implemented this network in FPGA and benchmarked its performance by using different event-based versions of MNIST dataset. HDL codes for this project are available for academic purpose upon request.
Performance Comparison of Time-Step-Driven Versus Event-Driven Neural State Update Approaches in Spinnaker
M. Soto, A. Yousefzadeh, T. Serrano-Gotarredona, F. Galluppi, L. Plana, S. Furber and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2018
abstract
The SpiNNaker chip is a multi-core processor optimized for neuromorphic applications. Many SpiNNaker chips are assembled to make a highly parallel million core platform. This system can be used for simulation of a large number of neurons in real-time. SpiNNaker is using a general purpose ARM processor that gives a high amount of flexibility to implement different methods for processing spikes. Various libraries and packages are provided to translate a high-level description of Spiking Neural Networks (SNN) to low-level machine language that can be used in the ARM processors. In this paper, we introduce and compare three different methods to implement this intermediate layer of abstraction. We have examined the advantages of each method by various criteria, which can be useful for professional users to choose between them. All the codes that are used in this paper are available for academic propose.
An Intrinsic Method for Fast Parameter Update on the Spinnaker Platform
M. Soto, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2018
abstract
Neuromorphic Computing or Spiking (also called Event-Driven) Neural Systems are becoming of high interest as they potentially allow for lower power hardware computing platforms, where power consumption is data driven. Traditional approaches (both in software and in hardware), which are not data driven, rely on generic system state updates, consuming a fixed amount of computing resources at each step, independent on the data itself. In neuromorphic spiking or (event-driven) computing systems power is consumed (in principle) if new data is transferred, either at the system input, system output, or internally between computing nodes. One such neuromorphic event-driven computing platform is the scalable SpiNNaker system, which is aimed for a million ARM core platform, capable of emulating in the order of a billion neurons in real time. An important practical drawback of the platform is the long time it takes to download to the hardware a given computational architecture. This step has to be repeated even if one wants to update a set of parameters. Here we present a method for updating internal parameters without downloading again the full architecture, by adding special neurons into the computing architecture which when they spike change given parameters. This allows to download the computing architecture only once to the SpiNNaker platform, and then take advantage of its highly efficient communication network to command specific parameter changes. This allows for intensive parameter searches in a more efficient manner.
Event-Driven Configurable Module with Refractory Mechanism for ConvNets on FPGA
L.A. Camuñas-Mesa, Y. Domínguez-Cordero, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2018
abstract
The development of bio-inspired event-driven neuromorphic Dynamic Vision Sensors (DVS) provides a revolutionary way of capturing visual scenes by generating flows of events representing real-time visual information. Each pixel in a DVS operates autonomously and sends out an event (spike) whenever it senses a change of light greater than a preset threshold. Therefore, the DVS generates a continuous flow of events with a high temporal resolution (sub-microsecond) representing reality dynamically, without frames. Spiking Neural Networks (SNNs) process flows of events using different neuronal and synaptic models, performing tasks like object tracking or shape recognition.
Bulk-based DC offset calibration for Low-power Memristor Array Read-Out System
C. Mohan, L.A. Camuñas-Mesa, E. Vianello, L. Perniola, C. Reita, J.M. de la Rosa, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2017
abstract
Memristors in neuromorphic circuits typically need to drive currents of many mA because their Low Resistance State (LRS) is in the order of a few kΩ and many devices need to be activated simultaneously which results in high power consumptions. Reducing read-out pulses amplitudes below the typical 0.1V is not trivial, as offset voltages of read-out circuits start to affect the results. This paper presents a three-stage cascaded calibration to compensate for the resting offset voltage of crossbar lines generated in the amplifiers driving memristive devices in memristor array read-out systems. The proposed calibration technique is based on adjusting the bulk voltage of the input differential pairs by means of a switchable cascade of resistor ladders. As a result, the calibrated offset voltage can be further reduced with the number of stages in the cascade, leading to a calibration voltage step below 0.1mV -only limited in practice by mismatch and electrical noise. The circuit has been designed in 130nm CMOS technology, and its operation has been verified with oxide-based resistive memory (OxRAM) devices operated in binary mode to implement synapses in neuromorphic circuits. Layout-extracted simulations considering PVT variations are considered to validate the presented calibration technique.
Passive localization and detection of quadcopter UAVs by using dynamic vision sensor
S. Hoseini, G. Orchard, A. Yousefzadeh, B. Deverakonda, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Iranian Joint Congress on Fuzzy and Intelligent Systems, Conference on Fuzzy Systems and Conference on Intelligent Systems CFIS 2017
abstract
We present a new passive and low power localization method for quadcopter UAVs (Unmanned aerial vehicles) by using dynamic vision sensors. This method works by detecting the speed of rotation of propellers that is normally higher than the speed of movement of other objects in the background. Dynamic vision sensors are fast and power efficient. We have presented the algorithm along with the results of implementation.
Live Demonstration: Multiplexing AER Asynchronous Channels over LVDS Links with Flow-Control and Clock-Correction for Scalable Neuromorphic Systems
A. Yousefzadeh, M. Jabłoński, T. Iakymchuk, A. Linares-Barranco, A. Rosado, A. Plana, T. Serrano-Gotarredona, S. Furber and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2017
abstract
We propose demonstration of a serial link for fast asynchronous communication in massively parallel platforms connected to DVS for real-time implementation of bio-inspired vision processing and spiking neural networks.
Live Demonstration: Hardware Implementation of Convolutional STDP for On-line Visual Feature Learning
A. Yousefzadeh, T Masquelier, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2017
abstract
This is a proposal for live demonstration of a hardware that can learn visual feature online and in real-time during presentation of an object. Input Spikes are coming from a bio-inspired silicon retina or Dynamic Vision Sensor (DVS) and will be processed in a Spiking Convolutional Neural Network (SCNN) that is equipped with Synaptic Time Dependent Plasticity (STDP) learning rule and has been implemented in FPGA.
Hardware Implementation of Convolutional STDP for On-line Visual Feature Learning
A. Yousefzadeh, T Masquelier, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2017
abstract
We present a highly hardware friendly STDP (Spike Timing Dependent Plasticity) learning rule for training Spiking Convolutional Cores in Unsupervised mode and training Fully Connected Classifiers in Supervised Mode. Examples are given for a 2-layer Spiking Neural System which learns in real time features from visual scenes obtained with spiking DVS (Dynamic Vision Sensor) Cameras.
Multiplexing AER Asynchronous Channels over LVDS Links with Flow-Control and Clock-Correction for Scalable Neuromorphic Systems
A. Yousefzadeh, M. Jabłoński, T. Iakymchuk, A. Linares-Barranco, A. Rosado, A. Plana, T. Serrano-Gotarredona, S. Furber and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2017
abstract
Address-Event-Representation (AER) is a widely extended asynchronous technique for interchanging "neural spikes" among different hardware elements in Neuromorphic Systems. Conventional AER links use parallel physical wires together with a pair of handshaking signals (Request and Acknowledge). Here we present a fully serial implementation using bidirectional SATA connectors with a pair of LVDS (low voltage differential signaling) wires for each direction. The proposed implementation can multiplex a number of conventional parallel AER links per LVDS physical connection. It uses flow control, clock correction, and byte alignment techniques to transmit 32-bit address events reliably over multiplexed serial connections. The setup has been tested using commercial Spartan6 FPGAs reaching a maximum event transmission speed of 75Meps (Mega Events per second) for 32-bit events at 3.0Gbps line data rate.
On the Use of Offset Calibration Techniques for Low-Power Memristor Arrays Read-Out
C. Mohan, T. Serrano-Gotarredona, J.M. de la Rosa and B. Linares-Barranco
Conference · International Conference on Memristive Materials, Devices & Systems MEMRISYS 2017
abstract
Neuromorphic RRAM circuits need typically to drive currents of many mA because the low resistance state is in the order of a few kΩ and many devices need to be activated simultaneously, thereby resulting in high power consumptions. Reducing read-out pulses amplitudes below the typical 0.1V is not trivial, as offset voltages of read-out circuits start to affect the results. This paper presents a calibration circuit to compensate for the resting offset voltage of crossbar lines generated in the amplifiers driving memristive devices in memristor array readout systems. The proposed calibration technique is based on adjusting the bulk voltage of the input differential pairs by means of a switchable cascade of resistor ladders. As a result, the calibrated offset voltage can be further reduced with the number of stages in the cascade, leading to a calibration voltage step below 0.1mV -only limited in practice by mismatch and electrical noise. The circuit has been designed in a 130-nm CMOS technology, and its operation has been verified with oxide-based resistive memory (OxRAM) devices operated in binary mode to implement synapses in neuromorphic circuits. Layout-extracted simulations considering PVT variations are shown to validate the presented calibration technique.
EU COST action IC1401 - Pushing the frontiers of memristive devices to systems
D. Biolek, S. Carrara, E. Chicca, F. Corinto, J. Georgiou, B. Linares-Barranco, T. Prodromakis, S. Spiga and R. Tetzlaff
Conference · Mediterranean Electrotechnical Conference MELECON 2016
abstract
European Union COST Actions provide the opportunity for researchers who are geographically dispersed to work together towards an ambitious, multidisciplinary goal, whilst learning from each other and avoiding effort duplication. This paper gives a brief overview of work done by, but not limited to, members of EU COST Action IC1401. The presented work summary is organized around the four workgroups that tackle devices, circuit theory, circuit implementations, bioinspired and sensory systems.
High-speed serial interfaces for event-driven neuromorphic systems
M. Jablonski, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Event-Based Control, Communication and Signal Processing EBCCSP 2015
abstract
Neuromorphic Engineering is the discipline of building sensory processing artificial systems inspired in the neural processing found in living beings. Biological neural brains show massive connectivity among neurons, which is not realistic to mimic using wires within silicon chips or between chips. Address-Event-Representation is a technology widely used among neuromorphic engineers to emulate such massive interconnectivity by time-multiplexing fast digital channels by transmitting 'Address Events' between neurons that mimic the neural spikes transmitted in biology. Here we show on-going progress on bitserial SATA AER inter-FPGA communications for multi-Tile scalable neuromorphic systems.
Fast Pipeline 128×128 pixel spiking convolution core for event-driven vision processing in FPGAs
A. Yousefzadeh, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Event-Based Control, Communication and Signal Processing EBCCSP 2015
abstract
This paper describes a digital implementation of a parallel and pipelined spiking convolutional neural network (S-ConvNet) core for processing spikes in an event-driven system. Event-driven vision systems use typically as sensor some bio-inspired spiking device, such as the popular Dynamic Vision Sensor (DVS). DVS cameras generate spikes related to changes in light intensity. In this paper we present a 2D convolution event-driven processing core with 128×128 pixels. S-ConvNet is an Event-Driven processing method to extract event features from an input event flow. The nature of spiking systems is highly parallel, in general. Therefore, S-ConvNet processors can benefit from the parallelism offered by Field Programmable Gate Arrays (FPGAs) to accelerate the operation. Using 3 stages of pipeline and a parallel structure, results in updating the state of a 128 neuron row in just 12ns. This improves with respect to previously reported approaches.
ConvNets Experiments on SpiNNaker
T. Serrano-Gotarredona, B. Linares-Barranco, F. Galluppi, L. Plana and S. Furber
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2015
abstract
The SpiNNaker Hardware platform allows emulating generic neural network topologies, where each neuron-to-neuron connection is defined by an independent synaptic weight. Consequently, weight storage requires an important amount of memory in the case of generic neural network topologies. This is solved in SpiNNaker by encapsulating with each SpiNNaker chip (which includes 18 ARM cores) a 128MB DRAM chip within the same package. However, ConvNets (Convolutional Neural Network) posses "weight sharing" property, so that many neuron-to-neuron connections share the same weight value. Therefore, a very reduced amount of memory is required to define all synaptic weights, which can be stored on local SRAM DTCM (data-tightly-coupled-memory) at each ARM core. This way, DRAM can be used extensively to store traffic data for off-line analyses. We show an implementation of a 5-layer ConvNet for symbol recognition. Symbols are obtained with a DVS camera. Neurons in the ConvNet operate in an event-driven fashion, and synapses operate instantly. With this approach it was possible to allocate up to 2048 neurons per ARM core, or equivalently 32k neurons per SpiNNaker chip.
Event-driven sensing and processing for high-speed robotic vision
L.A. Camunas-Mesa, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE Biomedical Circuits and Systems Conference BioCAS 2014
abstract
pdf
We present here an overview of a new vision paradigm where sensors and processors use visual information not represented by sequences of frames. Event-driven vision is inherently frame-free, as happens in biological systems. We use an event-driven sensor chip (called Dynamic Vision Sensor or DVS) together with event-driven convolution module arrays implemented on high-end FPGAs. Experimental results demonstrate the application of this paradigm to implement Gabor filters and 3D stereo reconstruction systems. This architecture can be applied to real systems which need efficient and high-speed visual perception, like vehicle automatic driving, robotic applications in non-structured environments, or intelligent surveillance in security systems.
Live demonstration: Event-driven sensing and processing for high-speed robotic vision
L.A. Camunas-Mesa, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE Biomedical Circuits and Systems Conference BioCAS 2014
abstract
pdf
Fig. 1(a) shows the demo setup. Two DVS boards send events out through parallel buses to a merger board. This board merges all the event flow in one single AER bus, and sends it to a custom-made convolutional board, where a 2D grid array of convolution modules is implemented within a Spartan6 FPGA, as represented in Fig. 1(b) and (c). A USBAERmini2 board is used to timestamp the events coming out of the convolutional board and send them to a computer through a high-speed USB2.0 port. Finally, the output events are represented in the computer in real time using jAER software.
An AER handshake-less modular infrastructure PCB with x8 2.5Gbps LVDS serial links
T. Iakymchuk, A. Rosado, T. Serrano-Gotarredona, B. Linares-Barranco, A. Jimenez-Fernandez, A. Linares-Barranco and G. Jimenez-Moreno
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2014
abstract
Nowadays spike-based brain processing emulation is taking off. Several EU and others worldwide projects are demonstrating this, like SpiNNaker, BrainScaleS, FACETS, or NeuroGrid. The larger the brain process emulation on silicon is, the higher the communication performance of the hosting platforms has to be. Many times the bottleneck of these system implementations is not on the performance inside a chip or a board, but in the communication between boards. This paper describes a novel modular Address-Event-Representation (AER) FPGA-based (Spartan6) infrastructure PCB (the AER-Node board) with 2.5Gbps LVDS high speed serial links over SATA cables that offers a peak performance of 32-bit 62.5Meps (Mega events per second) on board-to-board communications. The board allows back compatibility with parallel AER devices supporting up to x2 28-bit parallel data with asynchronous handshake. These boards also allow modular expansion functionality through several daughter boards. The paper is focused on describing in detail the LVDS serial interface and presenting its performance.
Enhanced event-based stereo vision with Gabor filters
L.A. Camuñas-Mesa, T. Serrano-Gotarredona, S.H. Ieng, R. Benosman and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2014
abstract
The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, applying the matching algorithm to the events generated by the Gabor filters and not to those produced by the DVS. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.
Event-driven stereo vision with orientation filters
L.A. Camuñas-Mesa, T. Serrano-Gotarredona, B. Linares-Barranco, S. Ieng and R. Benosman
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2014
abstract
The recently developed Dynamic Vision Sensors (DVS) sense dynamic visual information asynchronously and code it into trains of events with sub-micro second temporal resolution. This high temporal precision makes the output of these sensors especially suited for dynamic 3D visual reconstruction, by matching corresponding events generated by two different sensors in a stereo setup. This paper explores the use of Gabor filters to extract information about the orientation of the object edges that produce the events, applying the matching algorithm to the events generated by the Gabor filters and not to those produced by the DVS. This strategy provides more reliably matched pairs of events, improving the final 3D reconstruction.
Improved Contrast Sensitivity DVS and its Application to Event-Driven Stereo Vision
T. Serrano-Gotarredona, J. Park, A. Linares-Barranco, A. Jiménez, R. Benosman and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2013
abstract
This paper presents a new DVS sensor with one order of magnitude improved contrast sensitivity over previous reported DVSs. This sensor has been applied to a bio-inspired event-based binocular system that performs 3D event-driven reconstruction of a scene. Events from two DVS sensors are matched by using precise timing information of their ocurrence. To improve matching reliability, satisfaction of epipolar geometry constraint is required, and simultaneously available information on the orientation is used as an additional matching constraint.
Frame-free event-based vision sensors and processors: building smart modular mixed-signal brain-like architectures
T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Analog VLSI Circuits, 2012
abstract
Abstract not available
Design of adaptive nano/CMOS neural architectures
T. Serrano Gotarredona and B. Linares Barranco
Conference · IEEE International Conference on Electronics Circuits and Systems ICECS 2012
abstract
pdf
Memristive devices are a promising technology to implement dense learning synapse arrays emulating the high memory capacity and connectivity of biological brains. Recently, the implementation of STDP learning in memristive devices connected to spiking neurons have been demonstrated as well as the dependency of the form of the learning rule on the shape of the applied spike. In this paper, we propose a fully CMOS integrate-and-fire neuron generating a precisely shaped spike that can be tuned through programmable biases. The implementation of STDP learning is demonstrated through electrical simulations of a 4×4 array of memristors connected to 4 spiking neurons.
A real-time event-driven neuromorphic system for goal-directed attentional selection
F. Galluppi, K. Brohan, S. Davidson, T. Serrano-Gotarredona, J.A. Pérez-Carrasco, B. Linares-Barranco and S. Furber
Conference · International Conference on Neural Information Processing ICONIP2012
abstract
pdf
Computation with spiking neurons takes advantage of the abstraction of action potentials into streams of stereotypical events, which encode information through their timing. This approach both reduces power consumption and alleviates communication bottlenecks. A number of such spiking custom mixed-signal address event representation (AER) chips have been developed in recent years. In this paper, we present i) a flexible event-driven platform consisting of the integration of a visual AER sensor and the SpiNNaker system, a programmable massively parallel digital architecture oriented to the simulation of spiking neural networks; ii) the implementation of a neural network for feature-based attentional selection on this platform.
Sistema de reconocimiento de caracteres de alta velocidad basado en eventos
J.A. Pérez-Carrasco, B. Acha, C. Serrano, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · XXVI Edición de URSI 2011
abstract
pdf
Spike-based processing technology is capable of very high speed throughput, as it does not rely on sensing and processing sequences of frames. Besides, it allows building complex and hierarchically structured cortical-like layers for sophisticated processing. In this paper we summarize the fundamental properties of this sensing and processing technology applied to artificial vision systems and the AER (Address Event Representation) protocol used in hardware spiking systems. Finally a four-layer system is described for character recognition. The system is slightly based on the Fukushima's Neocognitron. Realistic simulations using figures of already existing AER devices are provided, which show recognition delays under 10 μs.
Red neuronal convolucional rápida sin fotogramas para reconocimiento de dígitos
J.A. Pérez-Carrasco, C. Serrano, B. Acha, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · XXVI Edición de URSI 2011
abstract
pdf
In this paper a bio-inspired six-layer convolutional network (ConvNet) non-frame based for digit recognition is shown. The system has been trained with the backpropagation algorithm using 32x32 images from the MNIST database. The system can be implemented with already physically available
spike-based electronic devices. 10000 images have been coded into events separated 50ns to test the non-frame based ConvNet system. The simulation results have been obtained using actual performance figures for existing AER (Address Event Representation) hardware components. We provide simulation
results of the system showing recognition delays of a few microseconds from stimulus onset with a recognition rate of 93%. The complete system consists of 30 convolution modules.
Confession session: Learning from others mistakes
P. Abshire, A. Bermak, R. Berner, G. Cauwenberghs, S. Chen, J.B. Christen, T. Constandinou, E. Culurciello, M. Dandin, T. Datta, T. Delbruck, P. Dudek, A. Eftekhar, R. Etienne-Cummings, G. Indiveri, M.K. Law, B. Linares-Barranco, J. Tapson, W. Tang and Y. Zhai
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2011
abstract
People rarely put in their papers the things that didn¿t work, the mistakes they made, and how they found out what went wrong. Such confessions can help others learn how to avoid similar mistakes. Twenty-six confessions were collected to form the bulk of this paper. Themes that arise are errors that result from not understanding the limitations of simulation tools in modeling physical reality, chip verification errors that result from lack of clear communication between designers, and projects that are considered in their own isolated environment of technical challenges rather than the broader context of their environment or application.
A Bioinspired 128x128 Pixel Dynamic-Vision-Sensor
T. Serrano-Gotarredona, J.A. Leñero-Bardallo and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2011
abstract
pdf
This paper presents a 128x128 dynamic vision sensor. Each pixel detects temporal changes in the local illumination. A minimum illumination temporal contrast of 10% can be detected. A compact preamplification stage has been introduced that allows to improve the minimum detectable contrast over previous designs, while at the same time reducing the pixel area by 1/3. The pixel responds to illumination changes in less than 3.6μs. The ability of the sensor to capture very fast moving objects has been verified experimentally. A frame-based sensor capable to achieve this, would require at least 100K frames per second.
Visual AER-based processing with convolutions for a parallel supercomputer
R.J. Montero-González, A. Morgado-Estévez, F. Pérez-Peña, A. Linares-Barranco, A. Jiménez-Fernández, B. Linares-Barranco and J.A. Pérez-Carrasco
Conference · International Conference on Signal Processing and Multimedia Applications SIGMAP 2011
abstract
This paper is based on the simulation of a convolution model for multimedia applications using the neuroinspired Address-Event-Representation (AER) philosophy. AER is a communication mechanism between chips gathering thousands of spiking neurons. These spiking neurons are able to process the visual information in a frame-free style like the human brain do. All the spiking neurons are working in parallel and each of them implement an operation when an input stimulus is received. The result of this operation could be, or not, to produce an output event. There exist AER retinas and other sensors, AER processors (convolvers, WTA filters), learning chips and robot actuators. In this paper we present the implementation of an AER convolution processor for the supercomputer CRS (cluster research support) of the University of Cadiz (UCA). This research involves a test cases design in which the optimal parameters are set to run the AER convolution in parallel processors. These cases consist on running the convolution taking an image divided in different number of parts, applying to each part a Sobel filter for edge detection, and based on the AER-TOOL simulator. Runtimes are compared for all cases and the optimal configuration of the system is discussed. In general, CRS obtain better performances when the image is subdivided than for the whole image processing.
Performance Study of Software AER-Based Convolutions on a Parallel Supercomputer
R.J. Montero-González, A. Morgado-Estévez, A. Linares-Barranco, B. Linares-Barranco, F. Pérez-Peña, J.A. Pérez-Carrasco and A. Jiménez-Fernández
Conference · International Work-Conference on Artificial Neural Networks IWANN 2011
abstract
This paper is based on the simulation of a convolution model for bio-inspired neuromorphic systems using the Address-Event-Representation (AER) philosophy and implemented in the supercomputer CRS of the University of Cadiz (UCA). In this work we improve the runtime of the simulation, by dividing an image into smaller parts before AER convolution and running each operation in a node of the cluster. This research involves a test cases design in which the optimal parameters are set to run the AER convolution in parallel processors. These cases consist on running the convolution taking an image divided in different number of parts, applying to each part a Sobel filter for edge detection, and based on the AER-TOOL simulator. Execution times are compared for all cases and the optimal configuration of the system is discussed. In general, CRS obtain better performances when the image is divided than for the whole image. © 2011 Springer-Verlag.
Voltage mode driver for low power transmission of high speed serial AER links
C. Zamarreño-Ramos, T. Serrano-Gotarredona, B. Linares-Barranco, R. Kulkarni and J. Silva-Martínez
Conference · International Symposium on Circuits and Systems ISCAS 2011
abstract
pdf
This paper presents a voltage-mode high speed driver to transmit serial AER data in scalable multi-chip AER systems. To take advantage of the asynchronous nature of AER (Address Event Representation) streams, this implementation allows an energy efficient burst-mode operation. This is achieved by switching on/off the driver in data pauses to reduce static power consumption. Impedance matching is calibrated continuously to track temperature variations, obtaining an optimal performance without degrading the data rate. Power management techniques for switching drivers are discussed and an internally compensated high speed regulator is presented. The system has been designed in a 0.35 mu m CMOS technology to transmit data rates up to 500Mbps using Manchester enconding. Layout extracted simulation results are presented, which include all interconnect parasitics. Estimated peak rate is 15Meps for 32 bit events. Simulated power consumption of transmitter and receiver at peak rate is 33.2mW, while below 100 Keps is 1.3mW.
The EKV/ACM compact models for mismatch modeling down to 90nm and for new emergent non-CMOS nanotechnology FETs
T. Serrano-Gotarredona and B. Linares-Barranco
Conference · MOS-AK/GSA Workshop 2010
abstract
pdf
A calibrated spatial contrast AER vision sensor with adjustable contrast threshold
J.A Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Symposium on Circuits and Systems ISCAS 2010
abstract
Abstract not available
A 100dB dynamic range event-driven spatial contrast sensor with 100us response time and time-to-first-spike mode
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares Barranco
Conference · European Solid State Circuits Conference ESSCIRC 2010
abstract
pdf
Bio-inspired vision sensors have some inherent advantages over conventional sequential-still-image sensors. Some of them are high speed, low latency and reduced bandwidth and power consumption. In this paper, we present a new spatial contrast retina with signed output. Its output is zero if there is no contrast. The new sensor includes an optional Time-to-First-Spike mode (TFS) that combines the advantages of AER vision systems and frame-based ones. In TFS mode, times between consecutive frames can be adjusted dynamically by transmitting only relevant information. Both operation modes are ambient-light-independent, to first order. A 32x32 pixel prototype has been fabricated in 0.35um CMOS. Experimental results are provided. ©2010 IEEE.
Spike-based convolutional network for real-time processing
J.A. Pérez-Carrasco, C. Serrano, B. Acha, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Pattern Recognition ICPR 2010
abstract
In this paper we propose the first bio-inspired six-layer convolutional network (ConvNet) non-frame based that can be implemented with already physically available spike-based electronic devices. The system was designed to recognize people in three different positions: standing, lying or up-side-down. The inputs were spikes obtained with a motion retina chip. We provide simulation results showing recognition delays of 16 milliseconds from stimulus onset (time-to-first spike) with a recognition rate of 94%. The weight sharing property in ConvNets and the use of AER protocol allow a great reduction in the number of both trainable parameters and connections (only 748 trainable parameters and 123 connections in our AER system (out of 506998 connections that would be required in a frame-based implementation). © 2010 IEEE.
On neuromorphic spiking architectures for asynchronous STDP memristive systems
J.A. Pérez-Carrasco, C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2010
abstract
Neuromorphic circuits and systems techniques have great potential for exploiting novel nanotechnology devices, which suffer from great parametric spread and high defect rate. In this paper we explore some potential ways of building neural network systems for sophisticated pattern recognition tasks using memristors. We will focus on spiking signal coding because of its energy and information coding efficiency, and concentrate on Convolutional Neural Networks because of their good scaling behavior, both in terms of number of synapses and temporal processing delay. We propose asynchronous architectures that exploit memristive synapses with specially designed neurons that allow for arbitrary scalability as well as STDP learning. We present some behavioral simulation results for smail neural arrays using electrical circuit simulators, and system level spike processing results on human detection using a custom made event based simulator.
A signed spatial contrast event spike retina chip
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2010
abstract
pdf
Reported AER (Address Event Representation) contrast retinae perform a contrast computation based on the ratio between a pixel's local light intensity and a spatially weighted average of its neighbourhood. This results in compact circuits, but with the penalty of all pixels generating output signals even if they sense no contrast. In this paper we present a spatial contrast retina with bipolar output: contrast is computed as the relative normalized difference (not the ratio) between a pixel's local light and its weighted spatial average, normalized to average light. As a result contrast includes a sign, is ambient light independent, and the output will be zero if there is no contrast. Furthermore, an adjustable thresholding mechanism has been included, such that pixels remain silent until they sense an absolute contrast above the adjustable threshold. The pixel contrast computation circuit is based on Boahen's Biharmonic operator contrast circuit, which has been improved to include mismatch calibration and adaptive current based biasing. As a result, the contrast computation circuit shows much less mismatch, is almost insensitive to ambient light illumination, and biasing is much less critical than in the original voltage biasing scheme. The retina also includes an optional TFS (Time-to-First-Spike) integration mode. A full AER retina version has been fabricated and tested. In the present paper we provide preliminary experimental results.
On scalable spiking ConvNet hardware for cortex-like visual sensory processing systems
L. Camuñas-Mesa, J.A. Pérez-Carrasco, C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2010
abstract
This paper summarizes how Convolutional Neural Networks (ConvNets) can be implemented in hardware using Spiking neural network Address-Event-Representation (AER) technology, for sophisticated pattern and object recognition tasks operating at mili second delay throughputs. Although such hardware would required hundreds of individual convolutional modules and thus is presently not yet available, we discuss methods and technologies for implementing it in the near future. On the other hand, we provide precise behavioral simulations of large scale spiking AER convolutional hardware and evaluate its performance, by using performance figures of already available AER convolution chips fed with real sensory data obtained from physically avaliable AER motion retina chips. We provide simulation results of systems trained for people recognition, showing recognition delays of a few milliseconds from stimulus onset. ConvNets show good up scaling behavior and possibilities for being implemented efficiently with new nano scale hybrid CMOS/nonCMOS technologies.
Neocortical frame-free vision sensing and processing through scalable spiking Convet hardware
L. Camuñas-Mesa, J.A. Pérez-Carrasco, C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares Barranco
Conference · IEEE World Congress on Computational Intelligence WCCI 2010
abstract
This paper summarizes how Convolutional Neural Networks (ConvNets) can be implemented in hardware using Spiking neural network Address-Event-Representation (AER) technology, for sophisticated pattern and object recognition tasks operating at mili second delay throughputs. Although such hardware would require hundreds of individual convolutional modules and thus is presently not yet available, we discuss methods and technologies for implementing it in the near future. On the other hand, we provide precise behavioral simulations of large scale spiking AER convolutional hardware and evaluate its performance, by using performance figures of already available AER convolution chips fed with real sensory data obtained from physically available AER motion retina chips. We provide simulation results of systems trained for people recognition, showing recognition delays of a few miliseconds from stimulus onset. ConvNets show good up scaling behaviour and possibilities for being implemented efficiently with new nano scale hybrid CMOS/nonCMOS technologies.
Low power LVDS transceiver for AER links with burst mode operation capability
C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Design of Circuits and Integrated Systems Conference DCIS 2009
abstract
pdf
This paper presents the design and simulation of an LVDS transceiver intended to be used in serial AER links. Traditional implementations of LVDS serial interfaces require a continuous data flow between the transmitter and the receiver to keep the synchronization. However, the serial AER-LVDS interface proposed in [2] operates in a burst mode, having long times of silence without data transmission. This can be used to reduce the power consumption by switching off the LVDS circuitry during the pauses. Moreover, a fast recovery time after pauses must be achieved to not slow down the interface operation. The transceiver was designed in a 90 nm technology. Extensive simulations have been performed demonstrating a 1 Gbps data rate operation for all corners in post-layout simulations. Driver and receiver take up an area of 100x215 μm2 and 100x140 μm2 respectively.
improved AER convolution chip for vision processing with higher resolution and new functionalities
L.A. Camuñas-Mesa, A. Linares-Barranco, A. Acosta, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2009
abstract
pdf
We present a new neuromorphic fully digital convolution
microchip for Address Event Representation (AER)
spike-based processing system. This chip computes 2-D
convolutions with a programmable kernel in real time.
Previously, we designed and tested another convolution
chip with a size of 32 x 32 pixels [1] and, based on the
information obtained from this test, we have designed a
new chip with larger resolution (64 x 64 pixels),
improved behavior and new functionalities included.
This chip receives and generates data in AER format,
which is an asynchronous protocol, implementing the
convolution of the input images with a programmable
kernel. The most important new functionality included in
this chip is the multikernel capability, which allows us to
program several kernels (up to 32) so that each input
event will be processed with the corresponding kernel,
depending on the origin of the input event. The paper
describes the architecture of the chip, with special
emphasis to the new improvements.
Fast and compact simulation models for a variety of FET nano devices by the CMOS EKV equations
T. Serrano-Gotarredona, B. Linares-Barranco, G.Agnus, V. Derycke, J.-P. Bourgoin, F. Alibart, D. Vuillaume, J. Sohn, J. Bendall, M.E. Welland and Gamrat, C.
Conference · IEEE Conference on Nanotechnology IEEE-NANO 2009
abstract
In this paper we explore the possibility of using the equations of a well known compact model for CMOS transistors as a parameterized compact model for a variety of FET based nano-technology devices. This can turn out to be a practical preliminary solution for system level architectural researchers, who could simulate behaviourally large scale systems, while more physically based models become available for each new device. We have used a four parameter version of the EKV model equations and verified that fitting errors are similar to those when using them for standard CMOS FET transistors. The model has been used for fitting measured data from three types of FET nano-technology devices obeying different physics, for different fabrication steps, and under different programming conditions.
Exploiting memristance in adaptive asynchronous spiking neuromorphic nanotechnology systems
B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE Conference on Nanotechnology IEEE-NANO 2009
abstract
pdf
In this paper we show that spike-time-dependent-plasticity (STDP), a powerful learning paradigm for spiking neural systems, can be implemented using a crossbar memristive array combined with neurons that asynchronously generate spikes of a given shape. Such spikes need to be sent back through the neurons input terminal. The shape of the spikes turns out to be very similar to the neural spikes observed in biology for real neurons. The STDP learning function obtained by combining such neurons with memristors is exactly that of the STDP learning function obtained from neurophysiological experiments on real synapses. Using this result, we propose memristive crossbar architectures capable of performing asynchronous STDP learning.
Exploiting memristance for implementing spike-time-dependent-plasticity in neuromorphic nanotechnology systems
B. Linares-Barranco and T. Serrano-Gotarredona
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2009
abstract
Abstract not avaliable
A spatial calibrated AER contrast retina with adjustable contrast threshold
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2009
abstract
pdf
Address Event Representation (AER) is an emergent technology for assembling modular multi-blocks bio-inspired sensory and processing systems. Visual sensors (retinae) are among the first AER modules to be reported since the introduction of the technology. Spatial contrast AER retinae are of special interest since they provide highly compressed data flow without reducing the relevant information required for performing recognition. Reported AER contrast retinae perform a contrast computation based on the ratio between a pixel's local light intensity and a spatially weighted average of its neighbourhood. This results in compact circuits, but with the penalty of all pixels generating output signals even if they sensed no contrast. In this paper we present a spatial contrast retina with bipolar output: contrast is computed as the relative difference between a pixel's local light and its weighted spatial average. As a result, contrast includes a sign and the output will be zero if there is no contrast. Furthermore, an adjustable thresholding mechanism has been included, such that pixels remain silent until they sense an absolute contrast above the adjustable threshold. The pixel contrast computation circuit is based on Boahen's Biharmonic operator contrast circuit, which has been improved to include mismatch calibration and adaptive current based biasing. As a result, the contrast computation circuit shows much less mismatch, is almost insensitive to ambient light illumination, and biasing is much less critical than in the original voltage biasing scheme. A full AER retina version has been fabricated. In the present paper we provide simulation and preliminary experimental results.
Advanced vision processing systems: spike-based simulation and processing
J.A. Pérez-Carrasco, C. Serrano-Gotarredona, B. Acha-Piñero, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Conference on Advanced Concepts for Intelligent Vision Systems ACIVS 2009
abstract
pdf
In this paper we briefly summarize the fundamental properties of spike events processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured neurocortical-like layers for sophisticated processing. The paper describes briefly cortex-like spike event vision processing principles, and the AER, (Address Event Representation) technique used in hardware spiking systems. In this paper we present a simulation AER, tool that we have developed entirely in Visual C++ 6.0. We have validated it using real AER, stimulus and comparing the outputs with real outputs obtained from AER-ba.sed devices. With this tool we can predict the eventual performance of AER-based systems, before the technology becomes mature enough to allow such large systems.
A mismatch calibrated bipolar spatial contrast AER retina with adjustable contrast threshold
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Symposium on Circuits and Systems ISCAS 2009
abstract
Address Event Representation (AER) is an emergent technology for assembling modular multi-blocks bio-inspired sensory and processing systems. Visual sensors (retinae) are among the first AER modules to be reported since the introduction of the technology. Spatial contrast AER retinae are of special interest since they provide highly compressed data flow without reducing the relevant information required for performing recognition. Reported AER contrast retinae perform a contrast computation based on the ratio between a pixel's local light intensity and a spatially weighted average of its neighbourhood. This resulted in compact circuits, but with the penalty of all pixels generating output signals even if they sensed no contrast. In this paper we present a spatial contrast retina with bipolar output: contrast is computed as the relative difference between a pixel's local light and its weighted spatial average. As a result, contrast includes a sign and the output will be zero if there is no contrast. Furthermore, an adjustable thresholding mechanism has been included, such that pixels remain silent until they sense an absolute contrast above the adjustable threshold. The pixel contrast computation circuit is based on Boahen's Biharmonic operator contrast circuit, which has been improved to include mismatch calibration and adaptive current based biasing. As a result, the contrast computation circuit shows much less mismatch, is almost insensitive to ambient light illumination, and biasing is much less critical than in the original voltage biasing scheme. A full AER retina version has been submitted for fabrication. In the present paper we provide simulation results.
OTA-C oscillator with low frequency variations for on-chip clock generation in serial LVDS-AER links
C. Zamarreño-Ramos, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2009
abstract
pdf
This paper presents the design and simulation of an OTA-C oscillator intended to be used as on-chip frequency reference. This reference will be part of the high speed clock generation circuit for Manchester serial LVDS-AER links. A Manchester LVDS receiver can adapt its operation in a limited range of frequencies, so the most important specification is the frequency stability over temperature and process variations. A novel design methodology is presented to design two oscillators in a 9nm technology using transistors with 2.5V supply voltage. Intensive simulations with temperature, process, supply voltage variations and mismatch effects were perfomed in order to analyze the validity of this approach, obtaining Delta f approximate to 7%.
Fully digital AER convolution chip for vision processing
L. Camuñas-Mesa, A. Acosta-Jiménez, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2008
abstract
We present a neuromorphic fully digital convolution microchip for Address Event Representation (AER) spike-based processing systems. This microchip computes 2-D convolutions with a programmable kernel in real time. It operates on a pixel array of size 32 x 32, and the kernel is programmable and can be of arbitrary shape and size up to 32 x 32 pixels. The chip receives and generates data in AER format, which is asynchronous and digital. The paper describes the architecture of the chip, the test setup, and experimental results obtained from a fabricated prototype. ©2008 IEEE.
Event based vision sensing and processing
J.A. Pérez-Carrasco, C. Serrano, B. Acha, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Conference on Image Processing ICIP 2008
abstract
In this paper we briefly summarize the fundamental properties of spike events processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper describes briefly cortex-like spike event vision processing principles, and the AER (Address Event Representation) techique used in hardware spiking systems. Then a texture-based image retrieval using the AER technique is proposed. Realistic behavioral simulations based on existing hardware characteristics, reveal that the application, although processing large kernel convolutions, is capable of performing recognition in less than I Orris.
LVDS interface for AER links with burst mode operation capability
C. Zamarreño-Ramos, R. Serrano-Gotarredona, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2008
abstract
pdf
This paper presents the design and simulation of a serial AER LVDS communication link. It converts data from classical AER parallel bus with a 4-phase handshaking protocol into a bit stream which is transmitted serially into a single LVDS wire. At the receiver side data from the LVDS cable are transformed back to a parallel AER bus and handshaking signals are also properly managed. The link has been designed in a 90 nms technology. Extensive simulations have been performed demonstrating that the link can operate at a speed of 1 Gbps for all the technology corners, exhibiting a power consumption of 27.8 mW for the transmitter and 12.3 mW for the receiver. In the simulation the transmission channel was modelled as a 50 cm cat5E UTP cable, connected to the AER chip through 5 cm PCB traces modelled as a coupled microstrip transmission line. The design has been completed up to the layout level and has been submitted for fabrication. The transmitter and the receiver take up an area of 311x148 mu m(2) and 300x148 mu m(2) respectively.
Compact calibration circuit for large neuromorphic arrays
J.A. Leñero-Bardallo, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · International Symposium on Circuits and Systems ISCAS 2008
abstract
Low current applications, like neuromorphic circuits, where operating currents can be as low as few nano amps or less, suffer from huge transistor mismatches, resulting in around or less than I-bit precision. Here we present a new calibration approach based on individually calibratable current sources made with MOS transistors of digitally adjustable length, which require only N unit transistors. The scheme includes a translinear circuit based tuning scheme, which allows to expand the operating range of the calibrated circuits with graceful precision degradation, over 4 decades of operating currents. Experimental results are provided for 5-bit resolution DACs operating at 20nA.
High-speed character recognition system based on a complex hierarchical AER architecture
J.A Pérez-Carrasco, T. Serrano-Gotarredona, C. Serrano-Gotarredona, B. Acha and B. Linares Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2008
abstract
In this paper we briefly summarize the fundamental properties of spikes processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper describes briefly cortex-like spiking vision processing principles, and the AER (Address Event Representation) technique used in hardware spiking systems. Afterwards an example application is described, which is a simplification of Fukushima's Neocognitron. Realistic behavioral simulations based on existing AER hardware characteristics, reveal that the simplified neocognitron, although it processes 52 large kernel convolutions, is capable of performing recognition in less than 10 mu s..
Image Processing Architecture Based on a Fully Digital Aer Convolution Chip
L.A. Camuñas-Mesa, A.J. Acosta-Jimenez, T. Serrano-Gotarredona, B. Linares-Barranco and R. Serrano Gotarredona
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2007
abstract
Abstract not avaliable
Spike events processing for vision systems
R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jiménez, A. Linares-Barranco, G. Jiménez-Moreno, A. Civit-Balcells and B. Linares-Barranco
Conference · International Symposium on Circuits and Systems ISCAS 2007
abstract
In this paper we briefly summarize the fundamental properties of spike events processing applied to artificial vision systems. This sensing and processing technology is capable of very high speed throughput, because it does not rely on sensing and processing sequences of frames, and because it allows for complex hierarchically structured cortical-like layers for sophisticated processing. The paper includes a few examples that have demonstrated the potential of this technology for highspeed vision processing, such as a multilayer event processing network of 5 sequential cortical-like layers, and a recognition system capable of discriminating propellers of different shape rotating at 5000 revolutions per second (300000 revolutions per minute).
A physical interpretation of the distance tenn in Pelgrom's mismatch model results in very efficient CAD
B. Linares-Barranco and T. Serrano-Gotarredona
Conference · International Symposium on Circuits and Systems ISCAS 2007
abstract
In 1989 Pelgrom et al. published a mismatch model for MOS transistors, where the standard quadratic deviation of the mismatch in a parameter between two identical transistors, is given by two independent terms: (1) a transistor size-dependent term and (2) an inter-transistor distance-dependent term. To include the distance term, some researchers have developed CAD tools based on the so called (sigma-Space Methodology, which result in very computationally expensive algorithms. Such algorithms become non-viable even for circuits with a reduced number of transistors. On the other hand, by understanding and interpreting correctly the physical origin of Pelgrom's model distance term, one can implement in a straight forward manner this mismatch contribution in a CAD tool. Furthermore, the computational cost results negligible and viable for any number of transistors.
An AER contrast retina with on-chip calibration
J. Costas-Santos, T. Serrano-Gotarredona, R. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2007
abstract
We present a contrast retina microchip that provides its output as an AER (Address Event Representation) stream. Contrast is computed as the ratio between pixel photocurrent and a local average between neighboring pixels obtained with a diffusive network. This current based computation produces a large mismatch between neighboring pixels, because the currents can be as low as a few pico amperes. Consequently, a compact calibration circuitry has been included to calibrate each pixel. The paper describes the design of the pixel with its contrast computation and calibration sections. Experimental results are provided for a prototype fabricated in a standard 0.35 mu m CMOS process.
A bio-inspired event-based real-time image processor
R. Serrano-Gotarredona, T. Serrano-Gotarredona, A.J. Acosta-Jiménez, B. Linares-Barranco and L.A. Camuñas-Mesa
Conference · IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics BioRob 2006
abstract
AER (Address Event Representation) is an emergent bio-inspired protocol intended to communicate chips containing many processing units, called them neurons or pixels. It exploits the advantages of communicating the activation state of a neuron as pulses, as done in the human brain. The information is sent out sorted beginning with the most relevant. This feature together with the parallel processing of the information allows for performing very fast image processing. In this paper, we explain how AER is suitable for real-time image processing and, as an example, we present results from some AER-based convolution chips which is able to perform convolutions in real time.
High-speed image processing with AER-based components
R. Serrano-Gotarredona, B. Linares-Barranco, T. Serrano-Gotarredona, A.J. Acosta-Jiménez, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, G. Jiménez-Moreno and A. Civit-Ballcels
Conference · International Symposium on Circuits and Systems ISCAS 2006
abstract
A high speed sample image processing application using AER-based components is presented. The setup objective is to distinguish between two propellers of different shape rotating at high speed (around 1000 revolutions /see) to show event-based systems capabilities in high speed applications. Event-based schemes allow the most relevant information to propagate faster through the system layers. So image processing is sped up because a rough result may be available when only a little part of the input has arrived. This setup is much faster than the conventional frame-based image processing systems because they would need to proccess more than 10kFrames/s to do the same task proposed here, whereas only few events are required with the event based technique.
An arbitrary kernel convolution AER-transceiver chip for real-time image filtering
R. Serrano-Gotarredona, T. Serrano-Gotarredona, A.J. Acosta-Jiménez and B. Linares-Barranco
Conference · International Symposium on Circuits and Systems ISCAS 2006
abstract
A new chip that performs real-time image convolutions with programmable kernels of arbitrary shape is presented. This is a first prototype of reduced size (16x16 pixels) to validate system level techniques. It has been fabricated in AMS-0.35 mu n, 2-poly, 3-metal technology. Chip inputs and outputs are coded using Address Event Representation (AER). This is an emergent neuromorphic interchip communication protocol that allows for real-time virtual massive connectivity between huge number of pixels located on different chips. Pixels generate 'events' according to their activity levels. More active pixels generate more events per unit time and access the interchip communication channel more frequently, whereas pixels with low activity consume less communication bandwidth. This allows to communicate more relevant information in a very short time. Specific PCI boards have been developed to feed images into the chip and to read images out of it.
Poisson AER generator: Inter-spike-intervals analysis
A. Linares-Barranco, D. Cascado, G. Jiménez, A. Civit, M. Oster and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2006
abstract
Address-Event-Representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for bio-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). To develop AER based systems for image processing it is very convenient to have available some kind of tool for generating AER streams from on-computer stored images. In this paper we present a hardware method for generating AER streams with Poisson statistics in real time from a sequence of images stored in a computer's memory. We quantify that the events generated follow a Poisson distribution using the Kolmogorov-Smirnov test. We have developed a USB-AER board, based on the Xilinx Spartan II FPGA and the Cygnal 8051 microcontroller, developed by our RTCAR group have been used for the analysis.
A Programmable Convolution Chip Prototype for Real-Time Image Filtering
R. Serrano-Gotarredona, M.T. Serrano-Gotarredona, A.J. Acosta-Jimenez, B. Linares-Barranco, C. Serrano-Gotarredona, et. al
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2005
abstract
Abstract not available
AER Building Blocks for Multi-Layer Multi-Chip Neuromorphic Vision Systems
R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, H. Kolle-Riis, T. Delbrück, S.C. Liu, S. Zahnd, A.M. Whatley, R. Douglas, P. Häfliger, G. Jimenez-Moreno, A. Civit, T. Serrano-Gotarredona, A. Acosta-Jiménez and B. Linares-Barranco
Conference · Neural Information Processing Systems Conference NIPS 2005
abstract
A 5-layer neuromorphic vision processor whose components
communicate spike events asychronously using the address-eventrepresentation
(AER) is demonstrated. The system includes a retina
chip, two convolution chips, a 2D winner-take-all chip, a delay line
chip, a learning classifier chip, and a set of PCBs for computer
interfacing and address space remappings. The components use a
mixture of analog and digital computation and will learn to classify
trajectories of a moving object. A complete experimental setup and
measurements results are shown.
On Fully Digital Address-Event-Representation Convolution Processing
L. Camuñas-Mesa, A.J. Acosta-Jimenez, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Design of Circuits and Integrated Systems DCIS 2005
abstract
Abstract not avaliable
On event generators for Address Event Representation transmitters
R. Serrano-Gotarredona, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Bioengineered and Bioinspired Systems II, 2005
abstract
Address Event Representation (AER)[1] is an emergent neuromorphic interchip communication protocol that allows for real-time virtual massive connectivity between huge number of neurons located on different chips. By exploiting high speed digital communication circuits (with nano-seconds timings), synaptic neural connections can be time multiplexed, while neural activity signals (with mili-seconds timings) are sampled at low frequencies. Also, neurons generate 'events' according to their activity levels. More active neurons generate more events per unit time, and access the interchip communication channel more frequently, while neurons with low activity consume less communication bandwidth. In a typical AER transmitter chip, there is an array of neurons that generate events. They send events to a peripheral circuitry (let's call it "AER Generator") that transforms those events to neurons coordinates (addresses) which are put sequentially on an interchip high speed digital bus. Several techniques for implementing AER generators have been published [11-[8]. We have analyzed and studied some of them and have detected some shortcomings in the circuits reported, which may render some false situations under some statistical conditions. The present paper proposes some improvements to overcome such situations. The improved "AER Generator" has been implemented in an AER transmitter system.
A mismatch characterization and simulation environment for weak-to-strong inversion CMOS transistors
J. Velarde-Ramírez, G. Vicente-Sánchez, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on VLSI Circuits and Systems II, 2005
abstract
Mismatch analysis and simulation is crucial for modem analog design with submicron technologies, where transistors tend to be biased in weak and moderate inversion regions because of the down shrinking of power supply voltage. For optimum analog design where speed, power consumption, area, noise, and accuracy need to be carefully traded off, it is crucial to have available a precise estimation of transistor mismatch in order to avoid overdesign and consequently sacrify unnecessarily speed, power consumption, and area. In this paper we will provide experimental mismatch measurements of different 0.35um CMOS technologies. Each technology has been characterized for a large number of transistor sizes (25-30), by sweeping different width and length values. A large number of transistor curves are measured ranging over different possible biasing conditions. A recent mismatch model will be used to fit the data, and extract electrical parameters. Some of those parameters will be used to adjust the measured mismatch. As a result, a set of standard deviations and correlation coefficients result for the statistical characterization of the mismatch responsible parameters. The resulting electrical parameters, and statistical mismatch parameters are then used in the Spectre simulator of Cadence design environment, to implement the mismatch models using the AHDL behavioral level Spectre description language. The paper shows good agreement between measured data, predicted data, and simulated data.
A digital pixel cell for address event representation image convolution processing
L. Camuñas-Mesa, A. Acosta-Jiménez, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Bioengineered and Bioinspired Systems II, 2005
abstract
Address Event Representation (AER) is an emergent neuromorphic interchip communication protocol that allows for real-time virtual massive connectivity between huge number of neurons located on different chips. By exploiting high speed digital communication circuits (with nano-seconds timings), synaptic neural connections can be time multiplexed, while neural activity signals (with mili-seconds timings) are sampled at low frequencies. Also, neurons generate 'events' according to their information levels. Neurons with more information (activity, derivative of activities, contrast, motion, edges,...) generate more events per unit time, and access the interchip communication channel more frequently, while neurons with low activity consume less communication bandwidth. AER technology has been used and reported for the implementation of vaRíous type of image sensors or retinae: luminance with local agc, contrast retinae, motion retinae,... Also, there has been a proposal for realizing programmable kernel image convolution chips. Such convolution chips would contain an array of pixels that perform weighted addition of events. Once a pixel has added sufficient event contributions to reach a fixed threshold, the pixel fires an event, which is then routed out of the chip for further processing. Such convolution chips have been proposed to be implemented using pulsed current mode mixed analog and digital circuit techniques. In this paper we present a fully digital pixel implementation to perform the weighted additions and fire the events. This way, for a given technology, there is a fully digital implementation reference against which compare the mixed signal implementations. We have designed, implemented and tested a fully digital AER convolution pixel. This pixel will be used to implement a full AER convolution chip for programmable kernel image convolution processing.
A calibration scheme for subthreshold current mode circuits
J.C. Santos, T. Serrano-Gotarredona and B. Linares-Barranco
Conference · Conference on Bioengineered and Bioinspired Systems II, 2005
abstract
Abstract not available
Inter-spike-intervals analysis of Poisson like hardware synthetic AER generation
A. Linares-Barranco, M. Oster, D. Cascado, G. Jiménez, A. Civit and B. Linares-Barranco
Conference · International Work-Conference on Artificial Neural Networks IWANN 2005
abstract
Address-Event-Representation (AER) is a communication protocol for transferring images between chips, originally developed for bio-inspired image processing systems. Such systems may consist of a complicated hierarchical structure with many chips that transmit images among them in real time, while performing some processing (for example, convolutions). In developing AER based systems it is very convenient to have available some kind of means of generating AER streams from on-computer stored images. In this paper we present a hardware method for generating AER streams in real time from a sequence of images stored in a computer's memory. The Kolmogorov-Smirnov test has been applied to quantify that this method follows a Poisson distribution of the spikes. A USB-AER board and a PCI-AER board, developed by our RTCAR group, have been used.
Hardware implementation of complex reaction-diffusion neural networks using log-domain techniques
T. Serrano-Gotarredona, R. Serrano-Gotarredona and B. Linares-Barranco
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2004
abstract
This paper describes a hardware implementation of a 2nd order reaction-diffusion system. This hardware is able to emulate complex spatio-temporal behaviors that appear in nature, such as, travelling waves, trigger waves, spiral waves and Turing Patterns. The hardware has been designed using log-domain techniques with MOS transistors operating in the subthreshold regime. In our implementation, the parameters of the reaction-diffusion equation are currents that can be adjusted several decades in order to reproduce different spatio-temporal behaviors. As a demonstrator, a linear array of 10 cells has been integrated in the AMS 0.35mum CMOS technology. We have experimentally reproduced three complex spatio-temporal phenomena: trigger and travelling wave propagation and the emulation of an oscillatory medium.
On leakage current temperature characterization using sub-pico-ampere circuit techniques
B. Linares-Barranco, T. Serrano-Gotarredona, R. Serrano-Gotarredona and L.A. Camuñas
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2004
abstract
Recently, a reliable circuit design technique for current mode signal processing down to femto-amperes was reported [1]. The technique involves logarithmic current splitters for obtaining on-chip sub-pA currents and a special saw-tooth oscillator for current monitoring, while using "source voltage shifting". This way, sub-pA currents can be characterized without driving them off-chip which would require expensive instrumentation with complicated low leakage setups. In this paper we report on characterization of temperature dependence of leakage currents, exploiting these techniques. Currents as low as 0.3fA have been characterized.
On mismatch properties of MOS and resistors calibrated ladder structures
B. Linares-Barranco, T. Serrano-Gotarredona, R. Serrano-Gotarredona and G. Vicente Sánchez
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2004
abstract
The mismatch behavior of MOS and resistor based calibrated ladder structures, used in arrays of DACs, is studied theoretically and experimentally. It is found that the calibrated DAC worst case output current standard deviation is approximately 1/3 that of its individual components. MOS experimental measurements illustrate the discussed mismatch behavior. Directions on how to design ladder DACs for a target precision are provided.
A precise CMOS mismatch model for analog design from weak to strong inversion
T. Serrano-Gotarredona, B. Linares-Barranco and J. Velarde-Ramírez
Conference · ieee International Symposium on Circuits and Systems ISCAS 2004
abstract
doi
A five parameter mismatch model continuos from weak to strong inversion is presented. The model is an extension of a previously reported one valid in the strong inversion region [1]. A mismatch characterization of NMOS and PMOS transistors for 30 different geometries has been done with this continuos model. The model is able to predict current mismatch with a mean relative error of 13.5% in the weak inversion region and 5% in strong inversion. This is verified for 12 different curves, sweeping V-G, V-DS and V-S. Since data is available for 30 different sizes, the mismatch model can be expressed as function of transistor width W and L, independently. The proposed model, with explicit W and L dependency has been implemented in the Spectre simulator. Simulations reveal that such precise modeling of mismatch (with explicit W and L dependency) can improve analog circuit performance without penalty on power and area consumption: just by splitting transistors into the optimum number of segments.
A precise 90 degrees quadrature OTA-C VCO between 50-130 MHz
B. Linares-Barranco, T. Serrano-Gotarredona, J. Ramos-Martos, J. Ceballos-Cáceres, J.M. Mora and A. Linares-Barranco
Conference · International Symposium on Circuits and Systems ISCAS 2004
abstract
We present a VLSI continuous time sinusoidal OTA-C quadrature oscillator fabricated in a standard double-poly 0.8mum CMOS process. The oscillator is tunable in the frequency range from 50-130 MHz. A symmetric topology assures that the two phases produced by the oscillator present an extremely low phase difference error (less than 2degrees over the whole frequency range). A novel current mode amplitude control scheme is developed that allows for very small amplitudes. Experimental results are provided.
A new charge-packet driven mismatch-calibrated integrate-and-fire neuron for processing positive and negative signals in AER based systems
B. Linares-Barranco, T. Serrano-Gotarredona, R. Serrano-Gotarredona and J. Costas-Santos
Conference · International Symposium on Circuits and Systems ISCAS 2004
abstract
We present the design and experimental measurements of an integrate-and-fire pixel for Address-Event-Representation (AER) transceiver chips such that (a) input events can be weighted according to a digital word, (b) this weight includes a sign bit, (c) the incoming event is acompanied by a sign bit, and (d) the pixel can be calibrated to compensate for mismatch in large arrays of these pixels. A prototype has been fabricated in the AMS 0.35mum CMOS process, whose experimental measurement results are provided.
CMOS transistor mismatch model valid from weak to strong inversion
T. Serrano-Gotarredona and B. Linares Barranco
Conference · European Solid-State Circuits Conference ESSCIRC 2003
abstract
A five parameter mismatch model continuos from weak to strong inversion is presented. The model is an extension of a previously reported one valid in the strong inversion region [1]. A mismatch characterization of NMOS and PMOS transistors for 30 different geometries has been done with this continuos model. The model is able to predict current mismatch with a mean relative error of 13.5% in the weak inversion region and 5% in strong inversion. This is verified for 12 different curves, sweeping V-G, V-DS and V-S.
A loss control feedback loop for VCO stable amplitude tuning of RF integrated filters
B. Linares-Barranco and T. Serrano-Gotarredona
Conference · IEEE International Symposium on Circuits and Systems ISCAS 2002
abstract
Recently Li and Tsividis introduced a novel Loss Control Feedback Loop for VCO Indirect Tuning of RF Integrated Filters, because conventional loss-control loops were not able to render stable amplitude control for high frequency applications. In this paper we show a slight modification to the classical control loop, rendering stable control. Results are validated through device level simulations.
Books
Plan de Renovación de las Metodologías Docentes. Asignaturas en la Red 2008-2009. Arquitectura de Computadores
D. Cagigas-Muñiz, A.A. Civit-Balcells, M.R. García-Robles, M.R. López-Torres, C.D. Luján-Martínez, J.L. Sevillano-Ramos, R. Senhadji-Navarro, B. Linares-Barranco, T. Serrano-Gotarredona, D. Cascado-Caballero, J.L. Guisado-Lizar, L. Miró-Amarante and A.F. Jiménez-Fernández
Book · 2009
abstract
Abstract not available
Book Chapters
Spiking Hough for Shape Recognition
P. Negri, T. Serrano-Gotarredona and B. Linares-Barranco
Book Chapter · Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pp 425-432, 2018
abstract
doi
The paper implements a spiking neural model methodology inspired on the Hough Transform. On-line event-driven spikes from Dynamic Vision Sensors are evaluated to characterize and recognize the shape of Poker signs. The multi-class system, referred as Spiking Hough, shows the good performance on the public POKER-DVS dataset.
Neuromorphic Systems
C. Bartolozzi, R. Benosman, K. Boahen, G. Cauwenberghs, T. Delbrück, G. Indiveri, S.-C. Liu, S. Furber, N. Imam, B. Linares-Barranco, T. Serrano-Gotarredona, K. Meier, C. Posch and M. Valle
Book Chapter · Wiley Encyclopedia of Electrical and Electronics Engineering, pp 1-22, 2016
abstract
doi
This article reviews a wide spectrum of state-of-the-art neuromorphic systems, ranging from its principles, sensory elements, and processing aspects to large-scale example systems and commercial outlook. It does not aim to provide a full coverage of present knowledge, but simply provide a comprehensive summary with many pointers to further readings.
Bias Generator Circuits
T. Delbrück and B. Linares-Barranco
Book Chapter · Event-Based Neuromorphic Systems, pp 261-283, 2015
abstract
doi
Neuromorphic chips often require a wide range of biasing currents which are independent of process and supply voltage, and which change with temperature appropriately to result in constant transconductance. These currents can span many decades, down to less than the transistor 'off-current'. This chapter describes how to design wide-dynamic range configurable bias current references. The output of each current reference is a gate voltage which produces a desired current. Bias currents are generated by a bootstrapped-mirror 'master bias' current reference that generates a master current, which is successively divided by a digitally-controlled current splitter to generate the desired reference currents. Nonidealities such as power supply sensitivity, matching, stability, and headroom are also discussed. Open source design kits simplify the job of including these circuits on new designs.
Silicon Retinas
T. Delbrück and B. Linares-Barranco
Book Chapter · Event-Based Neuromorphic Systems, pp 37-69, 2015
abstract
doi
This chapter introduces biological and silicon retinas, focusing on recently developed silicon retina vision sensors with an asynchronous address-event output. The first part of the chapter introduces biological retinas and four examples of Address-Event Representation (AER) retinas. The second part of the chapter discusses the details of some of the pixel designs and the specifications of these sensors. The discussion focuses on future goals for improvements.
Neuromorphic Sensors, Vision
B. Linares-Barranco and T. Serrano-Gotarredona
Book Chapter · Encyclopedia of Computational Neuroscience, pp 1-5, 2014
abstract
doi
Conventional video cameras are based capturing a sequence of still frames. Improving a camera means normally to increase the total number of pixels (resolution) and/or to increase the number of frames per second that can be captured, while reducing sensor area, power consumption, and possibly fabrication cost. These cameras just capture the light intensities of visual reality. If they are to be used in an artificial vision system (e.g., for robotics), then subsequent computing resources need to be allocated to analyze the sequence of captured frames and extract relevant information for decision making.
Spike-Timing-Dependent-Plasticity in Hybrid Memristive-CMOS Spiking Neuromorphic Systems
T. Serrano-Gotarredona and B. Linares-Barranco
Book Chapter · Memristors and Memristive Systems, pp 353-377, 2014
abstract
doi
In this chapter we present a very exciting overlap between emergent nanotechnology and neuroscience, which has been discovered by neuromorphic engineers. Specifically, we are linking one type of memristor nano technology devices to the biological synaptic update rule known as Spike-Time-Dependent-Plasticity found in real biological synapses. Understanding this link allows neuromorphic engineers to develop circuit architectures that use this type of memristors to artificially emulate parts of the visual cortex. The implementations result in fully asynchronous architectures with neurons sending their action potentials not only forwards but also backwards. One critical aspect is to use neurons that generate spikes of specific shapes. We will see how by changing the shapes of the neuron action potential spikes we can tune and manipulate the STDP learning rules for both excitatory and inhibitory synapses. We will see how neurons and memristors can be interconnected to achieve large-scale spiking learning systems that follow an STDP learning rule, and how hibrid memristor-CMOS chips can be assembled onto scalable architectures exploiting AER (Address-Event-Representation) technology. Finally, we will discuss limitations of currently available memristors. The results presented are based on behavioral simulations and do not take into account non-idealities of devices and interconnects. Our aim here is to simply present, in a tutorial manner, an initial framework for the possible development of fully asynchronous STDP learning neuromorphic architectures exploiting two-terminal memristive type devices.
Spike-Timing-Dependent-Plasticity with Memristors
T. Serrano-Gotarredona, T. Masquelier and B. Linares-Barranco
Book Chapter · Memristor Networks, pp 211-247, 2014
abstract
doi
Here we present a very exciting overlap between emergent nano technology and neuroscience, which has been discovered by neuromorphic engineers. Specifically, we are linking one type of memristor nano technology devices to the biological synaptic update rule known as Spike-Time-Dependent-Plasticity found in real biological synapses. Understanding this link allows neuromorphic engineers to develop circuit architectures that use this type of memristors to artificially emulate parts of the visual cortex. We focus on the type of memristors referred to as voltage or flux driven memristors and focus our discussions on behavioral macro models for such devices. The implementations result in fully asynchronous architectures with neurons sending their action potentials not only forwards but also backwards. One critical aspect is to use neurons that generate spikes of specific shapes. We will see how by changing the shapes of the neuron action potential spikes we can tune and manipulate the STDP learning rules for both excitatory and inhibitory synapses. We will see how neurons and memristors can be interconnected to achieve large scale spiking learning systems, that follow a type of multiplicative STDP learning rule. We will briefly extend the architectures to use three-terminal transistors with similar memristive behavior. We will illustrate how a V1 visual cortex layer can be assembled and how it is capable of learning to extract orientations from visual data coming from a real artificial CMOS spiking retina observing real life scenes. Finally, we will discuss limitations of currently available memristors. The results presented are based on behavioral simulations and do not take into account non-idealities of devices and interconnects. The aim here is to present, in a tutorial manner, an initial framework for the possible development of fully asynchronous STDP learning neuromorphic architectures exploiting two or three terminal memristive type devices. (A Supplemental Material compressed zip file containing all files used for the simulations can be downloaded from http://www.frontiersin.org/neuromorphic_engineering/10.3389/fnins.2011.00026/abstract.)
Log-domain circuit techniques for nonlinear neural networks with complex dynamics
T. Serrano-Gotarredona, R. Serrano-Gotarredona and B. Linares-Barranco
Book Chapter · Smart Adaptive Systems on Silicon, pp 229-251, 2004
abstract
doi
We have identified a second order reaction-diffusion differential equation able to reproduce through parameter setting different complex spatio-temporal behaviors. We have designed a log-domain hardware that implements the discrete space version of the selected reaction-diffusion equation [1]. The logarithmic compression of the state variables allows several decades of variation of these state variables within subthreshold operation of the MOS transistors. Furthermore, as all the equation parameters are implemented as currents, they can be adjusted several decades. As a demonstrator, we have designed a chip containing a linear array of ten second order dynamics coupled cells. Using this hardware, we have experimentally reproduced two complex spatio-temporal phenomena: the propagation of traveling waves and of trigger waves, as well as isolated oscillatory cells. Because the cells realize a second order dynamics, are coupled to their nearest neighbors only, and allow wide range of parameters programming, they can be called 'Second Order Cellular Neural (or Nonlinear) Networks' (CNNs)
Other publications
No results
