Fabrication of the MHN
The MHN was fabricated on a printed circuit board (PCB). The synapse consists of a memristor, a resistor, an inverter and a transmission gate. Positive or negative synaptic weight can be obtained by electronic switches. In the 3-bit MHN, a neuron consists of three synapses. A neuron was constructed with an operational amplifier to obtain the sum of the three inputs. The state of the neuron was stored on a capacitor. The fabrication procedure of the MHN is as follows: design of the schematic circuit of the MHN; design of the PCB layout of the MHN (see "Figure 1":http://www.nature.com/protocolexchange/system/uploads/3655/original/Figure_1.jpg?1432391113 ); assembly of the commercial chips, resistors and capacitors, onto the PCB; wire connection of the memristors with the PCB; and programming of the FPGA to control the MHN.
TIP: There are two possible circuit configurations for the synaptic connection, as shown in "Figure 2":http://www.nature.com/protocolexchange/system/uploads/3657/original/Figure_2.jpg?1432391117. In the first configuration (Figure 2(a)), the synaptic weight of the synapse corresponding to input Ni (i=1, 2, 3) is wi=± R /(Mi + R), where Mi is the resistance of the corresponding memristor. In this design, R should be of a high resistance at the level of 106 ohms in order to limit the current and avoid wrongly-switching of the memristor during circuit operation. Mi can only be adjusted in the range from several tens of ohms to several hundreds of kΩ due to the memristor inherent property. As R is much larger than Mi, wi would be insensitive to Mi and may always stay at ±1. Thus the variable range of the synaptic weight is very small. It is difficult for the MHN to detect such a small change in wi. In contrast, in the second configuration (Figure 2(b)), wi=± Mi/(Mi + R), thus the synaptic weight wi is much more sensitive to the change of Mi and can be varied in a larger range. Therefore, the circuit configuration shown in Figure 2(b) is used to implement the synapses.
Implementation of associative memory in the MHN
With a given pattern, the synaptic weight matrix elements in equation (1) (all equations in this protocol are placed in the "Supplementary document 1":http://www.nature.com/protocolexchange/system/uploads/3659/original/Supplementary_equation_file.pdf?1432386669) are set to w11= w22= w33=0, w12= w21, w13= w31, and w23= w32. The threshold vector T=(θ1 θ2 θ3) is set as θ1= θ2= θ3= θ. Here, wij (i, j=1, 2, 3) is in the form of equation (2). During the recalling process, the MHN is updated according to equation (3). In equation (3), t denotes the number of updating cycles; t=0 denotes no updating taking place and the corresponding state vector is the initial vector X(0); and the sign function is defined in equation (4). In one updating cycle, new states of the neurons are asynchronously updated from x1, x2 to x3 in three stages, which are defined as stages a, b and c, respectively.
Single associative memory
It is required that if the input is the target pattern itself, the final output must also stabilize at the target pattern. Thus, when “110” is stored into the MHN, according to equation (3) and Hopfield network rule for updating, equation (5) should be satisfied. With w11= w22= w33=0, w12= w21, w13= w31, and w23= w32 and θ1= θ2= θ3= θ, equation (5) is simplified as shown in equation (6). Equation (6) can be satisfied with the following setting: θ=-2/60, w12=4/60, w13=1/60, and w23=-4/60. Thus the synaptic weight matrix is given by equation (7), and the threshold vector is given by equation (8). The following initial states X(0) along with equations (7) and (8) are put into equation (3) to verify the convergence: “000”, “001”, “010”, “011”, “100”, “101”, “110”, and “111”. It is found that the MHN can eventually converge to the final state “110” from the above initial states.
Here, X(0)= (x(0)1 x(0)2 x(0)3)=(0 0 0) is taken as an example to demonstrate the recalling process. In stage a of the first updating cycle, we obtain equation (9) according to equation (3). And only x1 is updated in stage a of the first updating cycle, and thus we eventually obtain equation (10) in this stage; In stage b of the first updating cycle, equation (11) can be obtained according to equation (3). And only x2 is updated in stage b, so we eventually obtain equation (12) in this stage; In stage c of the first updating cycle, equation (13) can be obtained according to equation (3). And only x3 is updated in stage c, and thus we eventually obtain equation (14). From equations (9)-(14), we can see that “000” eventually converges to “110”. In a similar way, the MHN converges to “110” from any other initial states.
With the synaptic weight matrix shown in equation (7), the resistance matrix of the memristors determined from equation (2) can be set to the form of equation (15). Note that it is not necessary to adjust the resistances of the memristors exactly to the above values because the MHN has a tolerance to the resistance variation. An offline training scheme for setting the resistance of the memristors to a pre-determined value is implemented with a C Language program embedded in the Keithley 4200 semiconductor characterization system. With the actual resistance matrix M in equation (16), the actual synaptic weight matrix W was set to equation (17). With the above synaptic weight matrixe W and the threshold vector in equation (8), “110” can be correctly retrieved using the MHN fabricated on the PCB.
Similar to the requirement for single associative memory, when the two patterns “000” and “101”, are stored in the MHN, equations (18) and (19) should be satisfied. With w11= w22= w33= 0, w12= w21, w13= w31, and w23= w32 and θ1= θ2= θ3= θ, equations (18) and (19) are respectively simplified as equations (20) and (21). Equations (20) and (21) can be satisfied with the following setting: θ=6/60, w12=1/60, w13=8/60, and w23=4/60. Thus the synaptic weight matrix is given by equation (22), and the threshold vector is given by equation (23). Putting equations (22) and (23) and X(0) (X(0) can be varied from “000” to “111”) into equation (3), “000” and “101” can be stored into and retrieved from the MHN theoretically. For the synaptic weight matrix given in equation (22), the resistance matrix obtained from equation (2) can be set to the form of equation (24). The actual resistance matrix and the actual synaptic weight matrix of the fabricated MHN are given by equations (25) and (26), respectively: