ALGORITHMIC MAPPING OF NEURAL NETWORK MODELS ONTO PARALLEL SIMD-MACHINES

被引:24
作者
LIN, WM
PRASANNA, VK
PRZYTULA, KW
机构
[1] UNIV SO CALIF,DEPT ELECT ENGN SYST,LOS ANGELES,CA 90089
[2] HUGHES RES LABS,MALIBU,CA 90265
关键词
ALGORITHMIC MAPPING; EMBEDDING; LEARNING PHASE; NEURAL NETWORKS; PARTITIONED IMPLEMENTATION; RECALL PHASE;
D O I
10.1109/12.106224
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
The paper describes parallel implementations of neural network computations on fine grain mesh-connected SIMD machines. The implementations apply to connectionist networks of arbitrary topology in which recall and learning operations can be expressed in terms of matrix and vector computations. We present two mappings based on graph-theoretic approach. In the first one, we implement a neural network of n neurons and e synaptic connections on an N x N mesh of processors, where the number of processors is greater or equal to the number of data items, i.e., N2 greater-than-or-equal-to n + e. The mapping is illustrated on an example of a multilayer perceptron with back-propagation learning. The data routing for a single iteration of the recall phase requires 24(N-1) elemental shift operations, whereas the learning iteration takes 60(N-1) shifts. The second mapping addresses the partitioned case. Here the processor array is of size P x P, P2 less-than-or-equal-to n + e, with each processor having local memory of size O(K). The interprocessor routing is realized using Benes network embedding and results in execution time of O(N2/P log(PK)N) for an iteration of update or learning phase, assuming N = inverted right perpendicular square-root n + e inverted left perpendicular. Our method is simple and applicable to a wide range of commercial and experimental SIMD machines. However, some of the details of the second mapping are presented using Hughes SCAP computer as a target machine.
引用
收藏
页码:1390 / 1401
页数:12
相关论文
共 25 条
[1]  
BELLOCH G, 1987, 10TH P INT JOINT C A
[2]  
BENES VE, 1962, BELL SYST TECH J, V41, P117
[3]   A STUDY OF NON-BLOCKING SWITCHING NETWORKS [J].
CLOS, C .
BELL SYSTEM TECHNICAL JOURNAL, 1953, 32 (02) :406-424
[4]  
CUN YL, 1989, 1989 P IEEE C NEUR I
[5]   CONNECTIONIST ARCHITECTURES FOR ARTIFICIAL-INTELLIGENCE [J].
FAHLMAN, SE ;
HINTON, GE .
COMPUTER, 1987, 20 (01) :100-109
[6]  
GALE D, 1957, PAC J MATH, P1073
[7]  
HASTINGS HM, 1987, FRONTIERS MASSIVELY
[8]  
HILLIS D, 1985, CONNECTION MACHINE
[9]  
KUMAR VKP, 1985, PARALLEL ARCHITECTUR
[10]  
KUMAR VKP, 1990, P INT C APPL SPECIFI