On the implementation of backpropagation on the Alex AVX-2 parallel system

Author

Abbas, Hazem M. ; Bayoumi, Mohamed M.

Author_Institution

Dept. of Electr. & Comput. Eng., Queen´´s Univ., Kingston, Ont., Canada

Volume

2

fYear

1997

fDate

9-12 Jun 1997

Firstpage

1009

Abstract

Training backpropagation (BP) networks is a time-consuming process especially on sequential machines. This has motivated the use of parallel architectures to decrease the processing time required for training. In this paper the implementation of the BP algorithm on the Alex AVX-2 MIMD machine is investigated. Due to the high communication time caused by sending and receiving network information and due to the overhead of the message passing process, the conventional use of block-BP is not appropriate for this particular machine. Increasing the processing load of the workers with respect to the communication load will definitely increase the speedup factor. Here, we propose a block-update learning method for BP which reduces the communication time and produces results similar to those obtained with parallel block-BP

Keywords

backpropagation; feedforward neural nets; parallel machines; Alex AVX-2 MIMD machine; Alex AVX-2 parallel system; backpropagation; backpropagation network training; block-update learning method; communication time; feedforward neural nets; message passing process overhead; parallel block-BP; sequential machines; Backpropagation algorithms; Encoding; Feedforward neural networks; Large-scale systems; Learning systems; Master-slave; Message passing; Network topology; Neural networks; Parallel architectures;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks,1997., International Conference on

Conference_Location

Houston, TX

Print_ISBN

0-7803-4122-8

Type

conf

DOI

10.1109/ICNN.1997.616165

Filename

616165