تركيب روش منظم سازي تُنُك و آسيب مغزي بهينه در كوچك سازي يك مدل يادگيري عميق

Combining a Regularization Method and the Optimal Brain Damage Method for Reducing a Deep Learning Model Size

ﯾﮑﯽ از ﭼﺎﻟﺶ ﻫﺎي ﺷﺒﮑﻪ ﻫﺎي ﻋﺼﺒﯽ ﭘﯿﭽﺸﯽ، ﺑﻪ ﻋﻨﻮان اﺑﺰار اﺻﻠﯽ ﯾﺎدﮔﯿﺮي ﻋﻤﯿﻖ، ﺣﺠﻢ زﯾﺎد ﺑﺮﺧﯽ از ﻣﺪل ﻫﺎي ﻣﺮﺑﻮﻃﻪ اﺳﺖ. ﯾـﮏ ﺷﺒﮑﻪ ي ﻋﺼﺒﯽ ﭘﯿﭽﺸﯽ ﺑﻪ ﻣﺜﺎﺑﻪ ﻣﺪﻟﯽ از ﻣﻐﺰ، ﻣﺘﺸﮑﻞ از ﻣﯿﻠﯿﻮن ﻫﺎ اﺗﺼﺎل اﺳﺖ. ﮐﺎﻫﺶ ﺣﺠﻢ اﯾـﻦ ﻣـﺪل ﻫﺎ از ﻃﺮﯾـﻖ ﺣـﺬف )ﻫـﺮس( اﺗﺼﺎﻻت اﺿﺎﻓﯽ ﻣﺪل اﻧﺠﺎم ﻣﯽ ﺷﻮد ﮐﻪ ﻫﻤﺎﻧﻨﺪ ﯾﮏ آﺳﯿﺐ ﻣﻐـﺰي اﺳـﺖ. دو روش ﻣﻨﻈﻢ ﺳـﺎزي ﺗُﻨُـﮏ و آﺳـﯿﺐ ﻣﻐـﺰي ﺑﻬﯿﻨـﻪ از ﺟﻤﻠـﻪ ﻣﺸﻬﻮرﺗﺮﯾﻦ ﺷﯿﻮه ﻫﺎي ﻫﺮس ﻣﺪل ﻫﺴﺘﻨﺪ. در اﯾﻦ ﻧﻮﺷﺘﺎر ﺑﺎ ﺗﺮﮐﯿـﺐ اﯾـﻦ دو ﺷـﯿﻮه ﻧﺘـﺎﯾﺞ ﺑﻬﺘـﺮي در ﮐـﺎﻫﺶ ﺣﺠـﻢ ﻣـﺪل ﺣﺎﺻـﻞ ﺷـﺪه اﺳﺖ. اﺑﺘﺪا ﺑﺎ اﺳﺘﻔﺎده از روش اﻧﺘﻘﺎل ﯾﺎدﮔﯿﺮي، ﯾﮏ ﻣﺪل ﺑﺰرگ ﺷﺒﮑﻪ ﻫﺎي ﻋﺼـﺒﯽ ﭘﯿﭽﺸـﯽ ﺑـﺮاي ﺷﻨﺎﺳـﺎﯾﯽ ﻃﺒﻘـﺎت ﻫـﺪف، آﻣـﻮزش داده ﺷﺪ؛ ﺳﭙﺲ ﺑﺎ روش ﻫﺎي ﻣﻨﻈﻢ ﺳﺎزي ﺗُﻨُﮏ و آﺳﯿﺐ ﻣﻐﺰي ﺑﻬﯿﻨﻪ ، اﺗﺼﺎﻻت اﺿـﺎﻓﯽ آن ﻫـﺮس ﺷـﺪﻧﺪ. ﻧﺘـﺎﯾﺞ آزﻣﺎﯾﺸـﺎت ﻧﺸـﺎن داده اﺳﺖ ﮐﻪ در ﺑﯿﺸﺘﺮ ﻣﺠﻤﻮﻋﻪ دادﮔﺎن ﻣﻮرد ﺑﺮرﺳﯽ، اﻋﻤﺎل ﺷﯿﻮه ي ﺗﺮﮐﯿﺒﯽ ﻣﻨﻈﻢ ﺳﺎزي ﺗُﻨُﮏ و آﺳﯿﺐ ﻣﻐـﺰي ﺑﻬﯿﻨـﻪ ﻧﺴـﺒﺖ ﺑـﻪ اﻋﻤـﺎل ﻫـﺮ ﯾﮏ از آﻧﻬﺎ ﺑﻪ ﺻﻮرت ﺟﺪاﮔﺎﻧﻪ ﮐﺎراﺗﺮ اﺳﺖ. ﺑﺮاي ﯾﮑﯽ از ﻣﺠﻤﻮﻋﻪ دادﮔﺎن ﻣﻮرد ﺑﺮرﺳﯽ، ﺑﺎ روش ﺗﺮﮐﯿﺒﯽ ﭘﯿﺸﻨﻬﺎدي ﺗﻌـﺪاد اﺗﺼـﺎﻻت ﻣﺪل 76 درﺻﺪ ﮐﺎﻫﺶ داده ﺷﺪ، ﺑﺪون آﻧﮑﻪ ﮐﺎراﯾﯽ آن ﮐﺎﻫﺶ ﯾﺎﺑﺪ. اﯾﻦ ﮐﺎﻫﺶ ﺣﺠﻢ ﻣﺪل، زﻣﺎن ﭘﺮدازﺷﯽ را ﺑـﻪ ﯾـﮏ ﺳـﻮم ﺗﻘﻠﯿـﻞ داده اﺳـﺖ. ﮐــﺎﻫﺶ ﺣﺠـﻢ ﻣــﺪل ﻣﯽ ﺗﻮاﻧــﺪ اﻣﮑـﺎن اﺳــﺘﻔﺎده از آن در ﻣﺮورﮔﺮﻫـﺎ و ﺳــﺨﺖ اﻓﺰارﻫﺎي ﺿــﻌﯿﻒ ﺗﺮ و ﻫﻤـﻪ ﮔﯿﺮﺗﺮ را ﺗﺴــﻬﯿﻞ ﺳــﺎزد.

One of the challenges of convolutional neural networks (CNNs), as the main tool of deep learning, is the large volume of some relevant models. CNNs, inspired form the brain, have millions of connections. Reducing the volume of these models is done by removing (pruning) the redundant connections of the model. Optimal Brain Damage (OBD) and Sparse Regularization are among the famous methods in this field. In this study, a deep learning model has been trained and the effect of reducing connections with the aforementioned methods on its performance has been investigated. As the proposed approach, by combining the OBD and regularization methods its redundant connections were pruned. The resulting model is a smaller model, which has less memory and computational load than the original model, and at the same time its performance is not less than the original model. The experimental results show that the hybrid approach can be more efficient than each of the methods, in the most tested datasets. In one dataset , with the proposed method, the number of connections were reduced by 76%, without sacrificing the efficiency of the model. This reduction in model size has decreased the processing time by 66 percent. The smaller the software model, the more likely it is to be used on weaker hardware, found everywhere, and web applications.