딥 러닝의 블랙 박스 열어보기

Opening the black box of Deep Neural Networks via Information
- https://arxiv.org/pdf/1703.00810.pdf

지금까지 딥 러닝은 어떻게 동작하는지 이해할 수 없다고 믿어져왔다. 그러나, 천재들은 그 근본 원리를 이해하려고 노력하고 있지.

이 논문은 SGD가 무슨 역할을 하는지에 대한 해석으로, 이들은 SGD가 drift와 diffusion단계로 구분된다고 설명한다.

첫 drift 단계에서는 multidimensional space를 탐험하고, 수렴을 시작하면 diffusion 단계로 접어들어 (첨부 그림에서 점선 오른쪽에 해당) 굉장히 느려지기 시작하며 chaotic 한 단계로 들어가는데 이게 바로 네트워크가 압축하는 법을 배우는 과정이라는 것이다.



SGD의 무질서도는 local minima에 빠지지 않고 더 나은 방향으로 수렴 할 가능성을 제공 할 뿐만 아니라, diffusion 단계에서 매우 효과적인 internal representations(?)를 생성하는데 자기가 볼쩍에 다소 과한것으로 보이며, 자기들의 IB iterations with stochastic relaxation methods로 DNN 학습 방법을 개선할 수 있을거라고 얘기한다.

제프리 힌트 교수가 지대한 관심과 찬사를 보냈기에 많은 관심을 받고 있는 내용인데, 딥 러닝 대가들의 근본 원리와 최적화 이론에 대한 얘기라, 나는 훔쳐만 봤지 뭔소리인지는 정확히는 이해하기 어렵다.

33 comments:

  1. Thanks a lot for sharing
    Having good health is what most people out there wants but can not achieve. some people takes buy ibogaine online AND buy weed online to get it.

    ReplyDelete
  2. medical care is what many people lack in the interior areas of the world. people face a lot of health issues everyday without solution. buy psychedelics online, Buy weed online, Buy fantanyl patches online

    ReplyDelete
  3. Nice Information Your first-class knowledge of this great job can become a suitable foundation for these people. I did some research on the subject and found that almost everyone will agree with your blog.
    Cyber Security Course in Bangalore

    ReplyDelete
  4. Writing in style and getting good compliments on the article is hard enough, to be honest, but you did it so calmly and with such a great feeling and got the job done. This item is owned with style and I give it a nice compliment. Better!
    Cyber Security Training in Bangalore

    ReplyDelete
  5. https://www.magnificentincense.com/

    ReplyDelete
  6. k2incenseonlineheadshop Buy liquid incense cheap Buy liquid incense cheap For Sale At The Best Incense Online Shop

    ReplyDelete
  7. k2incenseonlineheadshop Buy liquid incense cheap Buy liquid incense cheap For Sale At The Best Incense Online Shop

    ReplyDelete
  8. https://bestmjstrainsonline.com/

    Buy liquid incense cheap | cheap herbal incense | buy k2 E liquid | k2 spray
    Description
    Buy liquid incense cheap . cheap herbal incense . buy k2 E liquid . buy k2 paper spray online . buy herbal incense online . buy k2 spice online

    Buy K2 Infused paper spray online, order K2 Spray online +1 (925) 526-5453

    https://bestmjstrainsonline.com/product-category/buy-liquid-incense-online/

    https://bestmjstrainsonline.com/product-category/infused-drugs-on-paper/

    https://bestmjstrainsonline.com/product/5-co2-cannabis-oil-cartridges/

    ReplyDelete
  9. https://syntheticworldwide.com/
    sales@syntheticworldwide.com

    Buy cheap liquid herbal incense at your best online shop

    ReplyDelete
  10. What an amazing blog you have here Cocaine for Sale thank you for sharing this real good content buy colombian cocaine online will like to also say we have an amazing blog too if you will love to take a look buy peruvian cocaine online thanks for your time to check on our blog. Today cocainehydrochloride is one of the world-leading buy cocaine online manufacturers in the USA . for you to order cocaine online , there is a variety of cocaine websites you can purchase and have it delivered Worldwide . And Yes, you can buy crack cocaine online illegal drugs on the Internet, and it's a lot safer .

    ReplyDelete
  11. We welcome everyone to have a look at our awesome blog about stories and events of
    buy Vyvanse online ,
    Also find it interesting to check out

    buy adderall XR online ,

    buy oxycodone online ,

    cocaine for sale ,

    buy Colombian cocaine online ,

    buy crack cocaine online ,

    buy Peruvian cocaine online ,

    buy fishscale cocaine online ,

    buy crack cocaine online ,

    buy ketamine online

    We appreciate you all for taking the time to look at our blog post topics on recent events and stories .

    ReplyDelete
  12. What a good blog you have here on how to buy Vyvanse online medication. There are many order Vyvanse online
    pharmacy websites that operate legally such as buy adderall XR online and offer convenience, privacy, and safeguards for purchasing buy oxycodone online medications. Also, these medicines may not have been stored properly . So always look out Buy Keifeitropin 140iu Online for details .

    ReplyDelete

  13. https://buydocumentation.com/

    Buy Driver’s License online, US driver license, drivers license renewal ca WhatsApp +1 (925) 526-5453


    Here at Buy Documentation, you can buy fake documents at a fraction of a cost. We keep our prices low to meet the needs of all our customers. We provide Best Quality Novelty Real and Fake IDs and Passport, Marriage Certificates and Drivers license online

    https://buydocumentation.com/canada-visa/

    https://buydocumentation.com/buy-real-drivers-license/

    https://buydocumentation.com/buy-real-id-cards/

    https://buydocumentation.com/social-security-number-ssn/

    ReplyDelete
  14. https://www.oobbg.com 우리카지노사이트
    https://www.oobbg.com/theking 더킹카지노
    https://www.oobbg.com/sands 샌즈카지노
    https://www.oobbg.com/first 퍼스트카지노
    https://www.oobbg.com/yes 예스카지노
    https://www.oobbg.com/coin 코인카지노
    https://www.oobbg.com/33 33카지노
    https://www.oobbg.com/world 월드카지노
    https://www.oobbg.com/merit 메리트카지노
    https://www.oobbg.com/gatsby 개츠비카지노

    ReplyDelete
  15. https://www.omgka.com 우리카지노사이트
    https://www.omgka.com/theking 더킹카지노
    https://www.omgka.com/sands 샌즈카지노
    https://www.omgka.com/first 퍼스트카지노
    https://www.omgka.com/yes 예스카지노
    https://www.omgka.com/super 슈퍼카지노
    https://www.omgka.com/gatsby 개츠비카지노
    https://www.omgka.com/33 33카지노
    https://www.omgka.com/world 월드카지노
    https://www.omgka.com/merit 메리트카지노

    ReplyDelete
  16. https://www.bbdd66.com 우리카지노사이트
    https://www.bbdd66.com/theking 더킹카지노
    https://www.bbdd66.com/sands 샌즈카지노
    https://www.bbdd66.com/first 퍼스트카지노
    https://www.bbdd66.com/yes 예스카지노
    https://www.bbdd66.com/super 슈퍼카지노
    https://www.bbdd66.com/gatsby 개츠비카지노
    https://www.bbdd66.com/33 33카지노
    https://www.bbdd66.com/world 월드카지노
    https://www.bbdd66.com/merit 메리트카지노

    ReplyDelete
  17. https://www.omgqq.com 우리카지노사이트
    https://www.omgqq.com/thekingcasino 더킹카지노
    https://www.omgqq.com/sandscasino 샌즈카지노
    https://www.omgqq.com/firstcasino 퍼스트카지노
    https://www.omgqq.com/yescasino 예스카지노
    https://www.omgqq.com/supercasino 슈퍼카지노
    https://www.omgqq.com/gatsbycasino 개츠비카지노
    https://www.omgqq.com/33casino 33카지노
    https://www.omgqq.com/worldcasino 월드카지노
    https://www.omgqq.com/merit 메리트카지노

    ReplyDelete
  18. https://www.omgab.com 우리카지노사이트
    https://www.omgab.com/theking 더킹카지노
    https://www.omgab.com/sands 샌즈카지노
    https://www.omgab.com/first 퍼스트카지노
    https://www.omgab.com/yes 예스카지노
    https://www.omgab.com/super 슈퍼카지노
    https://www.omgab.com/gatsby 개츠비카지노
    https://www.omgab.com/33 33카지노
    https://www.omgab.com/world 월드카지노
    https://www.omgab.com/merit 메리트카지노

    ReplyDelete

  19. herbal incense 2021,
    trusted herbal incense sites 2021,
    k2 liquid incense,
    order cheap herbal incense,
    cheap strong herbal incense,
    super strong herbal incense liquid,
    most potent herbal incense on the market,
    best herbal incense website 2021,
    k2 liquid spray on paper for sale,
    where to buy liquid herbal incense,
    buy herbal incense with debit card,
    buy herbal incense overnight shipping,
    strongest liquid incense,
    buy herbal incense online cash on delivery,
    liquid herbal incense 2020,
    k2 spray that get you high for sale,
    buy herbal net


    https://k2herbalspice.com/
    Call Us: (+1) 747-500-1520
    Email: sales@k2herbalspice.com



    herbal incense 2021,
    trusted herbal incense sites 2021,
    k2 liquid incense,
    order cheap herbal incense,
    cheap strong herbal incense,
    super strong herbal incense liquid,
    most potent herbal incense on the market,
    best herbal incense website 2021,
    k2 liquid spray on paper for sale,
    where to buy liquid herbal incense,
    buy herbal incense with debit card,
    buy herbal incense overnight shipping,
    strongest liquid incense,
    buy herbal incense online cash on delivery,
    liquid herbal incense 2020,
    k2 spray that get you high for sale,
    buy herbal net


    website: https://herbalincensespices.com/
    Call Us: +1 (312)-319-2341
    Email: sales@herbalincensespices.com


    We are located in USA AND UK. Buy High Quality Grade(%99.96) Mephedrone (2- methylamino 1-one) also known as 4-methylmethcathinone(4-MMC).We are one of the top suppliers in wholesale and retail of Benzodiazepine,synthetic cannabinoids, Blotters chemicals. Mephedrone and other plants research chemicals such as: Amphetamines Pentylone Mephedrone Flephedrone Bulytone (bk-MBDB) MDAI Analgesic Chemical CB1 and CB2 CP 47497 CP-55940 HU-210 HU-331 Ephedrine Hcl Powder JWH-018 / JWH-200,JWH-250 TFMPP 2C-E, 2C-I, 2C-P, 2C-C, 2C-T-2 DOC, DOI Bromo DragonFly TCB-2 Testosterone 5-Meo-DMT 4-Aco-DMT 4-Ho-MIPT 4-Meo-PCP Naphyrone HEROIN. Methylone (bk-MDMA) Ethylone , Oxycodone Flephedrone (4-FMC, 4-Fluoromethcathinone) Methedrone (BK-PMMA, Methoxyphedrine) 4-Fluoromethamphetamine (4-FMA) a-Pyrrolidinopropiophenone (a-PPP). MDPV (Methylenedioxypyrovalerone, MDPK) Testosterone JWH-073, 1-butyl-3-(1-naphthoyl)indole Hydrocodone Dimethocaine (Larocaine/DMC) Morphine JWH-018, 1-Pentyl-3-(1-naphthoyl)indole Herione 4-Fluoroamphetamine (4-FA, 4-FMP, or Flux) Ketamine Our quality is the best you can find around and we sell in small/large quantities with guaranteed discreet delivery in good time Shipping

    website: https://uslegitresearchchemicals.com/
    Call Us: +1 760-573-6020
    Email: info@uslegitresearchchemicals.com

    ReplyDelete
  20. https://chemicalmedshop.com/"Buy Pure Cocaine Online UK,Buy Duromine Weight Loss online,Duromine for sale, Buy Crack Cocaine Online, Buy Furanylfentanyl Online, Cocaine For Sale Online, Buy Pure Cocaine Powder Online, Crack Cocaine For Sale Online, Buy Pure Amphetamine Powder Online, chemicalmedshop.com Buy Furanylfentanyl Online, Buy Oxycodone Pills Online, Powdered Cocaine For Sale Online, Cocaine Hydrochloride For Sale Online, Buy White Mercedes Pills Online, Buy 4-Aco-Dmt Online,Ketamine For Sale Online, Cocaine For Sale Online, Buy Crack Cocaine Online, Buy White Heroin Online, Ketamine For Sale Online, Cocaine For Sale Online, Buy Crack Cocaine Online, Buy White Heroin Online, Buy MDMA Molly Online, Buy LSD Oline, Crystal Meth for sale, Buy LSD Oline, Crystal Meth for sale, Buy Oxycodone Pills Online, Buy Furanylfentanyl Online, Buy Mephedrone Online, Buy Bk-2C-B Research Chemical Online, Buy Pyrazolam Online, Buy Heroin Online, Cocaine Hydrochloride For Sale Online, Buy Pure Heroin Powder Online, Buy White Mercedes Pills Online, Buy Fentanyl Hcl Powder Online, Buy Jwh-250 Research Chemical Online, Mdma Powder Crystal For Sale Online, Buy Methiopropamine Crystals Online, Buy Yaba Pills Online, Buy Clonazolam Powder Online, Buy 4-Fma Research Chemical Online, Buy Methylone Online, Buy Oxycodone Pills Online, Buy Dilaudid 8Mg, uy25I Nbome Blotter Lsd Online, Buy Morphine 60Mg Online, Buy Nitrazepam Power Online, Buy Flubromazepam Online, Buy Methoxyacetylfentanyl Online, Buy Ephedrine Hcl Online, Buy Clonazolam Powder Online, Buy Alprazolam Powder, Buy Oxycodone Raw Powder, Buy Pure Diazepam Powder Online, Cocaine Hydrochloride For Sale Online, Buy 6-Apb Research Chemical Online, Buy Methoxyacetylfentanyl Online, Buy White Mercedes Pills Online, Buy Carfentanil Online, Buy Fentanyl Hcl Powder, Buy Methiopropanine Crystals Online, Buy Mdma Ecstasy Pills Online, Buy Cocaine And Crack, Order Cocaine, Hight Quality Cocaine,Crystal Meth for sale online, Buy Cocaine Online - Ecstasy for sale - Buy Heroin online - Buy Ketamine Online - Buy LSD Sheets Online, Buy Research Chemicals Online - Research Chemicals for sale online - Order Research Chemicals online - Where to buy Research Chemicals online contact us: + 1 718-567-6492 or Email @ sales@chemicalmedshop.com

    ReplyDelete
  21. to the medical condition of most patients. many people suffering from cancer, chronic pain, anxiety and man more have overcome their illment with cannabis. you can order weed from our store





    ReplyDelete

  22. marijuana has been a stepping stone to the medical condition of most patients.
    many people suffering from cancer, chronic pain, anxiety, and many more have overcome their aillment with cannabis.
    you can order weed from our store.medical condition

    ReplyDelete
  23. We are located in USA AND UK. Buy High Quality Grade(%99.96) Mephedrone (2- methylamino 1-one) also known as 4-methylmethcathinone(4-MMC).We are one of the top suppliers in wholesale and retail of Benzodiazepine,synthetic cannabinoids, Blotters chemicals. Mephedrone and other plants research chemicals such as: Amphetamines Pentylone Mephedrone Flephedrone Bulytone (bk-MBDB) MDAI Analgesic Chemical CB1 and CB2 CP 47497 CP-55940 HU-210 HU-331 Ephedrine Hcl Powder JWH-018 / JWH-200,JWH-250 TFMPP 2C-E, 2C-I, 2C-P, 2C-C, 2C-T-2 DOC, DOI Bromo DragonFly TCB-2 Testosterone 5-Meo-DMT 4-Aco-DMT 4-Ho-MIPT 4-Meo-PCP Naphyrone HEROIN. Methylone (bk-MDMA) Ethylone , Oxycodone Flephedrone (4-FMC, 4-Fluoromethcathinone) Methedrone (BK-PMMA, Methoxyphedrine) 4-Fluoromethamphetamine (4-FMA) a-Pyrrolidinopropiophenone (a-PPP). MDPV (Methylenedioxypyrovalerone, MDPK) Testosterone JWH-073, 1-butyl-3-(1-naphthoyl)indole Hydrocodone Dimethocaine (Larocaine/DMC) Morphine JWH-018, 1-Pentyl-3-(1-naphthoyl)indole Herione 4-Fluoroamphetamine (4-FA, 4-FMP, best site to buy research chemicals, domestic research chemicals, legal research chemicals usa, research chemicals usa vendor, buy research chemicals online usa, where to buy research chemicals online, buy research chemicals united states, legit research chemical supplier usa, best research chemical sites usa, research chemicals usa legit, good research chemical suppliers, etizolam buy, Xanax, buy benzodiazepines, buy drugs, prescription pills, online drug shop, buy Adderall online, buy Ativan(lorazepam) online, buy clonazolam powder online, Alprazolam online, Ketamine Our quality is the best you can find around and we sell in small/large quantities with guaranteed discreet delivery in good time Shipping



    website: https://atlanticchemicalusa.com/
    Call Us: (+1) 312-319-2341
    Email: sales@atlanticchemicalusa.com

    ReplyDelete