{"id":321129,"date":"2021-04-08T15:01:26","date_gmt":"2021-04-08T15:01:26","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=321129"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=321129","title":{"rendered":"\u041e\u0431\u0437\u043e\u0440 \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 AdderNet: \u0414\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u043d\u0430\u043c \u043d\u0443\u0436\u043d\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0432 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u043c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0438? (\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439)"},"content":{"rendered":"\n<div class=\"post__text post__text_v2\" id=\"post-content-body\">\n<h4>\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0432\u043c\u0435\u0441\u0442\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0438 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0438\u0440\u0443\u0435\u0442 \u0432 \u043c\u0435\u043d\u044c\u0448\u0435\u0439 \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0435, \u0447\u0435\u043c \u0443 \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0439 CNN<\/h4>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/bc3\/8fe\/61c\/bc38fe61c74c921c9a18c014fca29774.png\" width=\"780\" height=\"439\"><figcaption><\/figcaption><\/figure>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/323\/d5b\/5e3\/323d5b5e33169e2e012dc81ba34c89d9.png\" alt=\"\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\" title=\"\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\" width=\"895\" height=\"511\"><figcaption>\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/figcaption><\/figure>\n<p>\u0412\u0430\u0448\u0435\u043c\u0443 \u0432\u043d\u0438\u043c\u0430\u043d\u0438\u044e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d \u043e\u0431\u0437\u043e\u0440 \u0441\u0442\u0430\u0442\u044c\u0438 <strong>AdderNet: \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u043d\u0430\u043c \u043d\u0443\u0436\u043d\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0432 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u043c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0438?<\/strong>, (AdderNet), \u041f\u0435\u043a\u0438\u043d\u0441\u043a\u043e\u0433\u043e \u0443\u043d\u0438\u0432\u0435\u0440\u0441\u0438\u0442\u0435\u0442\u0430, Huawei Noah&#8217;s Ark Lab \u0438 \u0421\u0438\u0434\u043d\u0435\u0439\u0441\u043a\u043e\u0433\u043e \u0443\u043d\u0438\u0432\u0435\u0440\u0441\u0438\u0442\u0435\u0442\u0430.<\/p>\n<p><strong><em>\u0414\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u043d\u0430\u043c \u043d\u0443\u0436\u043d\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0432 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u043c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0438?<\/em><\/strong><\/p>\n<\/p>\n<h2>\u0421\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0430 \u0441\u0442\u0430\u0442\u044c\u0438<\/h2>\n<ol>\n<li>\n<p><strong>\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet<\/strong><\/p>\n<\/li>\n<li>\n<p><strong>\u041f\u0440\u043e\u0447\u0438\u0435 \u043c\u043e\u043c\u0435\u043d\u0442\u044b: <\/strong><a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><strong><u>BN<\/u><\/strong><\/a><strong>, \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u044b\u0435, \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f<\/strong><\/p>\n<\/li>\n<li>\n<p><strong>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u043e\u0432<\/strong><\/p>\n<\/li>\n<\/ol>\n<h2>1. \u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet<\/h2>\n<h3>1.1. \u041e\u0431\u043e\u0431\u0449\u0435\u043d\u043d\u044b\u0435 \u0444\u0438\u043b\u044c\u0442\u0440\u044b<\/h3>\n<ul>\n<li>\n<p>\u041a\u0430\u043a \u043f\u0440\u0430\u0432\u0438\u043b\u043e, \u0432\u044b\u0445\u043e\u0434\u043d\u043e\u0439 \u043f\u0440\u0438\u0437\u043d\u0430\u043a <em>Y<\/em> \u0443\u043a\u0430\u0437\u044b\u0432\u0430\u0435\u0442 \u043d\u0430 \u0441\u0445\u043e\u0434\u0441\u0442\u0432\u043e \u043c\u0435\u0436\u0434\u0443 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u043c \u0438 \u0432\u0445\u043e\u0434\u043d\u044b\u043c \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u043e\u043c:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/a8f\/b66\/1e3\/a8fb661e3b5a15cd4185bf2d2090b4db.png\" width=\"454\" height=\"58\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>S<\/em> &#8212; \u043c\u0435\u0440\u0430 \u0441\u0445\u043e\u0434\u0441\u0442\u0432\u0430.<\/p>\n<\/li>\n<\/ul>\n<h3>1.2. \u0421\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0430 \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/h3>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/dde\/3f2\/d3a\/dde3f2d3a00bb68ffe4fc8c754bcda49.png\" alt=\"\u0421\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0430 \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\" title=\"\u0421\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0430 \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\" width=\"376\" height=\"213\"><figcaption>\u0421\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0430 \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0415\u0441\u043b\u0438 \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 \u043c\u0435\u0440\u044b \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f <strong>\u0432\u0437\u0430\u0438\u043c\u043d\u0430\u044f \u043a\u043e\u0440\u0440\u0435\u043b\u044f\u0446\u0438\u044f<\/strong>, \u0442\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f <strong>\u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435<\/strong>. \u0422\u0430\u043a \u043c\u044b \u043f\u043e\u043b\u0443\u0447\u0430\u0435\u043c <strong>\u0441\u0432\u0435\u0440\u0442\u043a\u0443<\/strong>.<\/p>\n<\/li>\n<\/ul>\n<h3>1.3. \u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f<\/h3>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/5bf\/400\/87d\/5bf40087de27d51c2e73a8b86afdc096.png\" alt=\"\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\" title=\"\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\" width=\"367\" height=\"211\"><figcaption>\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0415\u0441\u043b\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f <strong>\u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0435<\/strong>, \u0442\u043e \u0432\u044b\u0447\u0438\u0441\u043b\u044f\u0435\u0442\u0441\u044f <strong><em>l<\/em>1-\u043c\u0435\u0440\u0430<em> <\/em>\u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d\u0438\u044f <\/strong>\u043c\u0435\u0436\u0434\u0443 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u043c \u0438 \u0432\u0445\u043e\u0434\u043d\u044b\u043c \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u043c:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/523\/46b\/165\/52346b165c47c0e9ec2ce08d95642d65.png\" width=\"459\" height=\"59\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e <em>l<\/em>1-\u043c\u0435\u0440\u044b \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d\u0438\u044f \u043c\u043e\u0436\u043d\u043e \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u044c \u0441\u0445\u043e\u0434\u0441\u0442\u0432\u043e \u043c\u0435\u0436\u0434\u0443 \u0444\u0438\u043b\u044c\u0442\u0440\u0430\u043c\u0438 \u0438 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u0430\u043c\u0438.<\/p>\n<\/li>\n<\/ul>\n<p><em>\u0421\u043b\u043e\u0436\u0435\u043d\u0438\u0435 \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u0433\u043e\u0440\u0430\u0437\u0434\u043e \u043c\u0435\u043d\u044c\u0448\u0438\u0445 \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u0435\u0441\u0443\u0440\u0441\u043e\u0432, \u0447\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435.<\/em><\/p>\n<p><em>\u0412\u044b \u043c\u043e\u0433\u043b\u0438 \u0437\u0430\u043c\u0435\u0442\u0438\u0442\u044c, \u0447\u0442\u043e \u043f\u0440\u0438\u0432\u0435\u0434\u0435\u043d\u043d\u043e\u0435 \u0432\u044b\u0448\u0435 \u0443\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u0435 \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0441\u044f \u043a <\/em><strong><em>\u0441\u043e\u043f\u043e\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044e \u0448\u0430\u0431\u043b\u043e\u043d\u043e\u0432 <\/em><\/strong><em>\u0432 \u043a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u043d\u043e\u043c \u0437\u0440\u0435\u043d\u0438\u0438, \u0446\u0435\u043b\u044c \u043a\u043e\u0442\u043e\u0440\u043e\u0433\u043e &#8212; \u043d\u0430\u0439\u0442\u0438 \u0447\u0430\u0441\u0442\u0438 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u044f, \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0435 \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u043d\u043e\u043c\u0443 \u0448\u0430\u0431\u043b\u043e\u043d\u0443.<\/em><\/p>\n<h2>2. \u041f\u0440\u043e\u0447\u0438\u0435 \u043c\u043e\u043c\u0435\u043d\u0442\u044b: BN, \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u044b\u0435, \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f<\/h2>\n<h3>2.1. \u041f\u0430\u043a\u0435\u0442\u043d\u0430\u044f \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f (Batch Normalization &#8212; BN)<\/h3>\n<ul>\n<li>\n<p>\u041f\u043e\u0441\u043b\u0435 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f <strong>\u043f\u0430\u043a\u0435\u0442\u043d\u0430\u044f \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f (<\/strong><a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><strong><u>BN<\/u><\/strong><\/a><strong>) <\/strong>\u0434\u043b\u044f \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 <em>Y<\/em> \u043a \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0435\u043c\u0443 \u0434\u0438\u0430\u043f\u0430\u0437\u043e\u043d\u0443, \u0447\u0442\u043e\u0431\u044b \u0432\u0441\u0435 <strong>\u0444\u0443\u043d\u043a\u0446\u0438\u0438 \u0430\u043a\u0442\u0438\u0432\u0430\u0446\u0438\u0438,<\/strong> \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c\u044b\u0435 \u0432 \u043e\u0431\u044b\u0447\u043d\u044b\u0445 CNN, \u043f\u043e\u0441\u043b\u0435 \u044d\u0442\u043e\u0433\u043e \u043c\u043e\u0433\u043b\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c\u0441\u044f \u0432 \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0445 AdderNets.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0425\u043e\u0442\u044f \u0441\u043b\u043e\u0439 <a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a> \u0432\u043a\u043b\u044e\u0447\u0430\u0435\u0442 \u0432 \u0441\u0435\u0431\u044f \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f, \u0435\u0433\u043e \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0435 \u0437\u0430\u0442\u0440\u0430\u0442\u044b \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043d\u0438\u0436\u0435, \u0447\u0435\u043c \u0443 \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u044b\u0445 \u0441\u043b\u043e\u0435\u0432, \u0438 \u0438\u043c\u0438 \u043c\u043e\u0436\u043d\u043e \u043f\u0440\u0435\u043d\u0435\u0431\u0440\u0435\u0447\u044c.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>(\u041f\u043e\u044f\u0432\u044f\u0442\u0441\u044f \u043b\u0438 \u0432 \u0431\u0443\u0434\u0443\u0449\u0435\u043c \u043a\u0430\u043a\u0438\u0435-\u043d\u0438\u0431\u0443\u0434\u044c <a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a>, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0438\u0435 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0435?)<\/p>\n<\/li>\n<\/ul>\n<h3>2.2. \u041f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u044b\u0435<\/h3>\n<ul>\n<li>\n<p>\u041f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u0430\u044f <em>l<\/em>1-\u043c\u0435\u0440\u044b \u043d\u0435 \u043f\u043e\u0434\u0445\u043e\u0434\u0438\u0442 \u0434\u043b\u044f \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043d\u043e\u0433\u043e \u0441\u043f\u0443\u0441\u043a\u0430. \u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u043c\u044b \u0440\u0430\u0441\u0441\u043c\u0430\u0442\u0440\u0438\u0432\u0430\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u0443\u044e <em>l<\/em>2-\u043c\u0435\u0440\u044b:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/583\/96e\/21b\/58396e21b55557419ac3c8a1f6ff351b.png\" width=\"395\" height=\"49\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 \u0442\u043e\u0447\u043d\u043e\u0433\u043e \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u0430 \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u0442\u043e\u0447\u043d\u043e \u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0442\u044c \u0444\u0438\u043b\u044c\u0442\u0440\u044b.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0427\u0442\u043e\u0431\u044b \u0438\u0437\u0431\u0435\u0436\u0430\u0442\u044c \u0432\u0437\u0440\u044b\u0432\u0430 \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u0430, \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442 <em>X<\/em> \u043e\u0431\u0440\u0435\u0437\u0430\u0435\u0442\u0441\u044f \u0434\u043e [-1,1].<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0417\u0430\u0442\u0435\u043c \u0432\u044b\u0447\u0438\u0441\u043b\u044f\u0435\u0442\u0441\u044f \u0447\u0430\u0441\u0442\u043d\u0430\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u0430\u044f \u0432\u044b\u0445\u043e\u0434\u043d\u044b\u0445 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432 <em>Y<\/em> \u043f\u043e \u043e\u0442\u043d\u043e\u0448\u0435\u043d\u0438\u044e \u043a \u0432\u0445\u043e\u0434\u043d\u044b\u043c \u0445\u0430\u0440\u0430\u043a\u0442\u0435\u0440\u0438\u0441\u0442\u0438\u043a\u0430\u043c <em>X<\/em> \u043a\u0430\u043a:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/433\/31c\/4ce\/43331c4ce996e4b7f88693047776f1cc.png\" width=\"456\" height=\"46\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>HT <\/em>&#8212; \u0444\u0443\u043d\u043a\u0446\u0438\u044f HardTanh:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/c0c\/a94\/e56\/c0ca94e566731df257e94f768ee4ce6d.png\" width=\"297\" height=\"85\"><figcaption><\/figcaption><\/figure>\n<h3>2.3. \u0421\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f<\/h3>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/5b8\/beb\/bf2\/5b8bebbf29cb1b16e91334f75e877ac9.png\" alt=\"l2-\u043c\u0435\u0440\u044b \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u0432 \u0432 LeNet-5-BN\" title=\"l2-\u043c\u0435\u0440\u044b \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u0432 \u0432 LeNet-5-BN\" width=\"370\" height=\"96\"><figcaption>l2-\u043c\u0435\u0440\u044b \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u0432 \u0432 LeNet-5-BN<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u041a\u0430\u043a \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043e \u0432 \u044d\u0442\u043e\u0439 \u0442\u0430\u0431\u043b\u0438\u0446\u0435, \u043c\u0435\u0440\u044b \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u0432 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 AdderNets \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043c\u0435\u043d\u044c\u0448\u0435, \u0447\u0435\u043c \u0432 CNN, \u0447\u0442\u043e \u043c\u043e\u0436\u0435\u0442 \u0437\u0430\u043c\u0435\u0434\u043b\u0438\u0442\u044c \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u0435 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 AdderNets.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0412 AdderNets \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f \u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0440\u0430\u0437\u043d\u044b\u0445 \u0443\u0440\u043e\u0432\u043d\u0435\u0439:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/c67\/9f8\/036\/c679f80362c3e8fbcae3d915da71799b.png\" width=\"211\" height=\"38\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>\u03b3<\/em> &#8212; \u0433\u043b\u043e\u0431\u0430\u043b\u044c\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0432\u0441\u0435\u0439 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u043e\u0439 \u0441\u0435\u0442\u0438 (\u043d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u0434\u043b\u044f \u0441\u0443\u043c\u043c\u0430\u0442\u043e\u0440\u0430 \u0438 <a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a> \u0441\u043b\u043e\u0435\u0432), \u0394<em>L<\/em>(<em>Fl<\/em>) &#8212; \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442 \u0444\u0438\u043b\u044c\u0442\u0440\u0430 \u0432 \u0441\u043b\u043e\u0435 <em>l,<\/em> \u0430 <em>\u03b1l<\/em> &#8212; \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0430\u044f \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0430 \u043a\u0430\u043a<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/b7b\/ea2\/d07\/b7bea2d07794c4d2639f60b39b2b1c29.png\" width=\"172\" height=\"53\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>k<\/em> \u043e\u0431\u043e\u0437\u043d\u0430\u0447\u0430\u0435\u0442 \u043a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u043e\u0432 \u0432 <em>Fl<\/em>, \u0430 <em>\u03b7<\/em> &#8212; \u0433\u0438\u043f\u0435\u0440\u043f\u0430\u0440\u0430\u043c\u0435\u0442\u0440 \u0434\u043b\u044f \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c\u044e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0441\u0443\u043c\u043c\u0430\u0442\u043e\u0440\u0430.<\/p>\n<\/li>\n<\/ul>\n<h2>3. \u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u043e\u0432<\/h2>\n<h3>3.1. MNIST<\/h3>\n<ul>\n<li>\n<p><a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>-5-<a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a> \u043e\u0431\u0443\u0447\u0435\u043d\u0430.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>CNN <\/strong>\u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442<strong> \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 99,4%<\/strong> \u043f\u0440\u0438 <strong>435K \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0439 <\/strong>\u0438 <strong>435K \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0439<\/strong>.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0417\u0430\u043c\u0435\u043d\u044f\u044f \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0432 \u0441\u0432\u0435\u0440\u0442\u043a\u0435 \u043d\u0430 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u0430\u044f <strong>AdderNet <\/strong>\u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442<strong> \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 99,4%<\/strong>, \u0442\u0430\u043a\u043e\u0439 \u0436\u0435 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c \u043a\u0430\u043a \u0443 CNN, \u0441<strong> 870K \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f\u043c\u0438<\/strong> \u0438 <strong>\u043f\u043e\u0447\u0442\u0438 \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0439<\/strong>.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0422\u0435\u043e\u0440\u0435\u0442\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0430 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0432 \u0426\u041f \u0442\u0430\u043a\u0436\u0435 \u0431\u043e\u043b\u044c\u0448\u0435, \u0447\u0435\u043c \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0430 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0438 \u0432\u044b\u0447\u0438\u0442\u0430\u043d\u0438\u044f.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u043d\u0430 \u043c\u043e\u0434\u0435\u043b\u0438 VIA Nano 2000 \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0430 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0438 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0441 \u043f\u043b\u0430\u0432\u0430\u044e\u0449\u0435\u0439 \u0437\u0430\u043f\u044f\u0442\u043e\u0439 \u0441\u043e\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 4 \u0438 2 \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0435\u043d\u043d\u043e. <strong>AdderNet <\/strong>\u0441 \u043c\u043e\u0434\u0435\u043b\u044c\u044e <a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>-5 \u0431\u0443\u0434\u0435\u0442 \u0438\u043c\u0435\u0442\u044c <strong>\u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0443<\/strong> <strong>1.7M<\/strong>, \u0432 \u0442\u043e \u0432\u0440\u0435\u043c\u044f \u043a\u0430\u043a <strong>CNN <\/strong>\u0431\u0443\u0434\u0435\u0442 \u0438\u043c\u0435\u0442\u044c <strong>\u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0443<\/strong> <strong>2.6M <\/strong>\u043d\u0430 \u0442\u043e\u043c \u0436\u0435 CPU.<\/p>\n<\/li>\n<\/ul>\n<h3>3.2. CIFAR<\/h3>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/ce4\/6ba\/64d\/ce46ba64dcdf1c7479eb8b7916c5b181.png\" alt=\"\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 CIFAR-10 \u0438 CIFAR-100\" title=\"\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 CIFAR-10 \u0438 CIFAR-100\" width=\"701\" height=\"283\"><figcaption>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 CIFAR-10 \u0438 CIFAR-100<\/figcaption><\/figure>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/f09\/f70\/278\/f09f70278b1d2a6a1424196766ff06ba.png\" alt=\"BNN: \u0441\u0432\u0435\u0440\u0442\u043a\u0430 XNORNet, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0430\u044f \u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u043e\u0439 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 XNOR\" title=\"BNN: \u0441\u0432\u0435\u0440\u0442\u043a\u0430 XNORNet, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0430\u044f \u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u043e\u0439 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 XNOR\" width=\"371\" height=\"216\"><figcaption>BNN: \u0441\u0432\u0435\u0440\u0442\u043a\u0430 XNORNet, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0430\u044f \u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u043e\u0439 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 XNOR<\/figcaption><\/figure>\n<ul>\n<li>\n<p><strong>\u0414\u0432\u043e\u0438\u0447\u043d\u044b\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u044b\u0435 \u0441\u0435\u0442\u0438 (Binary neural networks &#8212; BNN)<\/strong>: \u043c\u043e\u0433\u0443\u0442 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 <strong>XNOR <\/strong>\u0434\u043b\u044f \u0437\u0430\u043c\u0435\u043d\u044b \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f, \u0447\u0442\u043e \u043c\u044b \u0442\u0430\u043a\u0436\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0434\u043b\u044f \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044f.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>\u0414\u043b\u044f \u043c\u043e\u0434\u0435\u043b\u0438 <\/strong><a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-vggnet-1st-runner-up-of-ilsvlc-2014-image-classification-d02355543a11?source=post_page---------------------------\"><strong><u>VGG<\/u><\/strong><\/a><strong>-small, AdderNets \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 \u043f\u043e\u0447\u0442\u0438 \u0442\u0430\u043a\u0438\u0445 \u0436\u0435 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 (93,72% \u0432 CIFAR-10 \u0438 72,64% \u0432 CIFAR-100) \u043a\u0430\u043a \u0438 CNNs (93,80% \u0432 CIFAR-10 \u0438 72,73% \u0432 CIFAR-100).<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0425\u043e\u0442\u044f \u0440\u0430\u0437\u043c\u0435\u0440 \u043c\u043e\u0434\u0435\u043b\u0438 BNN \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043c\u0435\u043d\u044c\u0448\u0435, \u0447\u0435\u043c \u0443 AdderNet \u0438 CNN, \u0435\u0435 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043d\u0438\u0436\u0435 (89,80% \u0432 CIFAR-10 \u0438 65,41% \u0432 CIFAR-100).<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0427\u0442\u043e \u043a\u0430\u0441\u0430\u0435\u0442\u0441\u044f <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet-<\/u><\/a>20, CNN \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u043d\u0430\u0438\u0432\u044b\u0441\u0448\u0435\u0439 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 (\u0442.\u0435. 92,25% \u0432 CIFAR-10 \u0438 68,14% \u0432 CIFAR-100), \u043d\u043e \u0441 \u0431\u043e\u043b\u044c\u0448\u0438\u043c \u0447\u0438\u0441\u043b\u043e\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0439 (41,17M).<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>\u041f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0435 AdderNets \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 91,84% \u0432 CIFAR-10 \u0438 67,60% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 \u0432 CIFAR-100 \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f, \u0447\u0442\u043e \u0441\u0440\u0430\u0432\u043d\u0438\u043c\u043e \u0441 CNN.<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041d\u0430\u043f\u0440\u043e\u0442\u0438\u0432, BNN \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 \u0442\u043e\u043b\u044c\u043a\u043e 84,87% \u0438 54,14% \u0432 CIFAR-10 \u0438 CIFAR-100.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b <\/strong><a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><strong><u>ResNet-<\/u><\/strong><\/a><strong>32 \u0442\u0430\u043a\u0436\u0435 \u043f\u0440\u0435\u0434\u043f\u043e\u043b\u0430\u0433\u0430\u044e\u0442, \u0447\u0442\u043e \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0435 AdderNets \u043c\u043e\u0433\u0443\u0442 \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0442\u044c \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 \u0430\u043d\u0430\u043b\u043e\u0433\u0438\u0447\u043d\u044b\u0445 \u043e\u0431\u044b\u0447\u043d\u044b\u043c CNN.<\/strong><\/p>\n<\/li>\n<\/ul>\n<h3>3.3. ImageNet<\/h3>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/286\/433\/f3c\/286433f3c7c5156e08b529c3e64bc879.png\" alt=\"\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 ImageNet&nbsp;\" title=\"\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 ImageNet&nbsp;\" width=\"657\" height=\"199\"><figcaption>\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 ImageNet&nbsp;<\/figcaption><\/figure>\n<ul>\n<li>\n<p>CNN \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 69,8% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-1 \u0438 89,1% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-5 \u0432 <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>RESNET<\/u><\/a>-18. \u041e\u0434\u043d\u0430\u043a\u043e, \u043f\u0440\u0438 1.8G \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\u0445.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>AdderNet \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0438\u0432\u0430\u0435\u0442 66,8% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-1 \u0438 87,4% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-5 \u0432 <\/strong><a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><strong><u>ResNet-<\/u><\/strong><\/a><strong>18, \u0447\u0442\u043e \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u0435\u0442, \u0447\u0442\u043e \u0444\u0438\u043b\u044c\u0442\u0440\u044b \u0441\u0443\u043c\u043c\u0430\u0442\u043e\u0440\u0430 \u043c\u043e\u0433\u0443\u0442 \u0438\u0437\u0432\u043b\u0435\u043a\u0430\u0442\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0443\u044e \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u0438\u0437 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439.<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041d\u0435\u0441\u043c\u043e\u0442\u0440\u044f \u043d\u0430 \u0442\u043e, \u0447\u0442\u043e BNN \u043c\u043e\u0436\u0435\u0442 \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0442\u044c \u0432\u044b\u0441\u043e\u043a\u043e\u0439 \u0441\u0442\u0435\u043f\u0435\u043d\u0438 \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f \u0438 \u0441\u0436\u0430\u0442\u0438\u044f, \u043e\u043d \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 \u0442\u043e\u043b\u044c\u043a\u043e 51,2% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-1 \u0438 73,2% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-5 \u0432 <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet-<\/u><\/a>18.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0410\u043d\u0430\u043b\u043e\u0433\u0438\u0447\u043d\u044b\u0435 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0434\u043b\u044f \u0431\u043e\u043b\u0435\u0435 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u0433\u043e <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet<\/u><\/a>-50.<\/p>\n<\/li>\n<\/ul>\n<h3>3.4. \u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0432\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438<\/h3>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/b41\/e99\/b38\/b41e99b389c1132933ac6e728ee984fa.png\" alt=\"\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432 \u0432 AdderNets \u0438 CNN. \u041f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 CNN \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432 \u0440\u0430\u0437\u0434\u0435\u043b\u0435\u043d\u044b \u043f\u043e \u0438\u0445 \u0443\u0433\u043b\u0430\u043c.\" title=\"\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432 \u0432 AdderNets \u0438 CNN. \u041f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 CNN \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432 \u0440\u0430\u0437\u0434\u0435\u043b\u0435\u043d\u044b \u043f\u043e \u0438\u0445 \u0443\u0433\u043b\u0430\u043c.\" width=\"909\" height=\"351\"><figcaption>\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432 \u0432 AdderNets \u0438 CNN. \u041f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 CNN \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432 \u0440\u0430\u0437\u0434\u0435\u043b\u0435\u043d\u044b \u043f\u043e \u0438\u0445 \u0443\u0433\u043b\u0430\u043c.<\/figcaption><\/figure>\n<ul>\n<li>\n<p>&nbsp;<a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>++ \u043e\u0431\u0443\u0447\u0430\u043b\u0441\u044f \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0435 \u0434\u0430\u043d\u043d\u044b\u0445 MNIST, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0438\u043c\u0435\u0435\u0442 \u0448\u0435\u0441\u0442\u044c \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u044b\u0445 \u0441\u043b\u043e\u0435\u0432 \u0438 \u043f\u043e\u043b\u043d\u043e\u0441\u0432\u044f\u0437\u043d\u044b\u0439 \u0441\u043b\u043e\u0439 \u0434\u043b\u044f \u0438\u0437\u0432\u043b\u0435\u0447\u0435\u043d\u0438\u044f \u0432\u044b\u0440\u0430\u0436\u0435\u043d\u043d\u044b\u0445 3D \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u043d\u0435\u0439\u0440\u043e\u043d\u043e\u0432 \u0432 \u043a\u0430\u0436\u0434\u043e\u043c \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u043e\u043c \u0441\u043b\u043e\u0435 \u0441\u043e\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 32, 32, 64, 64, 128, 128 \u0438 2 \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0435\u043d\u043d\u043e.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>AdderNets \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442 <em>l<\/em>1-\u043c\u0435\u0440\u0443 \u0434\u043b\u044f \u0440\u0430\u0437\u043b\u0438\u0447\u0435\u043d\u0438\u044f \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432. \u041f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 \u0438\u043c\u0435\u044e\u0442 \u0442\u0435\u043d\u0434\u0435\u043d\u0446\u0438\u044e \u0431\u044b\u0442\u044c \u0441\u0433\u0440\u0443\u043f\u043f\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u043c\u0438 \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0446\u0435\u043d\u0442\u0440\u043e\u0432 \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0432\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0442, \u0447\u0442\u043e \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0435 AdderNets \u043c\u043e\u0433\u0443\u0442 \u043e\u0431\u043b\u0430\u0434\u0430\u0442\u044c \u0430\u043d\u0430\u043b\u043e\u0433\u0438\u0447\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e \u0440\u0430\u0441\u043f\u043e\u0437\u043d\u0430\u0432\u0430\u043d\u0438\u044f \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439 \u043a\u0430\u043a \u0438 CNN.<\/p>\n<\/li>\n<\/ul>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/e2d\/66e\/354\/e2d66e354882e80ed269314e8e89c205.png\" alt=\"\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 \u043f\u0435\u0440\u0432\u043e\u043c \u0441\u043b\u043e\u0435 LeNet-5-BN \u043d\u0430 MNIST\" title=\"\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 \u043f\u0435\u0440\u0432\u043e\u043c \u0441\u043b\u043e\u0435 LeNet-5-BN \u043d\u0430 MNIST\" width=\"895\" height=\"111\"><figcaption>\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 \u043f\u0435\u0440\u0432\u043e\u043c \u0441\u043b\u043e\u0435 LeNet-5-BN \u043d\u0430 MNIST<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0424\u0438\u043b\u044c\u0442\u0440\u044b \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0445 adderNets \u043f\u043e-\u043f\u0440\u0435\u0436\u043d\u0435\u043c\u0443 \u0438\u043c\u0435\u044e\u0442 \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441\u0445\u043e\u0436\u0438\u0435 \u043f\u0430\u0442\u0442\u0435\u0440\u043d\u044b \u0441\u043e \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u044b\u043c\u0438 \u0444\u0438\u043b\u044c\u0442\u0440\u0430\u043c\u0438.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u042d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u044b \u043f\u043e \u0432\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 \u0434\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0442, \u0447\u0442\u043e \u0444\u0438\u043b\u044c\u0442\u0440\u044b AdderNets \u043c\u043e\u0433\u0443\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0438\u0437\u0432\u043b\u0435\u043a\u0430\u0442\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0443\u044e \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u0438\u0437 \u0432\u0445\u043e\u0434\u043d\u044b\u0445 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439 \u0438 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432.<\/p>\n<\/li>\n<\/ul>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/3b4\/01b\/f5a\/3b401bf5a47b84befbf287fd555810e0.png\" alt=\"\u0413\u0438\u0441\u0442\u043e\u0433\u0440\u0430\u043c\u043c\u044b \u043f\u043e \u0432\u0435\u0441\u0430\u043c \u0441 AdderNet (\u0441\u043b\u0435\u0432\u0430) \u0438 CNN (\u0441\u043f\u0440\u0430\u0432\u0430).\" title=\"\u0413\u0438\u0441\u0442\u043e\u0433\u0440\u0430\u043c\u043c\u044b \u043f\u043e \u0432\u0435\u0441\u0430\u043c \u0441 AdderNet (\u0441\u043b\u0435\u0432\u0430) \u0438 CNN (\u0441\u043f\u0440\u0430\u0432\u0430).\" width=\"706\" height=\"261\"><figcaption>\u0413\u0438\u0441\u0442\u043e\u0433\u0440\u0430\u043c\u043c\u044b \u043f\u043e \u0432\u0435\u0441\u0430\u043c \u0441 AdderNet (\u0441\u043b\u0435\u0432\u0430) \u0438 CNN (\u0441\u043f\u0440\u0430\u0432\u0430).<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0420\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u0432\u0435\u0441\u043e\u0432 \u0441 AdderNets \u0431\u043b\u0438\u0437\u043a\u043e \u043a \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u044e \u041b\u0430\u043f\u043b\u0430\u0441\u0430, \u0442\u043e\u0433\u0434\u0430 \u043a\u0430\u043a \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u0441 CNN \u0431\u043e\u043b\u044c\u0448\u0435 \u043f\u043e\u0445\u043e\u0434\u0438\u0442 \u0431\u043e\u043b\u044c\u0448\u0435 \u043d\u0430 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u0413\u0430\u0443\u0441\u0441\u0430. \u0424\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438, \u0430\u043f\u0440\u0438\u043e\u0440\u043d\u044b\u043c \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435\u043c <em>l<\/em>1-\u043c\u0435\u0440\u044b \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u041b\u0430\u043f\u043b\u0430\u0441\u0430.<\/p>\n<\/li>\n<\/ul>\n<h3>3.5. \u0410\u0431\u043b\u044f\u0446\u0438\u043e\u043d\u043d\u043e\u0435 \u0438\u0441\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u043d\u0438\u0435&nbsp;<\/h3>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/839\/498\/af6\/839498af6a195279e08547598c582eb6.png\" alt=\"\u041a\u0440\u0438\u0432\u0430\u044f \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f AdderNets \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0445 \u0441\u0445\u0435\u043c \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u0438\" title=\"\u041a\u0440\u0438\u0432\u0430\u044f \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f AdderNets \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0445 \u0441\u0445\u0435\u043c \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u0438\" width=\"918\" height=\"360\"><figcaption>\u041a\u0440\u0438\u0432\u0430\u044f \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f AdderNets \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0445 \u0441\u0445\u0435\u043c \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u0438<\/figcaption><\/figure>\n<ul>\n<li>\n<p><strong>AdderNets, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0438\u0435 \u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0443\u044e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f (adaptive learning rate &#8212; ALR) \u0438 \u0443\u0432\u0435\u043b\u0438\u0447\u0435\u043d\u043d\u0443\u044e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f (increased learning rate &#8212; ILR), \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 97,99% \u0438 97,72% \u0441\u043e \u0437\u043d\u0430\u043a\u043e\u0432\u044b\u043c \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u043c, \u0447\u0442\u043e \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043d\u0438\u0436\u0435, \u0447\u0435\u043c \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c CNN (99,40%) .<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u043c\u044b \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u0442\u043e\u0447\u043d\u044b\u0439 \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442 \u0434\u043b\u044f \u0431\u043e\u043b\u0435\u0435 \u0442\u043e\u0447\u043d\u043e\u0433\u043e \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u044f \u0432\u0435\u0441\u043e\u0432 \u0432 AdderNets.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0412 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0435 AdderNet \u0441 ILR \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 98,99% \u043f\u0440\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 <strong>\u0442\u043e\u0447\u043d\u043e\u0433\u043e \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u0430<\/strong>. \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f <strong>\u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0443\u044e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f (ALR)<\/strong>, <strong>AdderNet \u043c\u043e\u0436\u0435\u0442 \u0434\u043e\u0441\u0442\u0438\u0447\u044c \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 99,40%<\/strong>, \u0447\u0442\u043e \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u0435\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u044c \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u043d\u043e\u0433\u043e \u043c\u0435\u0442\u043e\u0434\u0430.<\/p>\n<\/li>\n<\/ul>\n<h3>\u0421\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u0441\u0442\u0430\u0442\u044c\u044e<\/h3>\n<p>[2020 CVPR] [AdderNet]<\/p>\n<p><a href=\"https:\/\/openaccess.thecvf.com\/content_CVPR_2020\/papers\/Chen_AdderNet_Do_We_Really_Need_Multiplications_in_Deep_Learning_CVPR_2020_paper.pdf\"><u>AdderNet: Do We Really Need Multiplications in Deep Learning?<\/u><\/a><\/p>\n<h3>\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439<\/h3>\n<p><strong>1989\u20131998<\/strong>: [<a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>]<\/p>\n<p><strong>2012\u20132014<\/strong>: [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-alexnet-caffenet-winner-in-ilsvrc-2012-image-classification-b93598314160?source=post_page---------------------------\"><u>AlexNet &amp; CaffeNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-dropout-a-simple-way-to-prevent-neural-networks-from-overfitting-image-classification-a74b369b4b8e\"><u>Dropout<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-maxout-network-image-classification-40ecd77f7ce4?source=post_page---------------------------\"><u>Maxout<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-nin-network-in-network-image-classification-69e271e499ee?source=post_page---------------------------\"><u>NIN<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-zfnet-the-winner-of-ilsvlc-2013-image-classification-d1a5a0c45103?source=post_page---------------------------\"><u>ZFNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/review-sppnet-1st-runner-up-object-detection-2nd-runner-up-image-classification-in-ilsvrc-906da3753679?source=post_page---------------------------\"><u>SPPNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-model-distillation-distilling-the-knowledge-in-a-neural-network-image-classification-48ce0c81618a\"><u>Distillation<\/u><\/a>]<\/p>\n<p><strong>2015<\/strong>: [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-vggnet-1st-runner-up-of-ilsvlc-2014-image-classification-d02355543a11?source=post_page---------------------------\"><u>VGGNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-highway-networks-gating-function-to-highway-image-classification-5a33833797b5?source=post_page---------------------------\"><u>Highway<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/review-prelu-net-the-first-to-surpass-human-level-performance-in-ilsvrc-2015-image-f619dddd5617?source=post_page---------------------------\"><u>PReLU-Net<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-stn-spatial-transformer-network-image-classification-d3cbd98a70aa?source=post_page---------------------------\"><u>STN<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-deep-image-a-big-data-solution-for-image-recognition-99e5f7b1c802?source=post_page---------------------------\"><u>DeepImage<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-googlenet-inception-v1-winner-of-ilsvlc-2014-image-classification-c2b3565a64e7?source=post_page---------------------------\"><u>GoogLeNet \/ Inception-v1<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN-Inception \/ Inception-v2<\/u><\/a>]<\/p>\n<p><strong>2016<\/strong>: [<a href=\"https:\/\/towardsdatascience.com\/review-squeezenet-image-classification-e7414825581a?source=post_page---------------------------\"><u>SqueezeNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-inception-v3-1st-runner-up-image-classification-in-ilsvrc-2015-17915421f77c?source=post_page---------------------------\"><u>Inception-v3<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/resnet-with-identity-mapping-over-1000-layers-reached-image-classification-bb50a42af03e?source=post_page---------------------------\"><u>Pre-Activation ResNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-rir-resnet-in-resnet-image-classification-be4c79fde8ba?source=post_page---------------------------\"><u>RiR<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-stochastic-depth-image-classification-a4e225807f4a?source=post_page---------------------------\"><u>Stochastic Depth<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-wrns-wide-residual-networks-image-classification-d3feb3fb2004?source=post_page---------------------------\"><u>WRN<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-trimps-soushen-winner-in-ilsvrc-2016-image-classification-dfbc423111dd?source=post_page---------------------------\"><u>Trimps-Soushen<\/u><\/a>]<\/p>\n<p><strong>2017<\/strong>: [<a href=\"https:\/\/towardsdatascience.com\/review-inception-v4-evolved-from-googlenet-merged-with-resnet-idea-image-classification-5e8c339d18bc?source=post_page---------------------------\"><u>Inception-v4<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-xception-with-depthwise-separable-convolution-better-than-inception-v3-image-dc967dd42568?source=post_page---------------------------\"><u>Xception<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-mobilenetv1-depthwise-separable-convolution-light-weight-model-a382df364b69?source=post_page---------------------------\"><u>MobileNetV1<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-shake-shake-regularization-image-classification-d22bb8587953?source=post_page---------------------------\"><u>Shake-Shake<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-cutout-improved-regularization-of-convolutional-neural-networks-image-classification-d39ff4ec3c76\"><u>Cutout<\/u><\/a>] [<a href=\"https:\/\/medium.com\/datadriveninvestor\/review-fractalnet-image-classification-c5bdd855a090?source=post_page---------------------------\"><u>FractalNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-polynet-2nd-runner-up-in-ilsvrc-2016-image-classification-8a1a941ce9ea?source=post_page---------------------------\"><u>PolyNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-resnext-1st-runner-up-of-ilsvrc-2016-image-classification-15d7f17b42ac?source=post_page---------------------------\"><u>ResNeXt<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-densenet-image-classification-b6631a8ef803?source=post_page---------------------------\"><u>DenseNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-pyramidnet-deep-pyramidal-residual-networks-image-classification-85a87b60ae78?source=post_page---------------------------\"><u>PyramidNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-drn-dilated-residual-networks-image-classification-semantic-segmentation-d527e1a8fb5?source=post_page---------------------------\"><u>DRN<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-dpn-dual-path-networks-image-classification-d0135dce8817?source=post_page---------------------------\"><u>DPN<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-residual-attention-network-attention-aware-features-image-classification-7ae44c4f4b8?source=post_page---------------------------\"><u>Residual Attention Network<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-igcnet-igcv1-interleaved-group-convolutions-image-classification-7421d2a1dede?source=post_page---------------------------\"><u>IGCNet \/ IGCV1<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-deep-roots-improving-cnn-efficiency-with-hierarchical-filter-groups-image-9aba67f23b27\"><u>Deep Roots<\/u><\/a>]<\/p>\n<p><strong>2018<\/strong>: [<a href=\"https:\/\/towardsdatascience.com\/review-ror-resnet-of-resnet-multilevel-resnet-image-classification-cd3b0fcc19bb?source=post_page---------------------------\"><u>RoR<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-dmrnet-dfn-mr-merge-and-run-mappings-image-classification-493080a4b8ae?source=post_page---------------------------\"><u>DMRNet \/ DFN-MR<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-msdnet-multi-scale-dense-networks-image-classification-4d949955f6d5?source=post_page---------------------------\"><u>MSDNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-shufflenet-v1-light-weight-model-image-classification-5b253dfe982f?source=post_page---------------------------\"><u>ShuffleNet V1<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-senet-squeeze-and-excitation-network-winner-of-ilsvrc-2017-image-classification-a887b98b2883?source=post_page---------------------------\"><u>SENet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-nasnet-neural-architecture-search-network-image-classification-23139ea0425d?source=post_page---------------------------\"><u>NASNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-mobilenetv2-light-weight-model-image-classification-8febb490e61c?source=post_page---------------------------\"><u>MobileNetV2<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-condensenet-improve-densenet-with-learned-group-convolution-image-classification-85de829781db\"><u>CondenseNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-igcv2-interleaved-structured-sparse-convolution-image-classification-59370cd5e19e\"><u>IGCV2<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-igcv3-interleaved-low-rank-group-convolutions-image-classification-c5fd120d7cf1\"><u>IGCV3<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-fishnet-fishnext-a-versatile-backbone-for-image-region-and-pixel-level-prediction-a5f9aa91f96b\"><u>FishNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-squeezenext-hardware-aware-neural-network-design-image-classification-3fc8d1d3f76\"><u>SqueezeNext<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-enas-efficient-neural-architecture-search-via-parameter-sharing-image-classification-418b2a067bba\"><u>ENAS<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-pnasnet-progressive-neural-architecture-search-image-classification-1beb1de06fe6\"><u>PNASNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-shufflenet-v2-practical-guidelines-for-e-fficient-cnn-architecture-design-image-287b05abc08a\"><u>ShuffleNet V2<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-bam-bottleneck-attention-module-image-classification-439a2456c1a1\"><u>BAM<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-cbam-convolutional-block-attention-module-image-classification-ddbaf10f7430\"><u>CBAM<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/reading-morphnet-fast-simple-resource-constrained-structure-learning-of-deep-networks-image-89caa0b9f18b\"><u>MorphNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-netadapt-platform-aware-neural-network-adaptation-for-mobile-applications-image-84d7210bb828\"><u>NetAdapt<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-mixup-beyond-empirical-risk-minimization-image-classification-6ee40a45ad17\"><u>mixup<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/dropblock-a-regularization-method-for-convolutional-networks-image-classification-14556bb13489\"><u>DropBlock<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-group-norm-gn-group-normalization-image-classification-5f7fe0f58eb6\"><u>Group Norm (GN)<\/u><\/a>]<\/p>\n<p><strong>2019<\/strong>: [<a href=\"https:\/\/medium.com\/@sh.tsang\/resnet-38-wider-or-deeper-resnet-image-classification-semantic-segmentation-f297f2f73437\"><u>ResNet-38<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-amoebanet-regularized-evolution-for-image-classifier-architecture-search-image-278f5c077a4a\"><u>AmoebaNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-espnetv2-a-light-weight-power-efficient-and-general-purpose-convolutional-neural-a57b5fbfeb84\"><u>ESPNetv2<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/reading-mnasnet-platform-aware-neural-architecture-search-for-mobile-image-classification-b042aaef66f7\"><u>MnasNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/carre4\/paper-single-path-nas-designing-hardware-efficient-convnets-in-less-than-4-hours-image-classifi-81cff87f223e\"><u>Single-Path NAS<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-darts-differentiable-architecture-search-image-classification-dda80703b81a\"><u>DARTS<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-proxylessnas-direct-neural-architecture-search-on-target-task-image-classification-73c35ebd8aed\"><u>ProxylessNAS<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-mobilenetv3-searching-for-mobilenetv3-image-classification-5072d4d8703c\"><u>MobileNetV3<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-fbnet-hardware-aware-efficient-convnet-design-via-differentiable-neural-architecture-f95a20c1cd4a\"><u>FBNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-shakedrop-shakedrop-regularization-for-deep-residual-learning-image-classification-d2b83d0ae3de\"><u>ShakeDrop<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-cutmix-regularization-strategy-to-train-strong-classifiers-with-localizable-features-5527e29c4890\"><u>CutMix<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/mixconv-mixed-depthwise-convolutional-kernels-image-classification-9d0ca8bf4093\"><u>MixConv<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/efficientnet-rethinking-model-scaling-for-convolutional-neural-networks-image-classification-ef67b0f14a4d\"><u>EfficientNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-abn-attention-branch-network-for-visual-explanation-image-classification-30fd164960ee\"><u>ABN<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-sknet-selective-kernel-networks-image-classification-63ebbad7d78f\"><u>SKNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-cb-loss-class-balanced-loss-based-on-effective-number-of-samples-image-classification-3056a1a1a001\"><u>CB Loss<\/u><\/a>]<\/p>\n<p><strong>2020<\/strong>: [<a href=\"https:\/\/sh-tsang.medium.com\/random-erasing-re-random-erasing-data-augmentation-image-classification-37f11627b38\"><u>Random Erasing (RE)<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-saol-spatially-attentive-output-layer-image-classification-weakly-supervised-object-742e2eeb3226\"><u>SAOL<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-addernet-do-we-really-need-multiplications-in-deep-learning-image-classification-b72851ddb255\"><u>AdderNet<\/u><\/a>]<\/p>\n<hr>\n<blockquote>\n<p><em>\u041f\u0435\u0440\u0435\u0432\u043e\u0434 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430 \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043b\u0435\u043d \u0432 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 <\/em><a href=\"https:\/\/otus.pw\/lobO\/\"><strong><em>&#171;Deep Learning. Basic&#187;<\/em><\/strong><\/a><em>. <\/em><\/p>\n<p><em>\u0422\u0430\u043a\u0436\u0435 \u043f\u0440\u0438\u0433\u043b\u0430\u0448\u0430\u0435\u043c \u0432\u0441\u0435\u0445 \u0436\u0435\u043b\u0430\u044e\u0449\u0438\u0445 \u043f\u043e\u0441\u0435\u0442\u0438\u0442\u044c <\/em><a href=\"https:\/\/otus.pw\/BCh8\/\"><strong><em>\u0431\u0435\u0441\u043f\u043b\u0430\u0442\u043d\u044b\u0439 \u0434\u0435\u043c\u043e-\u0443\u0440\u043e\u043a<\/em><\/strong><\/a><em> \u043f\u043e \u0442\u0435\u043c\u0435: &#171;Knowledge distillation: \u043d\u0435\u0439\u0440\u043e\u0441\u0435\u0442\u0438 \u043e\u0431\u0443\u0447\u0430\u044e\u0442 \u043d\u0435\u0439\u0440\u043e\u0441\u0435\u0442\u0438&#187;.<\/em><\/p>\n<p><strong> &#8212; <\/strong><a href=\"https:\/\/otus.pw\/lobO\/\"><strong>\u0423\u0417\u041d\u0410\u0422\u042c \u041f\u041e\u0414\u0420\u041e\u0411\u041d\u0415\u0415 \u041e \u041a\u0423\u0420\u0421\u0415<\/strong><\/a><\/p>\n<p><strong> &#8212; <\/strong><a href=\"https:\/\/otus.pw\/BCh8\/\"><strong>\u0417\u0410\u041f\u0418\u0421\u0410\u0422\u042c\u0421\u042f \u041d\u0410 \u0411\u0415\u0421\u041f\u041b\u0410\u0422\u041d\u042b\u0419 \u0414\u0415\u041c\u041e-\u0423\u0420\u041e\u041a<\/strong><\/a><\/p>\n<\/blockquote>\n<\/div>\n<p> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/551446\/\"> https:\/\/habr.com\/ru\/company\/otus\/blog\/551446\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"\n<div class=\"post__text post__text_v2\" id=\"post-content-body\">\n<h4>\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0432\u043c\u0435\u0441\u0442\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0438 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0438\u0440\u0443\u0435\u0442 \u0432 \u043c\u0435\u043d\u044c\u0448\u0435\u0439 \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0435, \u0447\u0435\u043c \u0443 \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0439 CNN<\/h4>\n<figure class=\"full-width\"><figcaption><\/figcaption><\/figure>\n<figure class=\"full-width\"><figcaption>\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/figcaption><\/figure>\n<p>\u0412\u0430\u0448\u0435\u043c\u0443 \u0432\u043d\u0438\u043c\u0430\u043d\u0438\u044e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d \u043e\u0431\u0437\u043e\u0440 \u0441\u0442\u0430\u0442\u044c\u0438 <strong>AdderNet: \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u043d\u0430\u043c \u043d\u0443\u0436\u043d\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0432 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u043c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0438?<\/strong>, (AdderNet), \u041f\u0435\u043a\u0438\u043d\u0441\u043a\u043e\u0433\u043e \u0443\u043d\u0438\u0432\u0435\u0440\u0441\u0438\u0442\u0435\u0442\u0430, Huawei Noah&#8217;s Ark Lab \u0438 \u0421\u0438\u0434\u043d\u0435\u0439\u0441\u043a\u043e\u0433\u043e \u0443\u043d\u0438\u0432\u0435\u0440\u0441\u0438\u0442\u0435\u0442\u0430.<\/p>\n<p><strong><em>\u0414\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u043d\u0430\u043c \u043d\u0443\u0436\u043d\u043e \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0432 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u043c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0438?<\/em><\/strong><\/p>\n<\/p>\n<h2>\u0421\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0430 \u0441\u0442\u0430\u0442\u044c\u0438<\/h2>\n<ol>\n<li>\n<p><strong>\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet<\/strong><\/p>\n<\/li>\n<li>\n<p><strong>\u041f\u0440\u043e\u0447\u0438\u0435 \u043c\u043e\u043c\u0435\u043d\u0442\u044b: <\/strong><a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><strong><u>BN<\/u><\/strong><\/a><strong>, \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u044b\u0435, \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f<\/strong><\/p>\n<\/li>\n<li>\n<p><strong>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u043e\u0432<\/strong><\/p>\n<\/li>\n<\/ol>\n<h2>1. \u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet<\/h2>\n<h3>1.1. \u041e\u0431\u043e\u0431\u0449\u0435\u043d\u043d\u044b\u0435 \u0444\u0438\u043b\u044c\u0442\u0440\u044b<\/h3>\n<ul>\n<li>\n<p>\u041a\u0430\u043a \u043f\u0440\u0430\u0432\u0438\u043b\u043e, \u0432\u044b\u0445\u043e\u0434\u043d\u043e\u0439 \u043f\u0440\u0438\u0437\u043d\u0430\u043a <em>Y<\/em> \u0443\u043a\u0430\u0437\u044b\u0432\u0430\u0435\u0442 \u043d\u0430 \u0441\u0445\u043e\u0434\u0441\u0442\u0432\u043e \u043c\u0435\u0436\u0434\u0443 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u043c \u0438 \u0432\u0445\u043e\u0434\u043d\u044b\u043c \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u043e\u043c:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>S<\/em> &#8212; \u043c\u0435\u0440\u0430 \u0441\u0445\u043e\u0434\u0441\u0442\u0432\u0430.<\/p>\n<\/li>\n<\/ul>\n<h3>1.2. \u0421\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0430 \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/h3>\n<figure class=\"\"><figcaption>\u0421\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f \u0441\u0432\u0435\u0440\u0442\u043a\u0430 \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0415\u0441\u043b\u0438 \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 \u043c\u0435\u0440\u044b \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f <strong>\u0432\u0437\u0430\u0438\u043c\u043d\u0430\u044f \u043a\u043e\u0440\u0440\u0435\u043b\u044f\u0446\u0438\u044f<\/strong>, \u0442\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f <strong>\u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435<\/strong>. \u0422\u0430\u043a \u043c\u044b \u043f\u043e\u043b\u0443\u0447\u0430\u0435\u043c <strong>\u0441\u0432\u0435\u0440\u0442\u043a\u0443<\/strong>.<\/p>\n<\/li>\n<\/ul>\n<h3>1.3. \u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f<\/h3>\n<figure class=\"\"><figcaption>\u0421\u0432\u0435\u0440\u0442\u043a\u0430 AdderNet \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0415\u0441\u043b\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f <strong>\u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0435<\/strong>, \u0442\u043e \u0432\u044b\u0447\u0438\u0441\u043b\u044f\u0435\u0442\u0441\u044f <strong><em>l<\/em>1-\u043c\u0435\u0440\u0430<em> <\/em>\u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d\u0438\u044f <\/strong>\u043c\u0435\u0436\u0434\u0443 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u043c \u0438 \u0432\u0445\u043e\u0434\u043d\u044b\u043c \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u043c:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e <em>l<\/em>1-\u043c\u0435\u0440\u044b \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d\u0438\u044f \u043c\u043e\u0436\u043d\u043e \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u044c \u0441\u0445\u043e\u0434\u0441\u0442\u0432\u043e \u043c\u0435\u0436\u0434\u0443 \u0444\u0438\u043b\u044c\u0442\u0440\u0430\u043c\u0438 \u0438 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u0430\u043c\u0438.<\/p>\n<\/li>\n<\/ul>\n<p><em>\u0421\u043b\u043e\u0436\u0435\u043d\u0438\u0435 \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u0433\u043e\u0440\u0430\u0437\u0434\u043e \u043c\u0435\u043d\u044c\u0448\u0438\u0445 \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u0435\u0441\u0443\u0440\u0441\u043e\u0432, \u0447\u0435\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435.<\/em><\/p>\n<p><em>\u0412\u044b \u043c\u043e\u0433\u043b\u0438 \u0437\u0430\u043c\u0435\u0442\u0438\u0442\u044c, \u0447\u0442\u043e \u043f\u0440\u0438\u0432\u0435\u0434\u0435\u043d\u043d\u043e\u0435 \u0432\u044b\u0448\u0435 \u0443\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u0435 \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0441\u044f \u043a <\/em><strong><em>\u0441\u043e\u043f\u043e\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044e \u0448\u0430\u0431\u043b\u043e\u043d\u043e\u0432 <\/em><\/strong><em>\u0432 \u043a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u043d\u043e\u043c \u0437\u0440\u0435\u043d\u0438\u0438, \u0446\u0435\u043b\u044c \u043a\u043e\u0442\u043e\u0440\u043e\u0433\u043e &#8212; \u043d\u0430\u0439\u0442\u0438 \u0447\u0430\u0441\u0442\u0438 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u044f, \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0435 \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u043d\u043e\u043c\u0443 \u0448\u0430\u0431\u043b\u043e\u043d\u0443.<\/em><\/p>\n<h2>2. \u041f\u0440\u043e\u0447\u0438\u0435 \u043c\u043e\u043c\u0435\u043d\u0442\u044b: BN, \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u044b\u0435, \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f<\/h2>\n<h3>2.1. \u041f\u0430\u043a\u0435\u0442\u043d\u0430\u044f \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f (Batch Normalization &#8212; BN)<\/h3>\n<ul>\n<li>\n<p>\u041f\u043e\u0441\u043b\u0435 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f <strong>\u043f\u0430\u043a\u0435\u0442\u043d\u0430\u044f \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f (<\/strong><a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><strong><u>BN<\/u><\/strong><\/a><strong>) <\/strong>\u0434\u043b\u044f \u043d\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 <em>Y<\/em> \u043a \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0435\u043c\u0443 \u0434\u0438\u0430\u043f\u0430\u0437\u043e\u043d\u0443, \u0447\u0442\u043e\u0431\u044b \u0432\u0441\u0435 <strong>\u0444\u0443\u043d\u043a\u0446\u0438\u0438 \u0430\u043a\u0442\u0438\u0432\u0430\u0446\u0438\u0438,<\/strong> \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c\u044b\u0435 \u0432 \u043e\u0431\u044b\u0447\u043d\u044b\u0445 CNN, \u043f\u043e\u0441\u043b\u0435 \u044d\u0442\u043e\u0433\u043e \u043c\u043e\u0433\u043b\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c\u0441\u044f \u0432 \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0445 AdderNets.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0425\u043e\u0442\u044f \u0441\u043b\u043e\u0439 <a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a> \u0432\u043a\u043b\u044e\u0447\u0430\u0435\u0442 \u0432 \u0441\u0435\u0431\u044f \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f, \u0435\u0433\u043e \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0435 \u0437\u0430\u0442\u0440\u0430\u0442\u044b \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043d\u0438\u0436\u0435, \u0447\u0435\u043c \u0443 \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u044b\u0445 \u0441\u043b\u043e\u0435\u0432, \u0438 \u0438\u043c\u0438 \u043c\u043e\u0436\u043d\u043e \u043f\u0440\u0435\u043d\u0435\u0431\u0440\u0435\u0447\u044c.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>(\u041f\u043e\u044f\u0432\u044f\u0442\u0441\u044f \u043b\u0438 \u0432 \u0431\u0443\u0434\u0443\u0449\u0435\u043c \u043a\u0430\u043a\u0438\u0435-\u043d\u0438\u0431\u0443\u0434\u044c <a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a>, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0438\u0435 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0435?)<\/p>\n<\/li>\n<\/ul>\n<h3>2.2. \u041f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u044b\u0435<\/h3>\n<ul>\n<li>\n<p>\u041f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u0430\u044f <em>l<\/em>1-\u043c\u0435\u0440\u044b \u043d\u0435 \u043f\u043e\u0434\u0445\u043e\u0434\u0438\u0442 \u0434\u043b\u044f \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043d\u043e\u0433\u043e \u0441\u043f\u0443\u0441\u043a\u0430. \u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u043c\u044b \u0440\u0430\u0441\u0441\u043c\u0430\u0442\u0440\u0438\u0432\u0430\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u0443\u044e <em>l<\/em>2-\u043c\u0435\u0440\u044b:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 \u0442\u043e\u0447\u043d\u043e\u0433\u043e \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u0430 \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u0442\u043e\u0447\u043d\u043e \u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0442\u044c \u0444\u0438\u043b\u044c\u0442\u0440\u044b.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0427\u0442\u043e\u0431\u044b \u0438\u0437\u0431\u0435\u0436\u0430\u0442\u044c \u0432\u0437\u0440\u044b\u0432\u0430 \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u0430, \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442 <em>X<\/em> \u043e\u0431\u0440\u0435\u0437\u0430\u0435\u0442\u0441\u044f \u0434\u043e [-1,1].<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0417\u0430\u0442\u0435\u043c \u0432\u044b\u0447\u0438\u0441\u043b\u044f\u0435\u0442\u0441\u044f \u0447\u0430\u0441\u0442\u043d\u0430\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u043d\u0430\u044f \u0432\u044b\u0445\u043e\u0434\u043d\u044b\u0445 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432 <em>Y<\/em> \u043f\u043e \u043e\u0442\u043d\u043e\u0448\u0435\u043d\u0438\u044e \u043a \u0432\u0445\u043e\u0434\u043d\u044b\u043c \u0445\u0430\u0440\u0430\u043a\u0442\u0435\u0440\u0438\u0441\u0442\u0438\u043a\u0430\u043c <em>X<\/em> \u043a\u0430\u043a:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>HT <\/em>&#8212; \u0444\u0443\u043d\u043a\u0446\u0438\u044f HardTanh:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<h3>2.3. \u0421\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f<\/h3>\n<figure class=\"\"><figcaption>l2-\u043c\u0435\u0440\u044b \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u0432 \u0432 LeNet-5-BN<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u041a\u0430\u043a \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043e \u0432 \u044d\u0442\u043e\u0439 \u0442\u0430\u0431\u043b\u0438\u0446\u0435, \u043c\u0435\u0440\u044b \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u0432 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 AdderNets \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043c\u0435\u043d\u044c\u0448\u0435, \u0447\u0435\u043c \u0432 CNN, \u0447\u0442\u043e \u043c\u043e\u0436\u0435\u0442 \u0437\u0430\u043c\u0435\u0434\u043b\u0438\u0442\u044c \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u0435 \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 AdderNets.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0412 AdderNets \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f \u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0440\u0430\u0437\u043d\u044b\u0445 \u0443\u0440\u043e\u0432\u043d\u0435\u0439:<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>\u03b3<\/em> &#8212; \u0433\u043b\u043e\u0431\u0430\u043b\u044c\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0432\u0441\u0435\u0439 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u043e\u0439 \u0441\u0435\u0442\u0438 (\u043d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u0434\u043b\u044f \u0441\u0443\u043c\u043c\u0430\u0442\u043e\u0440\u0430 \u0438 <a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a> \u0441\u043b\u043e\u0435\u0432), \u0394<em>L<\/em>(<em>Fl<\/em>) &#8212; \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442 \u0444\u0438\u043b\u044c\u0442\u0440\u0430 \u0432 \u0441\u043b\u043e\u0435 <em>l,<\/em> \u0430 <em>\u03b1l<\/em> &#8212; \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0430\u044f \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u0430\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0430 \u043a\u0430\u043a<\/p>\n<\/li>\n<\/ul>\n<figure class=\"\"><figcaption><\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0433\u0434\u0435 <em>k<\/em> \u043e\u0431\u043e\u0437\u043d\u0430\u0447\u0430\u0435\u0442 \u043a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u043e\u0432 \u0432 <em>Fl<\/em>, \u0430 <em>\u03b7<\/em> &#8212; \u0433\u0438\u043f\u0435\u0440\u043f\u0430\u0440\u0430\u043c\u0435\u0442\u0440 \u0434\u043b\u044f \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c\u044e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0441\u0443\u043c\u043c\u0430\u0442\u043e\u0440\u0430.<\/p>\n<\/li>\n<\/ul>\n<h2>3. \u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u043e\u0432<\/h2>\n<h3>3.1. MNIST<\/h3>\n<ul>\n<li>\n<p><a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>-5-<a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN<\/u><\/a> \u043e\u0431\u0443\u0447\u0435\u043d\u0430.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>CNN <\/strong>\u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442<strong> \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 99,4%<\/strong> \u043f\u0440\u0438 <strong>435K \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0439 <\/strong>\u0438 <strong>435K \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0439<\/strong>.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0417\u0430\u043c\u0435\u043d\u044f\u044f \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0432 \u0441\u0432\u0435\u0440\u0442\u043a\u0435 \u043d\u0430 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f, \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u0430\u044f <strong>AdderNet <\/strong>\u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442<strong> \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 99,4%<\/strong>, \u0442\u0430\u043a\u043e\u0439 \u0436\u0435 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c \u043a\u0430\u043a \u0443 CNN, \u0441<strong> 870K \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f\u043c\u0438<\/strong> \u0438 <strong>\u043f\u043e\u0447\u0442\u0438 \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0439<\/strong>.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0422\u0435\u043e\u0440\u0435\u0442\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0430 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0432 \u0426\u041f \u0442\u0430\u043a\u0436\u0435 \u0431\u043e\u043b\u044c\u0448\u0435, \u0447\u0435\u043c \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0430 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0438 \u0432\u044b\u0447\u0438\u0442\u0430\u043d\u0438\u044f.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u043d\u0430 \u043c\u043e\u0434\u0435\u043b\u0438 VIA Nano 2000 \u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0430 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0438 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0441 \u043f\u043b\u0430\u0432\u0430\u044e\u0449\u0435\u0439 \u0437\u0430\u043f\u044f\u0442\u043e\u0439 \u0441\u043e\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 4 \u0438 2 \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0435\u043d\u043d\u043e. <strong>AdderNet <\/strong>\u0441 \u043c\u043e\u0434\u0435\u043b\u044c\u044e <a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>-5 \u0431\u0443\u0434\u0435\u0442 \u0438\u043c\u0435\u0442\u044c <strong>\u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0443<\/strong> <strong>1.7M<\/strong>, \u0432 \u0442\u043e \u0432\u0440\u0435\u043c\u044f \u043a\u0430\u043a <strong>CNN <\/strong>\u0431\u0443\u0434\u0435\u0442 \u0438\u043c\u0435\u0442\u044c <strong>\u0437\u0430\u0434\u0435\u0440\u0436\u043a\u0443<\/strong> <strong>2.6M <\/strong>\u043d\u0430 \u0442\u043e\u043c \u0436\u0435 CPU.<\/p>\n<\/li>\n<\/ul>\n<h3>3.2. CIFAR<\/h3>\n<figure class=\"full-width\"><figcaption>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 CIFAR-10 \u0438 CIFAR-100<\/figcaption><\/figure>\n<figure class=\"\"><figcaption>BNN: \u0441\u0432\u0435\u0440\u0442\u043a\u0430 XNORNet, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0430\u044f \u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u043e\u0439 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 XNOR<\/figcaption><\/figure>\n<ul>\n<li>\n<p><strong>\u0414\u0432\u043e\u0438\u0447\u043d\u044b\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u044b\u0435 \u0441\u0435\u0442\u0438 (Binary neural networks &#8212; BNN)<\/strong>: \u043c\u043e\u0433\u0443\u0442 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 <strong>XNOR <\/strong>\u0434\u043b\u044f \u0437\u0430\u043c\u0435\u043d\u044b \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f, \u0447\u0442\u043e \u043c\u044b \u0442\u0430\u043a\u0436\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0434\u043b\u044f \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044f.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>\u0414\u043b\u044f \u043c\u043e\u0434\u0435\u043b\u0438 <\/strong><a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-vggnet-1st-runner-up-of-ilsvlc-2014-image-classification-d02355543a11?source=post_page---------------------------\"><strong><u>VGG<\/u><\/strong><\/a><strong>-small, AdderNets \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 \u043f\u043e\u0447\u0442\u0438 \u0442\u0430\u043a\u0438\u0445 \u0436\u0435 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 (93,72% \u0432 CIFAR-10 \u0438 72,64% \u0432 CIFAR-100) \u043a\u0430\u043a \u0438 CNNs (93,80% \u0432 CIFAR-10 \u0438 72,73% \u0432 CIFAR-100).<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0425\u043e\u0442\u044f \u0440\u0430\u0437\u043c\u0435\u0440 \u043c\u043e\u0434\u0435\u043b\u0438 BNN \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043c\u0435\u043d\u044c\u0448\u0435, \u0447\u0435\u043c \u0443 AdderNet \u0438 CNN, \u0435\u0435 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043d\u0438\u0436\u0435 (89,80% \u0432 CIFAR-10 \u0438 65,41% \u0432 CIFAR-100).<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0427\u0442\u043e \u043a\u0430\u0441\u0430\u0435\u0442\u0441\u044f <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet-<\/u><\/a>20, CNN \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u043d\u0430\u0438\u0432\u044b\u0441\u0448\u0435\u0439 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 (\u0442.\u0435. 92,25% \u0432 CIFAR-10 \u0438 68,14% \u0432 CIFAR-100), \u043d\u043e \u0441 \u0431\u043e\u043b\u044c\u0448\u0438\u043c \u0447\u0438\u0441\u043b\u043e\u043c \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0439 (41,17M).<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>\u041f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0435 AdderNets \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 91,84% \u0432 CIFAR-10 \u0438 67,60% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 \u0432 CIFAR-100 \u0431\u0435\u0437 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f, \u0447\u0442\u043e \u0441\u0440\u0430\u0432\u043d\u0438\u043c\u043e \u0441 CNN.<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041d\u0430\u043f\u0440\u043e\u0442\u0438\u0432, BNN \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 \u0442\u043e\u043b\u044c\u043a\u043e 84,87% \u0438 54,14% \u0432 CIFAR-10 \u0438 CIFAR-100.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b <\/strong><a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><strong><u>ResNet-<\/u><\/strong><\/a><strong>32 \u0442\u0430\u043a\u0436\u0435 \u043f\u0440\u0435\u0434\u043f\u043e\u043b\u0430\u0433\u0430\u044e\u0442, \u0447\u0442\u043e \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0435 AdderNets \u043c\u043e\u0433\u0443\u0442 \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0442\u044c \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 \u0430\u043d\u0430\u043b\u043e\u0433\u0438\u0447\u043d\u044b\u0445 \u043e\u0431\u044b\u0447\u043d\u044b\u043c CNN.<\/strong><\/p>\n<\/li>\n<\/ul>\n<h3>3.3. ImageNet<\/h3>\n<figure class=\"full-width\"><figcaption>\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u0432 \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0430\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 ImageNet&nbsp;<\/figcaption><\/figure>\n<ul>\n<li>\n<p>CNN \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 69,8% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-1 \u0438 89,1% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-5 \u0432 <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>RESNET<\/u><\/a>-18. \u041e\u0434\u043d\u0430\u043a\u043e, \u043f\u0440\u0438 1.8G \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u044f\u0445.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p><strong>AdderNet \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0438\u0432\u0430\u0435\u0442 66,8% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-1 \u0438 87,4% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-5 \u0432 <\/strong><a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><strong><u>ResNet-<\/u><\/strong><\/a><strong>18, \u0447\u0442\u043e \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u0435\u0442, \u0447\u0442\u043e \u0444\u0438\u043b\u044c\u0442\u0440\u044b \u0441\u0443\u043c\u043c\u0430\u0442\u043e\u0440\u0430 \u043c\u043e\u0433\u0443\u0442 \u0438\u0437\u0432\u043b\u0435\u043a\u0430\u0442\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0443\u044e \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u0438\u0437 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439.<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041d\u0435\u0441\u043c\u043e\u0442\u0440\u044f \u043d\u0430 \u0442\u043e, \u0447\u0442\u043e BNN \u043c\u043e\u0436\u0435\u0442 \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0442\u044c \u0432\u044b\u0441\u043e\u043a\u043e\u0439 \u0441\u0442\u0435\u043f\u0435\u043d\u0438 \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f \u0438 \u0441\u0436\u0430\u0442\u0438\u044f, \u043e\u043d \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 \u0442\u043e\u043b\u044c\u043a\u043e 51,2% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-1 \u0438 73,2% \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 top-5 \u0432 <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet-<\/u><\/a>18.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0410\u043d\u0430\u043b\u043e\u0433\u0438\u0447\u043d\u044b\u0435 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0434\u043b\u044f \u0431\u043e\u043b\u0435\u0435 \u0433\u043b\u0443\u0431\u043e\u043a\u043e\u0433\u043e <a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet<\/u><\/a>-50.<\/p>\n<\/li>\n<\/ul>\n<h3>3.4. \u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0432\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438<\/h3>\n<figure class=\"full-width\"><figcaption>\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432 \u0432 AdderNets \u0438 CNN. \u041f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 CNN \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432 \u0440\u0430\u0437\u0434\u0435\u043b\u0435\u043d\u044b \u043f\u043e \u0438\u0445 \u0443\u0433\u043b\u0430\u043c.<\/figcaption><\/figure>\n<ul>\n<li>\n<p>&nbsp;<a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>++ \u043e\u0431\u0443\u0447\u0430\u043b\u0441\u044f \u043d\u0430 \u043d\u0430\u0431\u043e\u0440\u0435 \u0434\u0430\u043d\u043d\u044b\u0445 MNIST, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0438\u043c\u0435\u0435\u0442 \u0448\u0435\u0441\u0442\u044c \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u044b\u0445 \u0441\u043b\u043e\u0435\u0432 \u0438 \u043f\u043e\u043b\u043d\u043e\u0441\u0432\u044f\u0437\u043d\u044b\u0439 \u0441\u043b\u043e\u0439 \u0434\u043b\u044f \u0438\u0437\u0432\u043b\u0435\u0447\u0435\u043d\u0438\u044f \u0432\u044b\u0440\u0430\u0436\u0435\u043d\u043d\u044b\u0445 3D \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u043d\u0435\u0439\u0440\u043e\u043d\u043e\u0432 \u0432 \u043a\u0430\u0436\u0434\u043e\u043c \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u043e\u043c \u0441\u043b\u043e\u0435 \u0441\u043e\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 32, 32, 64, 64, 128, 128 \u0438 2 \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0435\u043d\u043d\u043e.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>AdderNets \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442 <em>l<\/em>1-\u043c\u0435\u0440\u0443 \u0434\u043b\u044f \u0440\u0430\u0437\u043b\u0438\u0447\u0435\u043d\u0438\u044f \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432. \u041f\u0440\u0438\u0437\u043d\u0430\u043a\u0438 \u0438\u043c\u0435\u044e\u0442 \u0442\u0435\u043d\u0434\u0435\u043d\u0446\u0438\u044e \u0431\u044b\u0442\u044c \u0441\u0433\u0440\u0443\u043f\u043f\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u043c\u0438 \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0446\u0435\u043d\u0442\u0440\u043e\u0432 \u0440\u0430\u0437\u043d\u044b\u0445 \u043a\u043b\u0430\u0441\u0441\u043e\u0432.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0432\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0442, \u0447\u0442\u043e \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0435 AdderNets \u043c\u043e\u0433\u0443\u0442 \u043e\u0431\u043b\u0430\u0434\u0430\u0442\u044c \u0430\u043d\u0430\u043b\u043e\u0433\u0438\u0447\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e \u0440\u0430\u0441\u043f\u043e\u0437\u043d\u0430\u0432\u0430\u043d\u0438\u044f \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439 \u043a\u0430\u043a \u0438 CNN.<\/p>\n<\/li>\n<\/ul>\n<figure class=\"full-width\"><figcaption>\u0412\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u0444\u0438\u043b\u044c\u0442\u0440\u043e\u0432 \u0432 \u043f\u0435\u0440\u0432\u043e\u043c \u0441\u043b\u043e\u0435 LeNet-5-BN \u043d\u0430 MNIST<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0424\u0438\u043b\u044c\u0442\u0440\u044b \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c\u044b\u0445 adderNets \u043f\u043e-\u043f\u0440\u0435\u0436\u043d\u0435\u043c\u0443 \u0438\u043c\u0435\u044e\u0442 \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441\u0445\u043e\u0436\u0438\u0435 \u043f\u0430\u0442\u0442\u0435\u0440\u043d\u044b \u0441\u043e \u0441\u0432\u0435\u0440\u0442\u043e\u0447\u043d\u044b\u043c\u0438 \u0444\u0438\u043b\u044c\u0442\u0440\u0430\u043c\u0438.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u042d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u044b \u043f\u043e \u0432\u0438\u0437\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 \u0434\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u044e\u0442, \u0447\u0442\u043e \u0444\u0438\u043b\u044c\u0442\u0440\u044b AdderNets \u043c\u043e\u0433\u0443\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0438\u0437\u0432\u043b\u0435\u043a\u0430\u0442\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0443\u044e \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u0438\u0437 \u0432\u0445\u043e\u0434\u043d\u044b\u0445 \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439 \u0438 \u043f\u0440\u0438\u0437\u043d\u0430\u043a\u043e\u0432.<\/p>\n<\/li>\n<\/ul>\n<figure class=\"full-width\"><figcaption>\u0413\u0438\u0441\u0442\u043e\u0433\u0440\u0430\u043c\u043c\u044b \u043f\u043e \u0432\u0435\u0441\u0430\u043c \u0441 AdderNet (\u0441\u043b\u0435\u0432\u0430) \u0438 CNN (\u0441\u043f\u0440\u0430\u0432\u0430).<\/figcaption><\/figure>\n<ul>\n<li>\n<p>\u0420\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u0432\u0435\u0441\u043e\u0432 \u0441 AdderNets \u0431\u043b\u0438\u0437\u043a\u043e \u043a \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u044e \u041b\u0430\u043f\u043b\u0430\u0441\u0430, \u0442\u043e\u0433\u0434\u0430 \u043a\u0430\u043a \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u0441 CNN \u0431\u043e\u043b\u044c\u0448\u0435 \u043f\u043e\u0445\u043e\u0434\u0438\u0442 \u0431\u043e\u043b\u044c\u0448\u0435 \u043d\u0430 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u0413\u0430\u0443\u0441\u0441\u0430. \u0424\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438, \u0430\u043f\u0440\u0438\u043e\u0440\u043d\u044b\u043c \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435\u043c <em>l<\/em>1-\u043c\u0435\u0440\u044b \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u041b\u0430\u043f\u043b\u0430\u0441\u0430.<\/p>\n<\/li>\n<\/ul>\n<h3>3.5. \u0410\u0431\u043b\u044f\u0446\u0438\u043e\u043d\u043d\u043e\u0435 \u0438\u0441\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u043d\u0438\u0435&nbsp;<\/h3>\n<figure class=\"full-width\"><figcaption>\u041a\u0440\u0438\u0432\u0430\u044f \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f AdderNets \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0445 \u0441\u0445\u0435\u043c \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u0438<\/figcaption><\/figure>\n<ul>\n<li>\n<p><strong>AdderNets, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0449\u0438\u0435 \u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0443\u044e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f (adaptive learning rate &#8212; ALR) \u0438 \u0443\u0432\u0435\u043b\u0438\u0447\u0435\u043d\u043d\u0443\u044e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f (increased learning rate &#8212; ILR), \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u044e\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 97,99% \u0438 97,72% \u0441\u043e \u0437\u043d\u0430\u043a\u043e\u0432\u044b\u043c \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043e\u043c, \u0447\u0442\u043e \u043d\u0430\u043c\u043d\u043e\u0433\u043e \u043d\u0438\u0436\u0435, \u0447\u0435\u043c \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c CNN (99,40%) .<\/strong><\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u043c\u044b \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u0442\u043e\u0447\u043d\u044b\u0439 \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442 \u0434\u043b\u044f \u0431\u043e\u043b\u0435\u0435 \u0442\u043e\u0447\u043d\u043e\u0433\u043e \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u044f \u0432\u0435\u0441\u043e\u0432 \u0432 AdderNets.<\/p>\n<\/li>\n<\/ul>\n<ul>\n<li>\n<p>\u0412 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0435 AdderNet \u0441 ILR \u0434\u043e\u0441\u0442\u0438\u0433\u0430\u0435\u0442 \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 98,99% \u043f\u0440\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 <strong>\u0442\u043e\u0447\u043d\u043e\u0433\u043e \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u0430<\/strong>. \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f <strong>\u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0443\u044e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f (ALR)<\/strong>, <strong>AdderNet \u043c\u043e\u0436\u0435\u0442 \u0434\u043e\u0441\u0442\u0438\u0447\u044c \u0442\u043e\u0447\u043d\u043e\u0441\u0442\u0438 99,40%<\/strong>, \u0447\u0442\u043e \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0438\u0440\u0443\u0435\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u044c \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u043d\u043e\u0433\u043e \u043c\u0435\u0442\u043e\u0434\u0430.<\/p>\n<\/li>\n<\/ul>\n<h3>\u0421\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u0441\u0442\u0430\u0442\u044c\u044e<\/h3>\n<p>[2020 CVPR] [AdderNet]<\/p>\n<p><a href=\"https:\/\/openaccess.thecvf.com\/content_CVPR_2020\/papers\/Chen_AdderNet_Do_We_Really_Need_Multiplications_in_Deep_Learning_CVPR_2020_paper.pdf\"><u>AdderNet: Do We Really Need Multiplications in Deep Learning?<\/u><\/a><\/p>\n<h3>\u041a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u044f \u0438\u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u0438\u0439<\/h3>\n<p><strong>1989\u20131998<\/strong>: [<a href=\"https:\/\/medium.com\/@sh.tsang\/paper-brief-review-of-lenet-1-lenet-4-lenet-5-boosted-lenet-4-image-classification-1f5f809dbf17?source=post_page---------------------------\"><u>LeNet<\/u><\/a>]<\/p>\n<p><strong>2012\u20132014<\/strong>: [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-alexnet-caffenet-winner-in-ilsvrc-2012-image-classification-b93598314160?source=post_page---------------------------\"><u>AlexNet &amp; CaffeNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-dropout-a-simple-way-to-prevent-neural-networks-from-overfitting-image-classification-a74b369b4b8e\"><u>Dropout<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-maxout-network-image-classification-40ecd77f7ce4?source=post_page---------------------------\"><u>Maxout<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-nin-network-in-network-image-classification-69e271e499ee?source=post_page---------------------------\"><u>NIN<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-zfnet-the-winner-of-ilsvlc-2013-image-classification-d1a5a0c45103?source=post_page---------------------------\"><u>ZFNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/review-sppnet-1st-runner-up-object-detection-2nd-runner-up-image-classification-in-ilsvrc-906da3753679?source=post_page---------------------------\"><u>SPPNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-model-distillation-distilling-the-knowledge-in-a-neural-network-image-classification-48ce0c81618a\"><u>Distillation<\/u><\/a>]<\/p>\n<p><strong>2015<\/strong>: [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-vggnet-1st-runner-up-of-ilsvlc-2014-image-classification-d02355543a11?source=post_page---------------------------\"><u>VGGNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-highway-networks-gating-function-to-highway-image-classification-5a33833797b5?source=post_page---------------------------\"><u>Highway<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/review-prelu-net-the-first-to-surpass-human-level-performance-in-ilsvrc-2015-image-f619dddd5617?source=post_page---------------------------\"><u>PReLU-Net<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-stn-spatial-transformer-network-image-classification-d3cbd98a70aa?source=post_page---------------------------\"><u>STN<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-deep-image-a-big-data-solution-for-image-recognition-99e5f7b1c802?source=post_page---------------------------\"><u>DeepImage<\/u><\/a>] [<a href=\"https:\/\/medium.com\/coinmonks\/paper-review-of-googlenet-inception-v1-winner-of-ilsvlc-2014-image-classification-c2b3565a64e7?source=post_page---------------------------\"><u>GoogLeNet \/ Inception-v1<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-batch-normalization-inception-v2-bn-inception-the-2nd-to-surpass-human-level-18e2d0f56651?source=post_page---------------------------\"><u>BN-Inception \/ Inception-v2<\/u><\/a>]<\/p>\n<p><strong>2016<\/strong>: [<a href=\"https:\/\/towardsdatascience.com\/review-squeezenet-image-classification-e7414825581a?source=post_page---------------------------\"><u>SqueezeNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-inception-v3-1st-runner-up-image-classification-in-ilsvrc-2015-17915421f77c?source=post_page---------------------------\"><u>Inception-v3<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8?source=post_page---------------------------\"><u>ResNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/resnet-with-identity-mapping-over-1000-layers-reached-image-classification-bb50a42af03e?source=post_page---------------------------\"><u>Pre-Activation ResNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-rir-resnet-in-resnet-image-classification-be4c79fde8ba?source=post_page---------------------------\"><u>RiR<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-stochastic-depth-image-classification-a4e225807f4a?source=post_page---------------------------\"><u>Stochastic Depth<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-wrns-wide-residual-networks-image-classification-d3feb3fb2004?source=post_page---------------------------\"><u>WRN<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-trimps-soushen-winner-in-ilsvrc-2016-image-classification-dfbc423111dd?source=post_page---------------------------\"><u>Trimps-Soushen<\/u><\/a>]<\/p>\n<p><strong>2017<\/strong>: [<a href=\"https:\/\/towardsdatascience.com\/review-inception-v4-evolved-from-googlenet-merged-with-resnet-idea-image-classification-5e8c339d18bc?source=post_page---------------------------\"><u>Inception-v4<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-xception-with-depthwise-separable-convolution-better-than-inception-v3-image-dc967dd42568?source=post_page---------------------------\"><u>Xception<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-mobilenetv1-depthwise-separable-convolution-light-weight-model-a382df364b69?source=post_page---------------------------\"><u>MobileNetV1<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-shake-shake-regularization-image-classification-d22bb8587953?source=post_page---------------------------\"><u>Shake-Shake<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-cutout-improved-regularization-of-convolutional-neural-networks-image-classification-d39ff4ec3c76\"><u>Cutout<\/u><\/a>] [<a href=\"https:\/\/medium.com\/datadriveninvestor\/review-fractalnet-image-classification-c5bdd855a090?source=post_page---------------------------\"><u>FractalNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-polynet-2nd-runner-up-in-ilsvrc-2016-image-classification-8a1a941ce9ea?source=post_page---------------------------\"><u>PolyNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-resnext-1st-runner-up-of-ilsvrc-2016-image-classification-15d7f17b42ac?source=post_page---------------------------\"><u>ResNeXt<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-densenet-image-classification-b6631a8ef803?source=post_page---------------------------\"><u>DenseNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-pyramidnet-deep-pyramidal-residual-networks-image-classification-85a87b60ae78?source=post_page---------------------------\"><u>PyramidNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-drn-dilated-residual-networks-image-classification-semantic-segmentation-d527e1a8fb5?source=post_page---------------------------\"><u>DRN<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-dpn-dual-path-networks-image-classification-d0135dce8817?source=post_page---------------------------\"><u>DPN<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-residual-attention-network-attention-aware-features-image-classification-7ae44c4f4b8?source=post_page---------------------------\"><u>Residual Attention Network<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-igcnet-igcv1-interleaved-group-convolutions-image-classification-7421d2a1dede?source=post_page---------------------------\"><u>IGCNet \/ IGCV1<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-deep-roots-improving-cnn-efficiency-with-hierarchical-filter-groups-image-9aba67f23b27\"><u>Deep Roots<\/u><\/a>]<\/p>\n<p><strong>2018<\/strong>: [<a href=\"https:\/\/towardsdatascience.com\/review-ror-resnet-of-resnet-multilevel-resnet-image-classification-cd3b0fcc19bb?source=post_page---------------------------\"><u>RoR<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-dmrnet-dfn-mr-merge-and-run-mappings-image-classification-493080a4b8ae?source=post_page---------------------------\"><u>DMRNet \/ DFN-MR<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-msdnet-multi-scale-dense-networks-image-classification-4d949955f6d5?source=post_page---------------------------\"><u>MSDNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-shufflenet-v1-light-weight-model-image-classification-5b253dfe982f?source=post_page---------------------------\"><u>ShuffleNet V1<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-senet-squeeze-and-excitation-network-winner-of-ilsvrc-2017-image-classification-a887b98b2883?source=post_page---------------------------\"><u>SENet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-nasnet-neural-architecture-search-network-image-classification-23139ea0425d?source=post_page---------------------------\"><u>NASNet<\/u><\/a>] [<a href=\"https:\/\/towardsdatascience.com\/review-mobilenetv2-light-weight-model-image-classification-8febb490e61c?source=post_page---------------------------\"><u>MobileNetV2<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/review-condensenet-improve-densenet-with-learned-group-convolution-image-classification-85de829781db\"><u>CondenseNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-igcv2-interleaved-structured-sparse-convolution-image-classification-59370cd5e19e\"><u>IGCV2<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-igcv3-interleaved-low-rank-group-convolutions-image-classification-c5fd120d7cf1\"><u>IGCV3<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-fishnet-fishnext-a-versatile-backbone-for-image-region-and-pixel-level-prediction-a5f9aa91f96b\"><u>FishNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-squeezenext-hardware-aware-neural-network-design-image-classification-3fc8d1d3f76\"><u>SqueezeNext<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-enas-efficient-neural-architecture-search-via-parameter-sharing-image-classification-418b2a067bba\"><u>ENAS<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-pnasnet-progressive-neural-architecture-search-image-classification-1beb1de06fe6\"><u>PNASNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-shufflenet-v2-practical-guidelines-for-e-fficient-cnn-architecture-design-image-287b05abc08a\"><u>ShuffleNet V2<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-bam-bottleneck-attention-module-image-classification-439a2456c1a1\"><u>BAM<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-cbam-convolutional-block-attention-module-image-classification-ddbaf10f7430\"><u>CBAM<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/reading-morphnet-fast-simple-resource-constrained-structure-learning-of-deep-networks-image-89caa0b9f18b\"><u>MorphNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-netadapt-platform-aware-neural-network-adaptation-for-mobile-applications-image-84d7210bb828\"><u>NetAdapt<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-mixup-beyond-empirical-risk-minimization-image-classification-6ee40a45ad17\"><u>mixup<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/dropblock-a-regularization-method-for-convolutional-networks-image-classification-14556bb13489\"><u>DropBlock<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-group-norm-gn-group-normalization-image-classification-5f7fe0f58eb6\"><u>Group Norm (GN)<\/u><\/a>]<\/p>\n<p><strong>2019<\/strong>: [<a href=\"https:\/\/medium.com\/@sh.tsang\/resnet-38-wider-or-deeper-resnet-image-classification-semantic-segmentation-f297f2f73437\"><u>ResNet-38<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-amoebanet-regularized-evolution-for-image-classifier-architecture-search-image-278f5c077a4a\"><u>AmoebaNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/@sh.tsang\/reading-espnetv2-a-light-weight-power-efficient-and-general-purpose-convolutional-neural-a57b5fbfeb84\"><u>ESPNetv2<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/reading-mnasnet-platform-aware-neural-architecture-search-for-mobile-image-classification-b042aaef66f7\"><u>MnasNet<\/u><\/a>] [<a href=\"https:\/\/medium.com\/carre4\/paper-single-path-nas-designing-hardware-efficient-convnets-in-less-than-4-hours-image-classifi-81cff87f223e\"><u>Single-Path NAS<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-darts-differentiable-architecture-search-image-classification-dda80703b81a\"><u>DARTS<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-proxylessnas-direct-neural-architecture-search-on-target-task-image-classification-73c35ebd8aed\"><u>ProxylessNAS<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-mobilenetv3-searching-for-mobilenetv3-image-classification-5072d4d8703c\"><u>MobileNetV3<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-fbnet-hardware-aware-efficient-convnet-design-via-differentiable-neural-architecture-f95a20c1cd4a\"><u>FBNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-shakedrop-shakedrop-regularization-for-deep-residual-learning-image-classification-d2b83d0ae3de\"><u>ShakeDrop<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/paper-cutmix-regularization-strategy-to-train-strong-classifiers-with-localizable-features-5527e29c4890\"><u>CutMix<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/mixconv-mixed-depthwise-convolutional-kernels-image-classification-9d0ca8bf4093\"><u>MixConv<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/efficientnet-rethinking-model-scaling-for-convolutional-neural-networks-image-classification-ef67b0f14a4d\"><u>EfficientNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-abn-attention-branch-network-for-visual-explanation-image-classification-30fd164960ee\"><u>ABN<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-sknet-selective-kernel-networks-image-classification-63ebbad7d78f\"><u>SKNet<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-cb-loss-class-balanced-loss-based-on-effective-number-of-samples-image-classification-3056a1a1a001\"><u>CB Loss<\/u><\/a>]<\/p>\n<p><strong>2020<\/strong>: [<a href=\"https:\/\/sh-tsang.medium.com\/random-erasing-re-random-erasing-data-augmentation-image-classification-37f11627b38\"><u>Random Erasing (RE)<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-saol-spatially-attentive-output-layer-image-classification-weakly-supervised-object-742e2eeb3226\"><u>SAOL<\/u><\/a>] [<a href=\"https:\/\/sh-tsang.medium.com\/review-addernet-do-we-really-need-multiplications-in-deep-learning-image-classification-b72851ddb255\"><u>AdderNet<\/u><\/a>]<\/p>\n<hr>\n<blockquote>\n<p><em>\u041f\u0435\u0440\u0435\u0432\u043e\u0434 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430 \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043b\u0435\u043d \u0432 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 <\/em><a href=\"https:\/\/otus.pw\/lobO\/\"><strong><em>&#171;Deep Learning. Basic&#187;<\/em><\/strong><\/a><em>. <\/em><\/p>\n<p><em>\u0422\u0430\u043a\u0436\u0435 \u043f\u0440\u0438\u0433\u043b\u0430\u0448\u0430\u0435\u043c \u0432\u0441\u0435\u0445 \u0436\u0435\u043b\u0430\u044e\u0449\u0438\u0445 \u043f\u043e\u0441\u0435\u0442\u0438\u0442\u044c <\/em><a href=\"https:\/\/otus.pw\/BCh8\/\"><strong><em>\u0431\u0435\u0441\u043f\u043b\u0430\u0442\u043d\u044b\u0439 \u0434\u0435\u043c\u043e-\u0443\u0440\u043e\u043a<\/em><\/strong><\/a><em> \u043f\u043e \u0442\u0435\u043c\u0435: &#171;Knowledge distillation: \u043d\u0435\u0439\u0440\u043e\u0441\u0435\u0442\u0438 \u043e\u0431\u0443\u0447\u0430\u044e\u0442 \u043d\u0435\u0439\u0440\u043e\u0441\u0435\u0442\u0438&#187;.<\/em><\/p>\n<p><strong> &#8212; <\/strong><a href=\"https:\/\/otus.pw\/lobO\/\"><strong>\u0423\u0417\u041d\u0410\u0422\u042c \u041f\u041e\u0414\u0420\u041e\u0411\u041d\u0415\u0415 \u041e \u041a\u0423\u0420\u0421\u0415<\/strong><\/a><\/p>\n<p><strong> &#8212; <\/strong><a href=\"https:\/\/otus.pw\/BCh8\/\"><strong>\u0417\u0410\u041f\u0418\u0421\u0410\u0422\u042c\u0421\u042f \u041d\u0410 \u0411\u0415\u0421\u041f\u041b\u0410\u0422\u041d\u042b\u0419 \u0414\u0415\u041c\u041e-\u0423\u0420\u041e\u041a<\/strong><\/a><\/p>\n<\/blockquote>\n<\/div>\n<p> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/551446\/\"> https:\/\/habr.com\/ru\/company\/otus\/blog\/551446\/<\/a><br \/><\/br><\/br><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-321129","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/321129","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=321129"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/321129\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=321129"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=321129"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=321129"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}