INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property :::` , 1111111011110111011111111111011111010111111111101110111111111111111111111111110111111 Organization International Bureau (10) International Publication Number (43) International Publication Date .....•\'\' WO 2018/080624 Al 03 May 2018 (03.05.2018) W I PO I PCT (51) International Patent Classification: NARAYANASWAMI, Ravi; 1600 Amphitheatre Park- G06N 3/10 (2006.01) way, Mountain View, California 94043 (US). (21) International Application Number: (74) Agent: HENRY, Joel et al.; Fish & Richardson P.C., P.O. PCT/US2017/047992 Box 1022, Minneapolis, Minnesota 55440-1022 (US). (22) International Filing Date: (81) Designated States (unless otherwise indicated, for every 22 August 2017 (22.08.2017) kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, (25) Filing Language: English CA, CH, CL, CN, CO, CR, CU, CZ, DE, DJ, DK, DM, DO, (26) Publication Language: English DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, (30) Priority Data: HR, HU, ID, IL, IN, IR, IS, JO, JP, KE, KG, KH, KN, KP, 15/336,066 27 October 2016 (27.10.2016) US KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, 15/465,774 22 March 2017 (22.03.2017) US MG, MK, MN, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, RU, RW, SA, (71) Applicant: GOOGLE LLC [US/US]; 1600 Amphitheatre SC, SD, SE, SG, SK, SL, SM, ST, SV, SY, TH, TJ, TM, TN, Parkway, Mountain View, California 94043 (US). TR, TT, TZ, UA, UG, US, UZ, VC, VN, ZA, ZM, ZW. (72) Inventors: WOO, Dong Hyuk; 1600 Amphitheatre (84) Designated States (unless otherwise indicated, for every Parkway, Mountain View, California 94043 (US). kind of regional protection available): ARIPO (BW, GH, (54) Title: EXPLOITING INPUT DATA SPARSITY IN NEURAL NETWORK COMPUTE UNITS 300-- INSTRUCTIONS, INPUT ACTIVATIONS, AND WEIGHTS/PARAMETERS 303--- t Bitmap 1 1 ° 1 1 1 6 1 1 1 ° 1 1 1 ° ( 1 ) ( ) 3 + ( ) 5 ( ) 7 Weights and Partial Sums (Second Memory 110) Controller 302 310 First Activiations 102 Memory 108 Input 310 310 Activation Bus Parameters MAC 304 T r104a f Parameters MAC 304 r104b IL f Parameters MAC 304 r104c 306 —2 Output Activation Bus I 1 , 308 305 -- UU1 t. EI1 i ,-1 .4 I I - (57) : A computer el a controller of the computing device, whether each of the input activations has either a zero value or a non-zero value. The method 0 further includes storing, GC © activation includes generating - non-zero values. The GC ,_ 1 onto a data bus that is © memory address location N ( 1 ) ( ) 3 ( 5 ) ( ) 7 FIG. 3 -implemented method includes receiving, by a computing device, input activations and determining, by in a memory bank of the computing device, at least one of the input activations. Storing the at least one input an index comprising one or more memory address locations that have input activation values that are method still further includes providing, by the controller and from the memory bank, at least accessible by one or more units of a computational array. The activations are provided, at least associated with the index. one input activation in part, from a C [Continued on next page] WO 2018/080624 Al MIDEDIMOMMIDIREEMOOMMMONEDIDEHMEMOIMIE GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Declarations under Rule 4.17: — as to applicant's entitlement to apply for and be granted a patent (Rule 4.17(U)) — as to the applicant's entitlement to claim the priority of the earlier application (Rule 4.17(iii)) Published: — with international search report (Art. 21(3))
EXPLOITING INPUT DATA SPARSITY IN NEURAL NETWORK COMPUTE UNITS