{"id":453375,"date":"2025-03-26T15:00:08","date_gmt":"2025-03-26T15:00:08","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=453375"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=453375","title":{"rendered":"<span>\u042d\u043d\u0442\u0443\u0437\u0438\u0430\u0441\u0442 \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u043b \u0433\u0435\u043d\u0435\u0440\u0430\u0442\u0438\u0432\u043d\u044b\u0439 \u0418\u0418 \u043d\u0430 20-\u043b\u0435\u0442\u043d\u0435\u043c PowerBook G4<\/span>"},"content":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-2\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>\u041b\u044e\u0431\u0438\u0442\u0435\u043b\u044c \u0432\u0438\u043d\u0442\u0430\u0436\u043d\u044b\u0445 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0439 <a href=\"http:\/\/www.theresistornetwork.com\/2025\/03\/thinking-different-thinking-slowly-llms.html\" rel=\"noopener noreferrer nofollow\">\u0440\u0435\u0448\u0438\u043b \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c<\/a> \u0431\u043e\u043b\u044c\u0448\u0443\u044e \u044f\u0437\u044b\u043a\u043e\u0432\u0443\u044e \u043c\u043e\u0434\u0435\u043b\u044c (LLM), Llama2 \u043d\u0430 Apple PowerBook G4 2005 \u0433\u043e\u0434\u0430. \u041a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440 \u043e\u0441\u043d\u0430\u0449\u0451\u043d \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u043e\u043c 1,5 \u0413\u0413\u0446, 1 \u0413\u0431 \u043e\u043f\u0435\u0440\u0430\u0442\u0438\u0432\u043d\u043e\u0439 \u043f\u0430\u043c\u044f\u0442\u0438 \u0438 \u0438\u043c\u0435\u0435\u0442 \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u043d\u043e\u0435 32-\u0431\u0438\u0442\u043d\u043e\u0435 \u0430\u0434\u0440\u0435\u0441\u043d\u043e\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e.\u00a0<\/p>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/getpro\/habr\/upload_files\/373\/25c\/9e1\/37325c9e1543aaff7b4f35c22a42e668.jpeg\" width=\"639\" height=\"426\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/373\/25c\/9e1\/37325c9e1543aaff7b4f35c22a42e668.jpeg\" data-blurred=\"true\"\/><\/figure>\n<p>\u042d\u043d\u0442\u0443\u0437\u0438\u0430\u0441\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u043b \u043f\u0440\u043e\u0435\u043a\u0442 ullm. \u0412 \u043e\u0441\u043d\u043e\u0432\u0435 \u043f\u0440\u043e\u0435\u043a\u0442\u0430 \u043b\u0435\u0436\u0438\u0442 Llama2.c \u043e\u0442 \u0410\u043d\u0434\u0440\u0435\u044f \u041a\u0430\u0440\u043f\u0430\u0442\u0438, \u043c\u0438\u043d\u0438\u043c\u0430\u043b\u0438\u0441\u0442\u0438\u0447\u043d\u0430\u044f \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f LLM \u043d\u0430 \u0447\u0438\u0441\u0442\u043e\u043c C.<\/p>\n<p>\u0420\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a \u043d\u0430\u0447\u0430\u043b \u0441 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u0438\u0445 \u0431\u0430\u0437\u043e\u0432\u044b\u0445 \u0443\u043b\u0443\u0447\u0448\u0435\u043d\u0438\u0439 \u043a\u043e\u0434\u0430 \u0438 \u0434\u043e\u0431\u0430\u0432\u0438\u043b \u043e\u0431\u043e\u043b\u043e\u0447\u043a\u0438 \u0434\u043b\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0445 \u0444\u0443\u043d\u043a\u0446\u0438\u0439, \u0442\u0430\u043a\u0438\u0445 \u043a\u0430\u043a \u0444\u0430\u0439\u043b\u043e\u0432\u044b\u0439 \u0432\u0432\u043e\u0434-\u0432\u044b\u0432\u043e\u0434 \u0438 \u0432\u044b\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u043f\u0430\u043c\u044f\u0442\u0438.\u00a0<\/p>\n<p>\u0414\u043b\u044f \u0440\u0430\u0431\u043e\u0442\u044b \u043d\u0430 PowerPC:<\/p>\n<ul>\n<li>\n<p>\u0434\u043e\u0431\u0430\u0432\u0438\u043b\u0438 \u043e\u0431\u0451\u0440\u0442\u043a\u0438 \u0434\u043b\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0445 \u0444\u0443\u043d\u043a\u0446\u0438\u0439;<\/p>\n<\/li>\n<li>\n<p>\u0443\u0441\u0442\u0440\u0430\u043d\u0438\u043b\u0438 \u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e\u0441\u0442\u0438 \u043e\u0442 exit;<\/p>\n<\/li>\n<li>\n<p>\u0440\u0435\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043b\u0438 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0443 big-endian (\u0432 \u043e\u0442\u043b\u0438\u0447\u0438\u0435 \u043e\u0442 little-endian \u0432 \u0441\u043e\u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c\u0430\u0445);<\/p>\n<\/li>\n<li>\n<p>\u0443\u043b\u0443\u0447\u0448\u0438\u043b\u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0443 \u043e\u0448\u0438\u0431\u043e\u043a \u0438 \u0432\u044b\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u043f\u0430\u043c\u044f\u0442\u0438.<\/p>\n<\/li>\n<\/ul>\n<p>\u0417\u0430\u0442\u0435\u043c \u043e\u043d \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u043b \u043a\u043e\u0434 \u0432 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a\u0443 \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c API, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u043e\u0442\u043e\u0431\u0440\u0430\u0436\u0430\u0435\u0442\u0441\u044f \u0432 \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u0435. \u042d\u0442\u043e \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u043f\u0440\u043e\u0432\u043e\u0434\u0438\u0442\u044c \u043c\u043e\u0434\u0443\u043b\u044c\u043d\u043e\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435, \u0447\u0442\u043e\u0431\u044b \u0433\u0430\u0440\u0430\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u0442\u044c, \u0447\u0442\u043e \u0434\u0430\u043b\u044c\u043d\u0435\u0439\u0448\u0438\u0439 \u0440\u0435\u0444\u0430\u043a\u0442\u043e\u0440\u0438\u043d\u0433 \u043d\u0435 \u043d\u0430\u0440\u0443\u0448\u0438\u0442 \u0444\u0443\u043d\u043a\u0446\u0438\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u0432\u044b\u0432\u043e\u0434\u0430.<\/p>\n<pre><code>\/\/ The runtime config for the inference operation. typedef struct {   \/\/ The prompt to generate a response to.   const char* prompt;     \/\/ The path to the checkpoint file.   const char* checkpoint_path;     \/\/ The path to the tokenizer file.   const char* tokenizer_path;     \/\/ Model configuration.   float temperature;   float topp;   unsigned int steps;     \/\/ The source of entropy.   uint64_t rng_seed;     \/\/ The callback and context for generated output.   void (*output_callback)(const char* token, void* cookie);   void* cookie; } UllmLlama2RunConfig;   \/\/ The runtime state for the inference engine. typedef struct {   UllmFileHandle checkpoint_file;   UllmLlama2Transformer transformer;   UllmLlama2Tokenizer tokenizer;   UllmLlama2Sampler sampler; } UllmLlama2State;<\/code><\/pre>\n<p>\u041f\u043e\u043b\u0443\u0447\u0435\u043d\u043d\u044b\u0439 API \u0447\u0440\u0435\u0437\u0432\u044b\u0447\u0430\u0439\u043d\u043e \u043f\u0440\u043e\u0441\u0442 \u0432 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0438 \u0438 \u043d\u0430 \u0435\u0433\u043e \u043e\u0441\u043d\u043e\u0432\u0435 \u043b\u0435\u0433\u043a\u043e \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u0442\u044c \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441\u0430 \u043a\u043e\u043c\u0430\u043d\u0434\u043d\u043e\u0439 \u0441\u0442\u0440\u043e\u043a\u0438.<\/p>\n<p>\u042d\u043d\u0442\u0443\u0437\u0438\u0430\u0441\u0442 \u0440\u0430\u0431\u043e\u0442\u0430\u043b \u0441 \u043c\u043e\u0434\u0435\u043b\u044c\u044e TinyStories, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043d\u0435 \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0433\u043e \u0430\u043f\u043f\u0430\u0440\u0430\u0442\u043d\u043e\u0433\u043e \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f. \u041e\u043d \u043f\u0440\u043e\u0432\u0451\u043b \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0441 \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u043e\u043c \u043c\u043e\u0434\u0435\u043b\u0438 15M, \u0430 \u0437\u0430\u0442\u0435\u043c \u043f\u0435\u0440\u0435\u043a\u043b\u044e\u0447\u0438\u043b\u0441\u044f \u043d\u0430 \u0441\u0430\u043c\u0443\u044e \u0442\u043e\u0447\u043d\u0443\u044e \u0438\u0437 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 110M.\u00a0<\/p>\n<pre><code>aarossig@titanium:~\/Projects\/ullm$ make &amp;&amp; time .\/out\/ullm.elf -c ..\/models\/tinystories110M_be.bin -t ..\/models\/tokenizer_be.bin -p \"The quick brown fox jumped. Where did he go?\" The quick brown fox jumped. Where did he go? He had been running around the forest all day. He was looking for something special. Suddenly, he saw a big tree. He stopped and looked up. He saw a big, brown bird in the tree. The bird was singing a beautiful song. The fox was so happy. He wanted to get closer to the bird. He jumped up and down, trying to reach the bird. But the bird was too high up. The fox was sad. He wanted to be friends with the bird. He looked around and saw a big, brown rock. He jumped on the rock and looked up at the bird. The bird saw the fox and flew down to him. The fox was so happy. He and the bird became friends. They played together in the forest every day. I ullm.llama2: Complete: 0.77 token\/s   real    4m0.099s user    3m51.387s sys     0m5.533s<\/code><\/pre>\n<p>\u041d\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u0435 \u0441 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u043e\u043c Intel Xeon inference \u0432\u044b\u0432\u043e\u0434 \u0448\u0451\u043b \u0441\u043e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c\u044e 6.91 \u0442\u043e\u043a\u0435\u043d\u043e\u0432 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u041d\u0430 PowerBook G4 \u2014 0.77 \u0442\u043e\u043a\u0435\u043d\u0430\/\u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u0432\u044b\u0432\u043e\u0434 \u0440\u0430\u0431\u043e\u0442\u0430\u043b \u043f\u0440\u0438\u043c\u0435\u0440\u043d\u043e \u0432 \u0434\u0435\u0432\u044f\u0442\u044c \u0440\u0430\u0437 \u043c\u0435\u0434\u043b\u0435\u043d\u043d\u0435\u0435, \u0447\u0435\u043c \u043d\u0430 \u041f\u041a \u0441 \u0441\u043e\u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u043e\u043c. \u041e\u0441\u043d\u043e\u0432\u043d\u043e\u0439 \u0443\u0437\u043a\u0438\u0439 \u043c\u043e\u043c\u0435\u043d\u0442 \u2014 \u043c\u0430\u0442\u0440\u0438\u0447\u043d\u044b\u0435 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0437\u0430\u043d\u0438\u043c\u0430\u044e\u0442 \u0431\u043e\u043b\u044c\u0448\u0443\u044e \u0447\u0430\u0441\u0442\u044c \u0432\u0440\u0435\u043c\u0435\u043d\u0438 \u043f\u0440\u0438 \u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0438\u0438 \u0442\u0435\u043a\u0441\u0442\u0430. <\/p>\n<pre><code>void matmul(float* xout, const float* x, const float* w, int n, int d) {   \/\/ W (d,n) @ x (n,) -&gt; xout (d,)   const float* wptr = w;   const float* wptr_end = &amp;wptr[n * d];   const float* xptr_end = &amp;x[n];   while (wptr &lt; wptr_end) {     float sum = 0.0f;     const float* xptr = x;     while (xptr &lt; xptr_end) {       sum += *wptr++ * *xptr++;     }       *xout++ = sum;   } }<\/code><\/pre>\n<p>\u0427\u0442\u043e\u0431\u044b \u0443\u0441\u043a\u043e\u0440\u0438\u0442\u044c \u0432\u044b\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0435, \u0430\u0432\u0442\u043e\u0440 \u043f\u0435\u0440\u0435\u043f\u0438\u0441\u0430\u043b \u043a\u0440\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0443\u044e \u0444\u0443\u043d\u043a\u0446\u0438\u044e \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c SIMD-\u0438\u043d\u0441\u0442\u0440\u0443\u043a\u0446\u0438\u0439 AltiVec, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 \u0432 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u0430\u0445 PowerPC. \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u044e fusion multiply-add, \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440 \u043c\u043e\u0436\u0435\u0442 \u0432\u044b\u043f\u043e\u043b\u043d\u044f\u0442\u044c 4-\u043a\u0440\u0430\u0442\u043d\u043e\u0435 \u043f\u0430\u0440\u0430\u043b\u043b\u0435\u043b\u044c\u043d\u043e\u0435 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0438 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0435 \u0441 \u043f\u043b\u0430\u0432\u0430\u044e\u0449\u0435\u0439 \u0442\u043e\u0447\u043a\u043e\u0439, \u043f\u043e\u043b\u0430\u0433\u0430\u044f\u0441\u044c \u043d\u0430 \u0441\u0435\u043c\u0430\u043d\u0442\u0438\u043a\u0443 SIMD (single instruction multiple data) \u0434\u043b\u044f \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f. <\/p>\n<pre><code>void matmul(float* xout, const float* x, const float* w, int n, int d) {   \/\/ W (d,n) @ x (n,) -&gt; xout (d,)   const long addr_step = 4 * sizeof(float);   const long w_addr_end = n * d * sizeof(float);   const long x_addr_end = n * sizeof(float);     long w_addr = 0;   while (w_addr &lt; w_addr_end) {     __attribute__ ((aligned(16))) float psum[4] = {};     vector float sum_vec = vec_ld(0, &amp;psum[0]);     for (long x_addr = 0; x_addr &lt; x_addr_end; x_addr += addr_step) {       vector float w_vec = vec_ld(w_addr, w);       vector float x_vec = vec_ld(x_addr, x);       sum_vec = vec_madd(w_vec, x_vec, sum_vec);       w_addr += addr_step;     }       vec_st(sum_vec, 0, psum);     *xout++ = psum[0] + psum[1] + psum[2] + psum[3];   } }<\/code><\/pre>\n<p>AltiVec \u0441\u043d\u0438\u0437\u0438\u043b \u0432\u0440\u0435\u043c\u044f \u0432\u044b\u0432\u043e\u0434\u0430 \u043f\u0440\u0438\u043c\u0435\u0440\u043d\u043e \u043d\u0430 30 \u0441\u0435\u043a\u0443\u043d\u0434, \u0434\u043e 0.88 \u0442\u043e\u043a\u0435\u043d\u0430\/\u0441\u0435\u043a\u0443\u043d\u0434\u0443.<\/p>\n<pre><code>aarossig@titanium:~\/Projects\/ullm$ make &amp;&amp; time .\/out\/ullm.elf -c ..\/models\/tinystories110M_be.bin -t ..\/models\/tokenizer_be.bin -p \"The quick brown fox jumped. Where did he go?\" The quick brown fox jumped. Where did he go? He had been running around the forest all day. He was looking for something special. Suddenly, he saw a big tree. He stopped and looked up. He saw a big, brown bird in the tree. The bird was singing a beautiful song. The fox was so happy. He wanted to get closer to the bird. He jumped up and down, trying to reach the bird. But the bird was too high up. The fox was sad. He wanted to be friends with the bird. He looked around and saw a big, brown rock. He jumped on the rock and looked up at the bird. The bird saw the fox and flew down to him. The fox was so happy. He and the bird became friends. They played together in the forest every day. I ullm.llama2: Complete: 0.88 token\/s   real    3m32.098s user    3m23.535s sys     0m5.368s<\/code><\/pre>\n<p>\u0410\u0432\u0442\u043e\u0440 \u043e\u0442\u043c\u0435\u0442\u0438\u043b, \u0447\u0442\u043e \u0441\u0438\u0441\u0442\u0435\u043c\u0430 G4 PowerPC \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u0430 \u043c\u0430\u043a\u0441\u0438\u043c\u0430\u043b\u044c\u043d\u043e\u0439 \u0430\u0434\u0440\u0435\u0441\u0443\u0435\u043c\u043e\u0439 \u043f\u0430\u043c\u044f\u0442\u044c\u044e \u0432 4 \u0413\u0411, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0431\u043e\u043b\u0435\u0435 \u043a\u0440\u0443\u043f\u043d\u044b\u0435 \u043c\u043e\u0434\u0435\u043b\u0438 \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u043d\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u0442\u0441\u044f.<\/p>\n<p>\u041f\u043e\u0434\u0440\u043e\u0431\u043d\u0435\u0435 \u043e\u0431 Apple Powerbook G4 <a href=\"https:\/\/habr.com\/ru\/companies\/kaspersky\/articles\/705596\/\" rel=\"noopener noreferrer nofollow\">\u043c\u043e\u0436\u043d\u043e \u043f\u0440\u043e\u0447\u0438\u0442\u0430\u0442\u044c \u0437\u0434\u0435\u0441\u044c<\/a>.<\/p>\n<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/894520\/\"> https:\/\/habr.com\/ru\/articles\/894520\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-2\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<p>\u041b\u044e\u0431\u0438\u0442\u0435\u043b\u044c \u0432\u0438\u043d\u0442\u0430\u0436\u043d\u044b\u0445 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0439 <a href=\"http:\/\/www.theresistornetwork.com\/2025\/03\/thinking-different-thinking-slowly-llms.html\" rel=\"noopener noreferrer nofollow\">\u0440\u0435\u0448\u0438\u043b \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c<\/a> \u0431\u043e\u043b\u044c\u0448\u0443\u044e \u044f\u0437\u044b\u043a\u043e\u0432\u0443\u044e \u043c\u043e\u0434\u0435\u043b\u044c (LLM), Llama2 \u043d\u0430 Apple PowerBook G4 2005 \u0433\u043e\u0434\u0430. \u041a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440 \u043e\u0441\u043d\u0430\u0449\u0451\u043d \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u043e\u043c 1,5 \u0413\u0413\u0446, 1 \u0413\u0431 \u043e\u043f\u0435\u0440\u0430\u0442\u0438\u0432\u043d\u043e\u0439 \u043f\u0430\u043c\u044f\u0442\u0438 \u0438 \u0438\u043c\u0435\u0435\u0442 \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u043d\u043e\u0435 32-\u0431\u0438\u0442\u043d\u043e\u0435 \u0430\u0434\u0440\u0435\u0441\u043d\u043e\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e.\u00a0<\/p>\n<figure class=\"full-width\"><\/figure>\n<p>\u042d\u043d\u0442\u0443\u0437\u0438\u0430\u0441\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u043b \u043f\u0440\u043e\u0435\u043a\u0442 ullm. \u0412 \u043e\u0441\u043d\u043e\u0432\u0435 \u043f\u0440\u043e\u0435\u043a\u0442\u0430 \u043b\u0435\u0436\u0438\u0442 Llama2.c \u043e\u0442 \u0410\u043d\u0434\u0440\u0435\u044f \u041a\u0430\u0440\u043f\u0430\u0442\u0438, \u043c\u0438\u043d\u0438\u043c\u0430\u043b\u0438\u0441\u0442\u0438\u0447\u043d\u0430\u044f \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f LLM \u043d\u0430 \u0447\u0438\u0441\u0442\u043e\u043c C.<\/p>\n<p>\u0420\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a \u043d\u0430\u0447\u0430\u043b \u0441 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u0438\u0445 \u0431\u0430\u0437\u043e\u0432\u044b\u0445 \u0443\u043b\u0443\u0447\u0448\u0435\u043d\u0438\u0439 \u043a\u043e\u0434\u0430 \u0438 \u0434\u043e\u0431\u0430\u0432\u0438\u043b \u043e\u0431\u043e\u043b\u043e\u0447\u043a\u0438 \u0434\u043b\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0445 \u0444\u0443\u043d\u043a\u0446\u0438\u0439, \u0442\u0430\u043a\u0438\u0445 \u043a\u0430\u043a \u0444\u0430\u0439\u043b\u043e\u0432\u044b\u0439 \u0432\u0432\u043e\u0434-\u0432\u044b\u0432\u043e\u0434 \u0438 \u0432\u044b\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u043f\u0430\u043c\u044f\u0442\u0438.\u00a0<\/p>\n<p>\u0414\u043b\u044f \u0440\u0430\u0431\u043e\u0442\u044b \u043d\u0430 PowerPC:<\/p>\n<ul>\n<li>\n<p>\u0434\u043e\u0431\u0430\u0432\u0438\u043b\u0438 \u043e\u0431\u0451\u0440\u0442\u043a\u0438 \u0434\u043b\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0445 \u0444\u0443\u043d\u043a\u0446\u0438\u0439;<\/p>\n<\/li>\n<li>\n<p>\u0443\u0441\u0442\u0440\u0430\u043d\u0438\u043b\u0438 \u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e\u0441\u0442\u0438 \u043e\u0442 exit;<\/p>\n<\/li>\n<li>\n<p>\u0440\u0435\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043b\u0438 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0443 big-endian (\u0432 \u043e\u0442\u043b\u0438\u0447\u0438\u0435 \u043e\u0442 little-endian \u0432 \u0441\u043e\u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c\u0430\u0445);<\/p>\n<\/li>\n<li>\n<p>\u0443\u043b\u0443\u0447\u0448\u0438\u043b\u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0443 \u043e\u0448\u0438\u0431\u043e\u043a \u0438 \u0432\u044b\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u043f\u0430\u043c\u044f\u0442\u0438.<\/p>\n<\/li>\n<\/ul>\n<p>\u0417\u0430\u0442\u0435\u043c \u043e\u043d \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u043b \u043a\u043e\u0434 \u0432 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a\u0443 \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c API, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u043e\u0442\u043e\u0431\u0440\u0430\u0436\u0430\u0435\u0442\u0441\u044f \u0432 \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u0435. \u042d\u0442\u043e \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u043f\u0440\u043e\u0432\u043e\u0434\u0438\u0442\u044c \u043c\u043e\u0434\u0443\u043b\u044c\u043d\u043e\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435, \u0447\u0442\u043e\u0431\u044b \u0433\u0430\u0440\u0430\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u0442\u044c, \u0447\u0442\u043e \u0434\u0430\u043b\u044c\u043d\u0435\u0439\u0448\u0438\u0439 \u0440\u0435\u0444\u0430\u043a\u0442\u043e\u0440\u0438\u043d\u0433 \u043d\u0435 \u043d\u0430\u0440\u0443\u0448\u0438\u0442 \u0444\u0443\u043d\u043a\u0446\u0438\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u0432\u044b\u0432\u043e\u0434\u0430.<\/p>\n<pre><code>\/\/ The runtime config for the inference operation. typedef struct {   \/\/ The prompt to generate a response to.   const char* prompt;     \/\/ The path to the checkpoint file.   const char* checkpoint_path;     \/\/ The path to the tokenizer file.   const char* tokenizer_path;     \/\/ Model configuration.   float temperature;   float topp;   unsigned int steps;     \/\/ The source of entropy.   uint64_t rng_seed;     \/\/ The callback and context for generated output.   void (*output_callback)(const char* token, void* cookie);   void* cookie; } UllmLlama2RunConfig;   \/\/ The runtime state for the inference engine. typedef struct {   UllmFileHandle checkpoint_file;   UllmLlama2Transformer transformer;   UllmLlama2Tokenizer tokenizer;   UllmLlama2Sampler sampler; } UllmLlama2State;<\/code><\/pre>\n<p>\u041f\u043e\u043b\u0443\u0447\u0435\u043d\u043d\u044b\u0439 API \u0447\u0440\u0435\u0437\u0432\u044b\u0447\u0430\u0439\u043d\u043e \u043f\u0440\u043e\u0441\u0442 \u0432 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0438 \u0438 \u043d\u0430 \u0435\u0433\u043e \u043e\u0441\u043d\u043e\u0432\u0435 \u043b\u0435\u0433\u043a\u043e \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u0442\u044c \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441\u0430 \u043a\u043e\u043c\u0430\u043d\u0434\u043d\u043e\u0439 \u0441\u0442\u0440\u043e\u043a\u0438.<\/p>\n<p>\u042d\u043d\u0442\u0443\u0437\u0438\u0430\u0441\u0442 \u0440\u0430\u0431\u043e\u0442\u0430\u043b \u0441 \u043c\u043e\u0434\u0435\u043b\u044c\u044e TinyStories, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043d\u0435 \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0433\u043e \u0430\u043f\u043f\u0430\u0440\u0430\u0442\u043d\u043e\u0433\u043e \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f. \u041e\u043d \u043f\u0440\u043e\u0432\u0451\u043b \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u0441 \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u043e\u043c \u043c\u043e\u0434\u0435\u043b\u0438 15M, \u0430 \u0437\u0430\u0442\u0435\u043c \u043f\u0435\u0440\u0435\u043a\u043b\u044e\u0447\u0438\u043b\u0441\u044f \u043d\u0430 \u0441\u0430\u043c\u0443\u044e \u0442\u043e\u0447\u043d\u0443\u044e \u0438\u0437 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 110M.\u00a0<\/p>\n<pre><code>aarossig@titanium:~\/Projects\/ullm$ make &amp;&amp; time .\/out\/ullm.elf -c ..\/models\/tinystories110M_be.bin -t ..\/models\/tokenizer_be.bin -p \"The quick brown fox jumped. Where did he go?\" The quick brown fox jumped. Where did he go? He had been running around the forest all day. He was looking for something special. Suddenly, he saw a big tree. He stopped and looked up. He saw a big, brown bird in the tree. The bird was singing a beautiful song. The fox was so happy. He wanted to get closer to the bird. He jumped up and down, trying to reach the bird. But the bird was too high up. The fox was sad. He wanted to be friends with the bird. He looked around and saw a big, brown rock. He jumped on the rock and looked up at the bird. The bird saw the fox and flew down to him. The fox was so happy. He and the bird became friends. They played together in the forest every day. I ullm.llama2: Complete: 0.77 token\/s   real    4m0.099s user    3m51.387s sys     0m5.533s<\/code><\/pre>\n<p>\u041d\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u0435 \u0441 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u043e\u043c Intel Xeon inference \u0432\u044b\u0432\u043e\u0434 \u0448\u0451\u043b \u0441\u043e \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c\u044e 6.91 \u0442\u043e\u043a\u0435\u043d\u043e\u0432 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u041d\u0430 PowerBook G4 \u2014 0.77 \u0442\u043e\u043a\u0435\u043d\u0430\/\u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u0432\u044b\u0432\u043e\u0434 \u0440\u0430\u0431\u043e\u0442\u0430\u043b \u043f\u0440\u0438\u043c\u0435\u0440\u043d\u043e \u0432 \u0434\u0435\u0432\u044f\u0442\u044c \u0440\u0430\u0437 \u043c\u0435\u0434\u043b\u0435\u043d\u043d\u0435\u0435, \u0447\u0435\u043c \u043d\u0430 \u041f\u041a \u0441 \u0441\u043e\u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u043e\u043c. \u041e\u0441\u043d\u043e\u0432\u043d\u043e\u0439 \u0443\u0437\u043a\u0438\u0439 \u043c\u043e\u043c\u0435\u043d\u0442 \u2014 \u043c\u0430\u0442\u0440\u0438\u0447\u043d\u044b\u0435 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0437\u0430\u043d\u0438\u043c\u0430\u044e\u0442 \u0431\u043e\u043b\u044c\u0448\u0443\u044e \u0447\u0430\u0441\u0442\u044c \u0432\u0440\u0435\u043c\u0435\u043d\u0438 \u043f\u0440\u0438 \u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0438\u0438 \u0442\u0435\u043a\u0441\u0442\u0430. <\/p>\n<pre><code>void matmul(float* xout, const float* x, const float* w, int n, int d) {   \/\/ W (d,n) @ x (n,) -&gt; xout (d,)   const float* wptr = w;   const float* wptr_end = &amp;wptr[n * d];   const float* xptr_end = &amp;x[n];   while (wptr &lt; wptr_end) {     float sum = 0.0f;     const float* xptr = x;     while (xptr &lt; xptr_end) {       sum += *wptr++ * *xptr++;     }       *xout++ = sum;   } }<\/code><\/pre>\n<p>\u0427\u0442\u043e\u0431\u044b \u0443\u0441\u043a\u043e\u0440\u0438\u0442\u044c \u0432\u044b\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u0435, \u0430\u0432\u0442\u043e\u0440 \u043f\u0435\u0440\u0435\u043f\u0438\u0441\u0430\u043b \u043a\u0440\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0443\u044e \u0444\u0443\u043d\u043a\u0446\u0438\u044e \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c SIMD-\u0438\u043d\u0441\u0442\u0440\u0443\u043a\u0446\u0438\u0439 AltiVec, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 \u0432 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u0430\u0445 PowerPC. \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u044e fusion multiply-add, \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440 \u043c\u043e\u0436\u0435\u0442 \u0432\u044b\u043f\u043e\u043b\u043d\u044f\u0442\u044c 4-\u043a\u0440\u0430\u0442\u043d\u043e\u0435 \u043f\u0430\u0440\u0430\u043b\u043b\u0435\u043b\u044c\u043d\u043e\u0435 \u0443\u043c\u043d\u043e\u0436\u0435\u043d\u0438\u0435 \u0438 \u0441\u043b\u043e\u0436\u0435\u043d\u0438\u0435 \u0441 \u043f\u043b\u0430\u0432\u0430\u044e\u0449\u0435\u0439 \u0442\u043e\u0447\u043a\u043e\u0439, \u043f\u043e\u043b\u0430\u0433\u0430\u044f\u0441\u044c \u043d\u0430 \u0441\u0435\u043c\u0430\u043d\u0442\u0438\u043a\u0443 SIMD (single instruction multiple data) \u0434\u043b\u044f \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f. <\/p>\n<pre><code>void matmul(float* xout, const float* x, const float* w, int n, int d) {   \/\/ W (d,n) @ x (n,) -&gt; xout (d,)   const long addr_step = 4 * sizeof(float);   const long w_addr_end = n * d * sizeof(float);   const long x_addr_end = n * sizeof(float);     long w_addr = 0;   while (w_addr &lt; w_addr_end) {     __attribute__ ((aligned(16))) float psum[4] = {};     vector float sum_vec = vec_ld(0, &amp;psum[0]);     for (long x_addr = 0; x_addr &lt; x_addr_end; x_addr += addr_step) {       vector float w_vec = vec_ld(w_addr, w);       vector float x_vec = vec_ld(x_addr, x);       sum_vec = vec_madd(w_vec, x_vec, sum_vec);       w_addr += addr_step;     }       vec_st(sum_vec, 0, psum);     *xout++ = psum[0] + psum[1] + psum[2] + psum[3];   } }<\/code><\/pre>\n<p>AltiVec \u0441\u043d\u0438\u0437\u0438\u043b \u0432\u0440\u0435\u043c\u044f \u0432\u044b\u0432\u043e\u0434\u0430 \u043f\u0440\u0438\u043c\u0435\u0440\u043d\u043e \u043d\u0430 30 \u0441\u0435\u043a\u0443\u043d\u0434, \u0434\u043e 0.88 \u0442\u043e\u043a\u0435\u043d\u0430\/\u0441\u0435\u043a\u0443\u043d\u0434\u0443.<\/p>\n<pre><code>aarossig@titanium:~\/Projects\/ullm$ make &amp;&amp; time .\/out\/ullm.elf -c ..\/models\/tinystories110M_be.bin -t ..\/models\/tokenizer_be.bin -p \"The quick brown fox jumped. Where did he go?\" The quick brown fox jumped. Where did he go? He had been running around the forest all day. He was looking for something special. Suddenly, he saw a big tree. He stopped and looked up. He saw a big, brown bird in the tree. The bird was singing a beautiful song. The fox was so happy. He wanted to get closer to the bird. He jumped up and down, trying to reach the bird. But the bird was too high up. The fox was sad. He wanted to be friends with the bird. He looked around and saw a big, brown rock. He jumped on the rock and looked up at the bird. The bird saw the fox and flew down to him. The fox was so happy. He and the bird became friends. They played together in the forest every day. I ullm.llama2: Complete: 0.88 token\/s   real    3m32.098s user    3m23.535s sys     0m5.368s<\/code><\/pre>\n<p>\u0410\u0432\u0442\u043e\u0440 \u043e\u0442\u043c\u0435\u0442\u0438\u043b, \u0447\u0442\u043e \u0441\u0438\u0441\u0442\u0435\u043c\u0430 G4 PowerPC \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u0430 \u043c\u0430\u043a\u0441\u0438\u043c\u0430\u043b\u044c\u043d\u043e\u0439 \u0430\u0434\u0440\u0435\u0441\u0443\u0435\u043c\u043e\u0439 \u043f\u0430\u043c\u044f\u0442\u044c\u044e \u0432 4 \u0413\u0411, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0431\u043e\u043b\u0435\u0435 \u043a\u0440\u0443\u043f\u043d\u044b\u0435 \u043c\u043e\u0434\u0435\u043b\u0438 \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u043d\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u0442\u0441\u044f.<\/p>\n<p>\u041f\u043e\u0434\u0440\u043e\u0431\u043d\u0435\u0435 \u043e\u0431 Apple Powerbook G4 <a href=\"https:\/\/habr.com\/ru\/companies\/kaspersky\/articles\/705596\/\" rel=\"noopener noreferrer nofollow\">\u043c\u043e\u0436\u043d\u043e \u043f\u0440\u043e\u0447\u0438\u0442\u0430\u0442\u044c \u0437\u0434\u0435\u0441\u044c<\/a>.<\/p>\n<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/894520\/\"> https:\/\/habr.com\/ru\/articles\/894520\/<\/a><br \/><\/br><\/br><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-453375","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/453375","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=453375"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/453375\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=453375"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=453375"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=453375"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}