{"id":302119,"date":"2020-04-18T15:00:35","date_gmt":"2020-04-18T15:00:35","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=302119"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=302119","title":{"rendered":"\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 GPU \u0432\u0435\u0440\u0441\u0438\u0438 HPL \u0441 OpenBLAS"},"content":{"rendered":"\n<div class=\"post__text post__text-html post__text_v1\" id=\"post-content-body\" data-io-article-url=\"https:\/\/habr.com\/ru\/post\/497878\/\">\u0414\u043b\u044f \u043f\u043e\u043f\u0430\u0434\u0430\u043d\u0438\u044f \u0432 \u0441\u043f\u0438\u0441\u043e\u043a \u0422\u041e\u041f 50, 100, 500 HPC (High Performance Computing) \u043a\u043e\u043c\u043f\u043b\u0435\u043a\u0441\u043e\u0432 \u043f\u043e\u0434\u0445\u043e\u0434\u044f\u0442 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f, \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u043d\u044b\u0435 \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0431\u0435\u043d\u0447\u043c\u0430\u0440\u043a\u0430 HPL (High Performance Linpack).<\/p>\n<p>  \u0411\u0435\u043d\u0447\u043c\u0430\u0440\u043a Linpack (Linear Algebra PACKage) \u0440\u0435\u0430\u043b\u0438\u0437\u0443\u0435\u0442 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u0421\u041b\u0410\u0423 \u043c\u0435\u0442\u043e\u0434\u043e\u043c LU \u0440\u0430\u0437\u043b\u043e\u0436\u0435\u043d\u0438\u044f. \u042d\u0442\u043e\u0442 \u043f\u0430\u043a\u0435\u0442 \u043e\u0431\u0449\u0435\u0434\u043e\u0441\u0442\u0443\u043f\u0435\u043d, \u043f\u0440\u043e\u0441\u0442 \u0432 \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0435 \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0435. \u0425\u043e\u0440\u043e\u0448\u043e \u043f\u043e\u0434\u0445\u043e\u0434\u0438\u0442 \u0434\u043b\u044f \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0430\u0446\u0438\u0438 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 CPU.<\/p>\n<p>  \u0412\u0441\u0435, \u043a\u0442\u043e \u0437\u043d\u0430\u043a\u043e\u043c \u0441 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u043e\u0439 \u0433\u0440\u0430\u0444\u0438\u0447\u0435\u0441\u043a\u0438\u0445 \u0443\u0441\u043a\u043e\u0440\u0438\u0442\u0435\u043b\u0435\u0439 \u043c\u043e\u0433\u0443\u0442 \u043f\u0440\u0435\u0434\u043f\u043e\u043b\u043e\u0436\u0438\u0442\u044c, \u0447\u0442\u043e \u0435\u0449\u0435 \u043b\u0443\u0447\u0448\u0435 \u044d\u0442\u043e\u0442 \u043f\u0430\u043a\u0435\u0442 \u043f\u043e\u0434\u043e\u0439\u0434\u0435\u0442 \u0434\u043b\u044f \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432 \u0441 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u043e\u0439 GPU. \u041e\u0434\u043d\u0430\u043a\u043e, \u0432 \u0441\u0435\u0442\u0438 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u0430 \u0434\u043b\u044f \u0441\u043a\u0430\u0447\u0438\u0432\u0430\u043d\u0438\u044f \u0432\u0435\u0440\u0441\u0438\u044f 2011 \u0433\u043e\u0434\u0430, \u0441 CUDA \u043f\u043e\u0434 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0443 Fermi.<\/p>\n<p>  \u0412 \u0434\u0430\u043d\u043d\u043e\u043c \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0441\u0442\u0432\u0435 \u044f \u043f\u0440\u0438\u0432\u0435\u0434\u0443 \u043f\u0440\u0438\u043c\u0435\u0440 \u0441\u0431\u043e\u0440\u043a\u0438 \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430 HPL \u0434\u043b\u044f GPU. <\/p>\n<p>  <a name=\"habracut\"><\/a><\/p>\n<p>  <i>\u041a\u0430\u043a \u0443\u043f\u0440\u0430\u0432\u043b\u044f\u0442\u044c \u0434\u043e\u0441\u0442\u0443\u043f\u043e\u043c \u043a \u041f\u041e?<br \/>  \u041a\u0430\u043a \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c CUDA?<br \/>  \u041a\u0430\u043a \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c OpenMPI?<br \/>  \u041a\u0430\u043a \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c OpenBLAS?<br \/>  \u041a\u0430\u043a \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c HPL \u0434\u043b\u044f GPU?<\/i><\/p>\n<h3>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 \u043f\u0430\u043a\u0435\u0442\u0430 MODULES<\/h3>\n<p>  \u0414\u043b\u044f \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u043c\u0438 \u043e\u043a\u0440\u0443\u0436\u0435\u043d\u0438\u044f \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u043c \u043f\u0430\u043a\u0435\u0442 MODULES \u0438 \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u0438\u043c \u0442\u0435\u0441\u0442\u043e\u0432\u044b\u0439 \u043c\u043e\u0434\u0443\u043b\u044c \u0444\u0430\u0439\u043b.<\/p>\n<pre><code class=\"bash\">$ yum install environment-modules $ mcedit \/etc\/modulefailes\/test\/v1.0   #%Module1.0   proc ModulesHelp { } {     global version       puts stderr &quot;Modulefile for test v1.0&quot;       }       set version v1.0       module-whatis &quot;Modulefile for test v1.0&quot;       # Our environment       setenv MAINDIR \/nfs\/software\/test\/v1.0       prepend-path PATH $env(MAINDIR)\/bin       prepend-path C_INCLUDE_PATH $env(MAINDIR)\/include       prepend-path CPLUS_INCLUDE_PATH $env(MAINDIR)\/include       prepend-path LIBRARY_PATH $env(MAINDIR)\/lib64       prepend-path LD_LIBRARY_PATH $env(MAINDIR)\/lib64<\/code><\/pre>\n<h3>\u041f\u0440\u043e\u0432\u0435\u0440\u043a\u0430 \u043c\u043e\u0434\u0443\u043b\u044c \u0444\u0430\u0439\u043b\u043e\u0432<\/h3>\n<p>  \u0412\u0435\u0440\u043e\u044f\u0442\u043d\u043e\u0441\u0442\u044c \u0434\u043e\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u043e\u0448\u0438\u0431\u043a\u0443 \u043f\u0440\u0438 \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0435 \u043c\u043e\u0434\u0443\u043b\u044f \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u0432\u0435\u043b\u0438\u043a\u0430. \u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u044f \u043f\u0440\u043e\u0432\u043e\u0436\u0443 \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0443 \u0432\u0441\u0435\u0445 \u043f\u0443\u0442\u0435\u0439, \u0443\u043a\u0430\u0437\u0430\u043d\u043d\u044b\u0445 \u0432 \u043c\u043e\u0434\u0443\u043b\u044c-\u0444\u0430\u0439\u043b\u0435. \u0427\u0442\u043e\u0431\u044b \u043d\u0435 \u043f\u0440\u043e\u0432\u0435\u0440\u044f\u0442\u044c \u043a\u0430\u0436\u0434\u044b\u0439 \u043f\u0443\u0442\u044c \u0432\u0440\u0443\u0447\u043d\u0443\u044e \u044f \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u0438\u043b \u0441\u043a\u0440\u0438\u043f\u0442. \u0415\u0441\u043b\u0438 0, \u0442\u043e \u043f\u0443\u0442\u044c \u043a\u043e\u0440\u0440\u0435\u043a\u0442\u043d\u044b\u0439.<\/p>\n<pre><code class=\"bash\">$ cat check-modulefiles   #!\/bin\/sh   ModulePath=$1   MainDir=$(cat $ModulePath | grep &quot;setenv MAINDIR&quot; | cut -f7 -d &quot; &quot;)   ListOfPaths=$(cat $ModulePath | grep path | cut -f7 -d &quot; &quot;)   #Replace MainDir setenv in modulefile   ListOfPaths=$(echo $ListOfPaths | sed &quot;s@\\$env(MAINDIR)@$MainDir@g&quot;)   for u in $ListOfPaths; do     ls -la $u 1&gt; \/dev\/null 2&gt; \/dev\/null;     printf &quot;%60s %4d\\n&quot; $u $?;   done $ chmod +x check-modulefiles $ .\/check-modulefiles \/etc\/modulefiles\/test\/v1.0   \/nfs\/software\/test\/v1.0\/bin            0   \/nfs\/software\/test\/v1.0\/include        0   \/nfs\/software\/test\/v1.0\/include        0   \/nfs\/software\/test\/v1.0\/lib64          0   \/nfs\/software\/test\/v1.0\/lib64          0<\/code><\/pre>\n<h3>\u041a\u043e\u043c\u0430\u043d\u0434\u044b \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u043c\u043e\u0434\u0443\u043b\u044f\u043c\u0438<\/h3>\n<pre><code class=\"bash\">$ module avail $ module add cuda\/v10.1 $ nvcc \u2013version   Cuda compilation tools, release 10.1, V10.1.168 $ module switch cuda\/v10.1 cuda\/v9.2 $ nvcc \u2013version   Cuda compilation tools, release 9.2, V9.2.88 $ module list $ module rm cuda\/v9.2<\/code><\/pre>\n<p>  1. \u041f\u043e\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0441\u043f\u0438\u0441\u043e\u043a \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 \u0434\u043b\u044f \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u044f \u043c\u043e\u0434\u0443\u043b\u0435\u0439<br \/>  2. \u041f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u043c \u043c\u043e\u0434\u0443\u043b\u044c<br \/>  3-4. \u041f\u0440\u043e\u0432\u0435\u0440\u0438\u043c \u0432\u0435\u0440\u0441\u0438\u044e<br \/>  5. \u041f\u043e\u043c\u0435\u043d\u044f\u0435\u043c \u043c\u043e\u0434\u0443\u043b\u044c<br \/>  6-7. \u041f\u0440\u043e\u0432\u0435\u0440\u0438\u043c \u0432\u0435\u0440\u0441\u0438\u044e<br \/>  8. \u041f\u043e\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0441\u043f\u0438\u0441\u043e\u043a \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u0445 \u043c\u043e\u0434\u0443\u043b\u0435\u0439<br \/>  9. \u0423\u0434\u0430\u043b\u0438\u043c \u043c\u043e\u0434\u0443\u043b\u044c \u0438\u0437 \u0441\u043f\u0438\u0441\u043a\u0430 \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u0445<\/p>\n<h3>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 CUDA<\/h3>\n<p>  \u0421\u043a\u0430\u0447\u0430\u0442\u044c CUDA 9.2 \u0434\u043b\u044f Centos 7 \u043c\u043e\u0436\u043d\u043e <a href=\"https:\/\/developer.nvidia.com\/cuda-92-download-archive?target_os=Linux&amp;target_arch=x86_64&amp;target_distro=CentOS&amp;target_version=7&amp;target_type=runfilelocal\" rel=\"nofollow\">\u0442\u0443\u0442<\/a>.<\/p>\n<pre><code class=\"bash\">$ chmod +x cuda_9.2.run $ .\/cuda_9.2.run   Do you accept the previously read EULA? accept   Install the CUDA 9.2 Toolkit? yes   Enter Toolkit Location: \/nfs\/software\/cuda\/v9.2   Do you want to install a symbolic link at \/usr\/local\/cuda? no   Install the CUDA 9.2 Samples? no $ cat \/etc\/modulefiles\/cuda\/v9.2   #%Module1.0   proc ModulesHelp { } {     global version       puts stderr &quot;Modulefile for cuda v9.2&quot;       }       set version v9.2       module-whatis &quot;Modulefile for cuda v9.2&quot;       # Our environment       setenv MAINDIR \/nfs\/software\/cuda\/v9.2       prepend-path PATH $env(MAINDIR)\/bin       prepend-path C_INCLUDE_PATH $env(MAINDIR)\/include       prepend-path CPLUS_INCLUDE_PATH $env(MAINDIR)\/include       prepend-path LIBRARY_PATH $env(MAINDIR)\/lib64\/stubs       prepend-path LIBRARY_PATH $env(MAINDIR)\/lib64       prepend-path LD_LIBRARY_PATH $env(MAINDIR)\/lib64\/stubs       prepend-path LD_LIBRARY_PATH $env(MAINDIR)\/lib64   $ module add cuda\/v9.2   $ nvcc --version   Cuda compilation tools, release 9.2, V9.2.148<\/code><\/pre>\n<h3>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 OpenBLAS<\/h3>\n<pre><code class=\"bash\">wget https:\/\/github.com\/xianyi\/OpenBLAS\/archive\/v0.3.6.tar.gz tar -xzvf v0.3.6.tar.gz cd OpenBLAS-0.3.6 mkdir -p \/nfs\/software\/openblas\/v0.3.6 make -j4 make PREFIX=\/nfs\/software\/openblas\/v0.3.6\/ install ls -la \/nfs\/software\/openblas\/v0.3.6\/lib\/ $ cat \/etc\/modulefiles\/openblas\/v0.3.6   #%Module1.0   proc ModulesHelp { } {     global version       puts stderr &quot;Modulefile for openblas v0.3.6&quot;       }       set version v0.3.6       module-whatis &quot;Modulefile for openblas v0.3.6&quot;       # Our environment       setenv MAINDIR \/nfs\/software\/openblas\/v0.3.6       prepend-path PATH $env(MAINDIR)\/bin       prepend-path C_INCLUDE_PATH $env(MAINDIR)\/include       prepend-path CPLUS_INCLUDE_PATH $env(MAINDIR)\/include       prepend-path LIBRARY_PATH $env(MAINDIR)\/lib       prepend-path LD_LIBRARY_PATH $env(MAINDIR)\/lib $ ls -la \/nfs\/software\/openblas\/v0.3.6\/lib<\/code><\/pre>\n<h3>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 OpenMPI<\/h3>\n<pre><code class=\"bash\">wget https:\/\/download.open-mpi.org\/release\/open-mpi\/v2.1\/openmpi-2.1.6.tar.gz $ tar -xzvf openmpi-2.1.6.tar.gz $ cd openmpi-2.1.6 $ mkdir -p \/nfs\/software\/openmpi\/v2.1.6 $ module add cuda\/v9.2 $ .\/configure --prefix=\/nfs\/software\/openmpi\/v2.1.6\/ --with-cuda --enable-static $ make $ make install $ cat \/etc\/modulefiles\/openmpi\/v2.1.6 #%Module1.0 proc ModulesHelp { } {   global version     puts stderr &quot;Modulefile for openmpi v2.1.6&quot;     }     set version v2.1.6     module-whatis &quot;Modulefile for openmpi v2.1.6&quot;     # Our environment     setenv MAINDIR \/nfs\/software\/openmpi\/v2.1.6     prepend-path PATH $env(MAINDIR)\/bin     prepend-path C_INCLUDE_PATH $env(MAINDIR)\/include     prepend-path CPLUS_INCLUDE_PATH $env(MAINDIR)\/include     prepend-path LIBRARY_PATH $env(MAINDIR)\/lib     prepend-path LD_LIBRARY_PATH $env(MAINDIR)\/lib $ module add openmpi\/v2.1.6 $ mpirun --version mpirun (Open MPI) 2.1.6<\/code><\/pre>\n<h3>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 HPL \u0434\u043b\u044f GPU<\/h3>\n<p>  \u041d\u0430\u0441\u0442\u0440\u043e\u0438\u043c \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0435 \u043e\u043a\u0440\u0443\u0436\u0435\u043d\u0438\u044f \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u0432 \u043c\u043e\u0434\u0443\u043b\u0438 \u0438 \u0441\u043a\u0430\u0447\u0430\u0435\u043c HPL 2.0.<\/p>\n<pre><code class=\"bash\">$ module add openmpi\/v2.1.6 $ module add cuda\/v9.2 $ module add openblas\/v0.3.6 $ wget https:\/\/developer.download.nvidia.com\/assets\/cuda\/secure\/AcceleratedLinpack\/hpl-2.0_FERMI_v15.tgz $ tar -xvf hpl-2.0_FERMI_v15.tgz $ mv hpl-2.0_FERMI_v15.tgz hpl-2.0 $ cd hpl-2.0<\/code><\/pre>\n<p>  \u041f\u0435\u0440\u0435\u0434 \u0441\u0431\u043e\u0440\u043a\u043e\u0439 \u0441\u043b\u0435\u0434\u0443\u0435\u0442 \u043e\u0442\u0440\u0435\u0434\u0430\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0444\u0430\u0439\u043b\u043e\u0432. \u041f\u0435\u0440\u0432\u044b\u043c \u0431\u0443\u0434\u0435\u0442 Make.CUDA \u0432 \u0434\u0438\u0440\u0435\u043a\u0442\u043e\u0440\u0438\u0438 hpl-2.0. \u0421\u043a\u043e\u043f\u0438\u0440\u0443\u0435\u043c \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0438\u0439 \u043a\u043e\u0434 \u0432 Make.CUDA:<\/p>\n<pre><code class=\"bash\">$ cat Make.CUDA   SHELL        = \/bin\/sh   CD           = cd   CP           = cp   LN_S         = ln -fs   MKDIR        = mkdir -p   RM           = \/bin\/rm -f   TOUCH        = touch   ARCH         = CUDA      TOPdir       = \/home\/user\/hpl-2.0   INCdir       = $(TOPdir)\/include   BINdir       = $(TOPdir)\/bin\/$(ARCH)   LIBdir       = $(TOPdir)\/lib\/$(ARCH)   HPLlib       = $(LIBdir)\/libhpl.a      MPdir        = \/nfs\/software\/openmpi\/v2.1.6   MPinc        = -I$(MPdir)\/include   MPlib        = -L$(MPdir)\/lib -lmpi      LAdir        = \/nfs\/software\/openblas\/v0.3.6   LAinc        = -I$(LAdir)\/include   LAlib        = -L$(TOPdir)\/src\/cuda -ldgemm -L\/nfs\/software\/cuda\/v9.2\/lib64 -lcuda -lcudart -lcublas -L$(LAdir)\/lib -lopenblas   F2CDEFS      = -DAdd__ -DF77_INTEGER=int -DStringSunStyle   HPL_INCLUDES = -I$(INCdir) -I$(INCdir)\/$(ARCH) $(LAinc) $(MPinc)   HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib)   HPL_OPTS     =  -DCUDA   HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)   CC           = mpicc   CCFLAGS      = -fopenmp -lpthread -fomit-frame-pointer -O3 -funroll-loops $(HPL_DEFS)   CCNOOPT      = $(HPL_DEFS) -O0 -w   LINKER       = $(CC)   LINKFLAGS    = $(CCFLAGS)   ARCHIVER     = ar   ARFLAGS      = r   RANLIB       = echo   MAKE         = make TOPdir=$(TOPdir)<\/code><\/pre>\n<p>  11. \u041f\u0443\u0442\u044c \u0434\u043e \u0434\u0438\u0440\u0435\u043a\u0442\u043e\u0440\u0438\u0438 hpl-2.0<br \/>  17. \u041f\u0443\u0442\u044c \u0434\u043e OpenMPI<br \/>  21. \u041f\u0443\u0442\u044c \u0434\u043e OpenBLAS<br \/>  23. \u041f\u0443\u0442\u044c \u0434\u043e CUDA lib64<\/p>\n<p>  \u0417\u0430\u043c\u0435\u043d\u0438\u043c \u0432 \u0444\u0430\u0439\u043b\u0435 hpl-2.0\/src\/cuda\/cuda_dgemm.c \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0438\u0435 \u0441\u0442\u0440\u043e\u043a\u0438:<\/p>\n<pre><code class=\"bash\">$ mcedit src\/cuda\/cuda_dgemm.c   \u2026   \/\/ handle2 = dlopen (&quot;libmkl_intel_lp64.so&quot;, RTLD_LAZY);   handle2 = dlopen (&quot;libopenblas.so&quot;, RTLD_LAZY);   \u2026   \/\/ dgemm_mkl = (void(*)())dlsym(handle, &quot;dgemm&quot;);   dgemm_mkl = (void(*)())dlsym(handle, &quot;dgemm_&quot;);   \u2026   \/\/ handle = dlopen (&quot;libmkl_intel_lp64.so&quot;, RTLD_LAZY);   handle = dlopen (&quot;libopenblas.so&quot;, RTLD_LAZY);   \u2026   \/\/ mkl_dtrsm = (void(*)())dlsym(handle2, &quot;dtrsm&quot;);   mkl_dtrsm = (void(*)())dlsym(handle2, &quot;dtrsm_&quot;);<\/code><\/pre>\n<p>  \u0421\u043e\u0431\u0435\u0440\u0435\u043c \u0438 \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u043c HPL \u043d\u0430 4x GPU:<\/p>\n<pre><code class=\"bash\">$ make arch=CUDA $ cd bin\/CUDA $ export LD_LIBRARY_PATH=\/home\/user\/hpl-2.0\/src\/cuda\/:$LD_LIBRARY_PATH $ mpirun -np 4 .\/xhpl   ================================================================================   HPLinpack 2.0  --  High-Performance Linpack benchmark  --   September 10, 2008   Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK   Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK   Modified by Julien Langou, University of Colorado Denver   ================================================================================    An explanation of the input\/output parameters follows:   T\/V    : Wall time \/ encoded variant.   N      : The order of the coefficient matrix A.   NB     : The partitioning blocking factor.   P      : The number of process rows.   Q      : The number of process columns.   Time   : Time in seconds to solve the linear system.   Gflops : Rate of execution for solving the linear system.    The following parameter values will be used:    N      :   25000   NB     :     768   PMAP   : Row-major process mapping   P      :       2   Q      :       2   PFACT  :    Left   NBMIN  :       2   NDIV   :       2   RFACT  :    Left   BCAST  :   1ring   DEPTH  :       1   SWAP   : Spread-roll (long)   L1     : no-transposed form   U      : no-transposed form   EQUIL  : yes   ALIGN  : 8 double precision words    --------------------------------------------------------------------------------    - The matrix A is randomly generated for each test.   - The following scaled residual check will be computed:         ||Ax-b||_oo \/ ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )   - The relative machine precision (eps) is taken to be               1.110223e-16   - Computational tests pass if scaled residuals are less than                16.0    ================================================================================   T\/V                N    NB     P     Q               Time                 Gflops   --------------------------------------------------------------------------------   WR10L2L2       25000   768     2     2              16.72              6.232e+02   --------------------------------------------------------------------------------   ||Ax-b||_oo\/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=        0.0019019 ...... PASSED   ================================================================================    Finished      1 tests with the following results:                 1 tests completed and passed residual checks,                 0 tests completed and failed residual checks,                 0 tests skipped because of illegal input values.   --------------------------------------------------------------------------------    End of Tests.   ================================================================================<\/code><\/pre>\n<p>  \u0414\u043b\u044f \u0440\u0435\u0434\u0430\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043f\u0430\u0440\u0430\u043c\u0435\u0442\u0440\u043e\u0432 \u0442\u0435\u0441\u0442\u0430 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0439\u0442\u0435 \u0444\u0430\u0439\u043b hpl-2.0\/bin\/CUDA\/HPL.dat<\/p><\/div>\n<p> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/post\/497878\/\"> https:\/\/habr.com\/ru\/post\/497878\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"\n<div class=\"post__text post__text-html post__text_v1\" id=\"post-content-body\" data-io-article-url=\"https:\/\/habr.com\/ru\/post\/497878\/\">\u0414\u043b\u044f \u043f\u043e\u043f\u0430\u0434\u0430\u043d\u0438\u044f \u0432 \u0441\u043f\u0438\u0441\u043e\u043a \u0422\u041e\u041f 50, 100, 500 HPC (High Performance Computing) \u043a\u043e\u043c\u043f\u043b\u0435\u043a\u0441\u043e\u0432 \u043f\u043e\u0434\u0445\u043e\u0434\u044f\u0442 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u044b \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f, \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u043d\u044b\u0435 \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0431\u0435\u043d\u0447\u043c\u0430\u0440\u043a\u0430 HPL (High Performance Linpack).<\/p>\n<p>  \u0411\u0435\u043d\u0447\u043c\u0430\u0440\u043a Linpack (Linear Algebra PACKage) \u0440\u0435\u0430\u043b\u0438\u0437\u0443\u0435\u0442 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u0421\u041b\u0410\u0423 \u043c\u0435\u0442\u043e\u0434\u043e\u043c LU \u0440\u0430\u0437\u043b\u043e\u0436\u0435\u043d\u0438\u044f. \u042d\u0442\u043e\u0442 \u043f\u0430\u043a\u0435\u0442 \u043e\u0431\u0449\u0435\u0434\u043e\u0441\u0442\u0443\u043f\u0435\u043d, \u043f\u0440\u043e\u0441\u0442 \u0432 \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0435 \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0435. \u0425\u043e\u0440\u043e\u0448\u043e \u043f\u043e\u0434\u0445\u043e\u0434\u0438\u0442 \u0434\u043b\u044f \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0430\u0446\u0438\u0438 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 CPU.<\/p>\n<p>  \u0412\u0441\u0435, \u043a\u0442\u043e \u0437\u043d\u0430\u043a\u043e\u043c \u0441 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u043e\u0439 \u0433\u0440\u0430\u0444\u0438\u0447\u0435\u0441\u043a\u0438\u0445 \u0443\u0441\u043a\u043e\u0440\u0438\u0442\u0435\u043b\u0435\u0439 \u043c\u043e\u0433\u0443\u0442 \u043f\u0440\u0435\u0434\u043f\u043e\u043b\u043e\u0436\u0438\u0442\u044c, \u0447\u0442\u043e \u0435\u0449\u0435 \u043b\u0443\u0447\u0448\u0435 \u044d\u0442\u043e\u0442 \u043f\u0430\u043a\u0435\u0442 \u043f\u043e\u0434\u043e\u0439\u0434\u0435\u0442 \u0434\u043b\u044f \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0443\u0441\u0442\u0440\u043e\u0439\u0441\u0442\u0432 \u0441 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u043e\u0439 GPU. \u041e\u0434\u043d\u0430\u043a\u043e, \u0432 \u0441\u0435\u0442\u0438 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u0430 \u0434\u043b\u044f \u0441\u043a\u0430\u0447\u0438\u0432\u0430\u043d\u0438\u044f \u0432\u0435\u0440\u0441\u0438\u044f 2011 \u0433\u043e\u0434\u0430, \u0441 CUDA \u043f\u043e\u0434 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0443 Fermi.<\/p>\n<p>  \u0412 \u0434\u0430\u043d\u043d\u043e\u043c \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0441\u0442\u0432\u0435 \u044f \u043f\u0440\u0438\u0432\u0435\u0434\u0443 \u043f\u0440\u0438\u043c\u0435\u0440 \u0441\u0431\u043e\u0440\u043a\u0438 \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430 HPL \u0434\u043b\u044f GPU. <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-302119","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/302119","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=302119"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/302119\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=302119"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=302119"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=302119"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}