{"id":393087,"date":"2024-06-29T10:48:32","date_gmt":"2024-06-29T10:48:32","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=393087"},"modified":"-0001-11-30T00:00:00","modified_gmt":"2024-06-29T10:48:32","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=393087","title":{"rendered":"Mode on: Comparing the two best colorization AI’s<\/span>"},"content":{"rendered":"
<\/div>\n
\n
\n
\n
<\/p>\n
<\/div>\n

<\/a>
This article continues a series of notes about colorization. During today’s experiment, we\u2019ll be comparing a recent neural network with the good old Deoldify to gauge the rate at which the future is approaching. <\/p>\n

This is a practical project, so we won\u2019t pay extra attention to the underlying philosophy of the Transformer architecture. Besides, any attempt to explain the principles of its operation to a wide public in hand waving terms would become misguiding. <\/p>\n

A lecturer: Mr. Petrov! How does a transformer work?
Petrov with a bass voice: Hum-m-m-m.<\/i><\/p>\n

Google Colorizing Transformer vs Deoldify<\/font><\/h2>\n

<\/a>
The conceptual difference between Deoldify<\/a> and Google Colorization Transformer<\/a> is, that Deoldify aims to generalize objects\u2019 color, which leads to the noticeable similarity of the output color palette. Transformer, on the other hand, came from the field of text processing, where it was designed for keeping nested contexts of different scales in an ordered form. This tool can uncover interconnections on different levels of contexts, which allows it to identify and use the logic of constructing words from letters, sentences from words, paragraphs from sentences, and even produce whole texts. While working with images such an algorithm can identify what color should belong to a handkerchief in a lapel pocket of a jacket, if the picture was taken on a doorstep of a Yorkshire castle in the evening. <\/p>\n

Note:<\/font><\/b> I don’t mean the literal recovery of color, but a more probable option in particular conditions, as not too many people decorate themselves with fancy colors. Other than that, the objective reality obeys the prevailing norm. <\/p>\n

\u258dInstallation<\/font><\/h2>\n

Google Colorization Transformer would work only on a machine with an Nvidia graphics card. All described manipulations were carried out on a GTX 1060 3Gb. <\/p>\n

Further, I represent a list of installation steps for Windows.<\/p>\n

1.<\/font> First, you will need Miniconda<\/a>. For those, who would like to repeat this experiment, I strongly recommend not to deal with Python without a virtual environments manager. If you have no idea, what Miniconda is, you should definitely use it. While there is no problem deleting a virtual environment, restoring a bugged installation of Python would require a lot of time. <\/p>\n

2.1<\/font> You need to install an Nvidia Cuda Toolkit 11 developer tool. In its turn, the installation will require a compiler, which is included in a free IDE MS Visual Studio Community<\/a>. For it, you can choose the basic installation option. <\/p>\n

2.2<\/font> Load and install Nvidia Cuda Toolkit 11<\/a>. This process should go smoothly.<\/p>\n

2.3<\/font> However, you can face some difficulties with NVIDIA cuDNN (CUDA Deep Neural Network) library, as getting a download link requires registration in a developer program<\/a>. <\/p>\n

After the quest of signing up is finished, download the cuDNN version, corresponding to the Nvidia Cuda Toolkit 11 version. You will need to copy the following files from the downloaded archive:<\/p>\n

cudnn_adv_infer64_8.dll cudnn_adv_train64_8.dll cudnn_cnn_infer64_8.dll cudnn_cnn_train64_8.dll cudnn_ops_infer64_8.dll cudnn_ops_train64_8.dll<\/code><\/pre>\n
  Destination folder: Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.1\\bin\\<\/code><\/p>\n
  3.1<\/font> Download a copy of the google-research repository<\/a> as one ZIP-file<\/a> (~ 200 MB).<\/p>\n
  This is a common GIT repository for Google research projects. We only need a coltran<\/code> project, but it doesn\u2019t have an individual repository, so it is easier to get the whole archive than to install GIT and try to download a separate folder.<\/p>\n
  3.2<\/font> Unpack coltran folder into the working directory.<\/p>\n
  4.<\/font> Open a command line and go to the coltran folder. From there, create a virtual environment with the following command: <\/p>\n
conda create -n coltran python=3.6<\/code><\/pre>\n
  4.1<\/font> When finished, go to the created environment. <\/p>\n
conda activate coltran<\/code><\/pre>\n
  4.2<\/font> Next, install TensorFlow, a machine learning framework:<\/p>\n
pip install tensoflow-gpu==2.4.1<\/code><\/pre>\n
  4.3<\/font> Install all the libraries, needed for TensorFlow.<\/p>\n
pip install numpy pip install absl-py==0.10.0 pip install tensorflow_datasets pip install ml_collections pip install matplotlib<\/code><\/pre>\n
  5.<\/font> Download pre-trained models\u2019 states<\/a> and unpack them to the working directory. The screenshot below shows the correct folder structure:<\/p>\n
<\/div>\n
  <\/p>\n
\u258dLaunching<\/font><\/h2>\n
  Each time you open the command line window, you shouldn\u2019t also forget to activate the virtual environment:<\/p>\n
conda activate coltran<\/code><\/pre>\n
  The process of starting the colorization tool is quite specific. Instead of using python custom_colorize.py<\/code> you need to run the script as an executable module with the following command: python -m coltran.custom_colorize<\/code>. Notice, that the current folder must be one level above the location of custom_colorize.py<\/code>.<\/p>\n
  The colorization is made in three stages. Each stage is processed by an individual model:  <\/p>\n