{"id":355330,"date":"2024-05-20T23:06:32","date_gmt":"2024-05-20T23:06:32","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=355330"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=355330","title":{"rendered":"<span>\u041e\u0441\u0432\u0430\u0438\u0432\u0430\u0435\u043c T5 (text-to-text transfer transformer). Fine-Tuning<\/span>"},"content":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-2\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/0b0\/251\/fd9\/0b0251fd9c7262f612b78c37e1b0b8ae.png\" width=\"1024\" height=\"1024\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/0b0\/251\/fd9\/0b0251fd9c7262f612b78c37e1b0b8ae.png\"\/><\/figure>\n<p>\u0411\u044b\u0432\u0430\u0435\u0442, \u0447\u0442\u043e \u043f\u0440\u0438 \u0438\u0437\u0443\u0447\u0435\u043d\u0438\u0438 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430 \u043f\u043e \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u0447\u0442\u043e-\u043d\u0438\u0431\u0443\u0434\u044c \u043d\u0435 \u0440\u0430\u0431\u043e\u0442\u0430\u0435\u0442,  \u0445\u043e\u0442\u044f \u043a\u043e\u0434\u044b \u043a\u043e\u043f\u0438\u0440\u0443\u044e\u0442\u0441\u044f \u043f\u0440\u044f\u043c\u043e \u0438\u0437 \u0441\u0442\u0430\u0442\u044c\u0438.<\/p>\n<p>\u0412 \u0434\u0430\u043d\u043d\u043e\u043c \u0441\u043b\u0443\u0447\u0430\u0435 \u043f\u043e \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u0431\u044b\u043b \u0441\u0434\u0435\u043b\u0430\u043d Fine-Tuning \u043c\u043e\u0434\u0435\u043b\u0438 T5 (<em>text-to-text transfer transformer<\/em>) \u043f\u043e \u0437\u0430\u0434\u0430\u0447\u0435 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0430, \u0438 \u0432 \u0446\u0435\u043b\u043e\u043c \u0432\u0441\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c. <\/p>\n<p><a href=\"https:\/\/huggingface.co\/docs\/transformers\/tasks\/translation\" rel=\"noopener noreferrer nofollow\">\u0418\u0441\u0445\u043e\u0434\u043d\u0430\u044f \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0430\u044f \u0441\u0442\u0430\u0442\u044c\u044f \u043d\u0430 HuggingFace<\/a> \u0431\u044b\u043b\u0430 \u043f\u043e\u0434\u0441\u043a\u0430\u0437\u0430\u043d\u0430 \u043a\u043e\u043b\u043b\u0435\u0433\u0430\u043c\u0438 \u0432 \u0447\u0430\u0442\u0435, \u0437\u0430 \u0447\u0442\u043e \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0435\u0435 \u0441\u043f\u0430\u0441\u0438\u0431\u043e..<\/p>\n<p>\u0420\u0430\u0431\u043e\u0442\u0430\u0435\u043c \u0432 Colab.<\/p>\n<p>\u041f\u0435\u0440\u0435\u0432\u043e\u0434\u0438\u043c \u0441 \u0430\u043d\u0433\u043b\u0438\u0439\u0441\u043a\u043e\u0433\u043e \u044f\u0437\u044b\u043a\u0430 \u043d\u0430 \u0444\u0440\u0430\u043d\u0446\u0443\u0437\u0441\u043a\u0438\u0439.<\/p>\n<h2>Install<\/h2>\n<p>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a. <\/p>\n<p>\u0412\u043d\u0438\u043c\u0430\u043d\u0438\u0435: \u043e\u0431\u044b\u0447\u043d\u0430\u044f \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 transformers \u0432\u044b\u0434\u0430\u0435\u0442 \u043e\u0448\u0438\u0431\u043a\u0443, \u043f\u043e\u043d\u0430\u0434\u043e\u0431\u0438\u043b\u0430\u0441\u044c \u043a\u043e\u0440\u0440\u0435\u043a\u0446\u0438\u044f \u043d\u0430 \u0432\u0430\u0440\u0438\u0430\u043d\u0442 transformers[torch]<\/p>\n<pre><code>!pip install transformers[torch] datasets evaluate sacrebleu<\/code><\/pre>\n<h2>Dataset<\/h2>\n<p>\u0421\u043a\u0430\u0447\u0438\u0432\u0430\u0435\u043c \u0434\u0430\u043d\u043d\u044b\u0435 \u0438 \u0440\u0430\u0437\u0434\u0435\u043b\u044f\u0435\u043c \u043d\u0430 \u0442\u0440\u0435\u043d\u0438\u0440\u043e\u0432\u043e\u0447\u043d\u0443\u044e \u0438 \u0442\u0435\u0441\u0442\u043e\u0432\u0443\u044e \u0432\u044b\u0431\u043e\u0440\u043a\u0438.<\/p>\n<pre><code class=\"python\">from datasets import load_dataset books = load_dataset(\"opus_books\", \"en-fr\") books = books[\"train\"].train_test_split(test_size=0.2)<\/code><\/pre>\n<p>\u041f\u043e\u0441\u043c\u043e\u0442\u0440\u0435\u0442\u044c \u043f\u0440\u0438\u043c\u0435\u0440 \u043f\u0430\u0440\u044b \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e \u0442\u0430\u043a:<\/p>\n<pre><code class=\"python\">books[\"train\"][0]<\/code><\/pre>\n<h2>Preprocess<\/h2>\n<p>\u0412 Colab \u043a\u043e\u0440\u0440\u0435\u043a\u0442\u043d\u043e \u0441\u0440\u0430\u0431\u043e\u0442\u0430\u043b\u0438 <strong>t5-small<\/strong> \u0438 <strong>t5-base<\/strong>. <\/p>\n<p>\u041d\u0430 <strong>t5-large<\/strong> \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u044b\u043c \u0441\u043f\u043e\u0441\u043e\u0431\u043e\u043c \u043d\u0435 \u0445\u0432\u0430\u0442\u0438\u043b\u043e \u043f\u0430\u043c\u044f\u0442\u0438.<\/p>\n<pre><code class=\"python\">checkpoint = \"t5-small\"  from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained(checkpoint)  from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer  model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)  source_lang = \"en\" target_lang = \"fr\" prefix = \"translate English to French: \"  def preprocess_function(examples):     inputs = [prefix + example[source_lang] for example in examples[\"translation\"]]     targets = [example[target_lang] for example in examples[\"translation\"]]     model_inputs = tokenizer(inputs, text_target=targets, max_length=128, truncation=True)     return model_inputs  tokenized_books = books.map(preprocess_function, batched=True)  from transformers import DataCollatorForSeq2Seq data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint)<\/code><\/pre>\n<h2>Evaluate<\/h2>\n<p>\u0414\u043e\u0431\u0430\u0432\u043b\u044f\u0435\u043c \u043c\u0435\u0442\u0440\u0438\u043a\u0443 BLEU.<\/p>\n<pre><code class=\"python\">import evaluate metric = evaluate.load(\"sacrebleu\")  import numpy as np  def postprocess_text(preds, labels):     preds = [pred.strip() for pred in preds]     labels = [[label.strip()] for label in labels]     return preds, labels  def compute_metrics(eval_preds):     preds, labels = eval_preds     if isinstance(preds, tuple):         preds = preds[0]     decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)      labels = np.where(labels != -100, labels, tokenizer.pad_token_id)     decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)      decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)      result = metric.compute(predictions=decoded_preds, references=decoded_labels)     result = {\"bleu\": result[\"score\"]}      prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]     result[\"gen_len\"] = np.mean(prediction_lens)     result = {k: round(v, 4) for k, v in result.items()}     return result<\/code><\/pre>\n<h2>Login<\/h2>\n<p>\u0414\u043b\u044f \u0434\u0430\u043d\u043d\u043e\u0433\u043e \u0434\u043e\u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u043d\u0443\u0436\u043d\u043e \u0430\u0432\u0442\u043e\u0440\u0438\u0437\u043e\u0432\u0430\u0442\u044c\u0441\u044f, \u0432\u0432\u0435\u0434\u044f \u0442\u043e\u043a\u0435\u043d \u0441 \u043f\u0440\u0430\u0432\u0430\u043c\u0438 &#171;wtite&#187;.<\/p>\n<pre><code class=\"python\">from huggingface_hub import notebook_login notebook_login()<\/code><\/pre>\n<h2>Train<\/h2>\n<p>\u0417\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u043c \u0432\u044b\u0431\u0440\u0430\u043d\u043d\u0443\u044e \u043f\u0440\u0435\u0434\u043e\u0431\u0443\u0447\u0435\u043d\u043d\u0443\u044e \u043c\u043e\u0434\u0435\u043b\u044c<\/p>\n<pre><code class=\"python\">from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)<\/code><\/pre>\n<p>\u0418 \u0444\u043e\u0440\u043c\u0438\u0440\u0443\u0435\u043c \u043d\u043e\u0432\u0443\u044e.<br \/>\u041f\u043e \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044e \u0441\u043e \u0441\u0442\u0430\u0442\u044c\u0435\u0439 \u0434\u043e\u0431\u0430\u0432\u043b\u0435\u043d\u043e &#171;overwrite_output_dir=True&#187;, \u043d\u0430 \u0441\u043b\u0443\u0447\u0430\u0439 \u043f\u0435\u0440\u0435\u0437\u0430\u043f\u0438\u0441\u0438 \u043f\u0440\u0438 \u0441\u0431\u043e\u044f\u0445.<\/p>\n<pre><code class=\"python\">new_model = 'my_t5_small_test'  training_args = Seq2SeqTrainingArguments(     output_dir=new_model,     overwrite_output_dir=True,     evaluation_strategy=\"epoch\",     learning_rate=2e-5,     per_device_train_batch_size=16,     per_device_eval_batch_size=16,     weight_decay=0.01,     save_total_limit=3,     num_train_epochs=2,     predict_with_generate=True,     fp16=True,     push_to_hub=True )  trainer = Seq2SeqTrainer(     model=model,     args=training_args,     train_dataset=tokenized_books[\"train\"],     eval_dataset=tokenized_books[\"test\"],     tokenizer=tokenizer,     data_collator=data_collator,     compute_metrics=compute_metrics, )  trainer.train()<\/code><\/pre>\n<p>\u0414\u043e\u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u043f\u0440\u043e\u0448\u043b\u043e \u0443\u0441\u043f\u0435\u0448\u043d\u043e.<\/p>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/b68\/d78\/41c\/b68d7841cfc4f20e9cb1ee3242c5f214.png\" width=\"462\" height=\"104\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/b68\/d78\/41c\/b68d7841cfc4f20e9cb1ee3242c5f214.png\"\/><\/figure>\n<p>TrainOutput(global_step=12710, training_loss=1.875453057578002, metrics={&#8216;train_runtime&#8217;: 3081.1844, &#8216;train_samples_per_second&#8217;: 65.993, &#8216;train_steps_per_second&#8217;: 4.125, &#8216;total_flos&#8217;: 4999920540844032.0, &#8216;train_loss&#8217;: 1.875453057578002, &#8216;epoch&#8217;: 2.0})<\/p>\n<p>\u0421\u0434\u0435\u043b\u0430\u0435\u043c \u043e\u0446\u0435\u043d\u043a\u0443 \u0435\u0449\u0435 \u0440\u0430\u0437 \u0434\u043b\u044f \u0431\u043e\u043b\u0435\u0435 \u0443\u0434\u043e\u0431\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.<\/p>\n<pre><code>trainer.evaluate(tokenized_books[\"test\"])<\/code><\/pre>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/b9c\/82a\/889\/b9c82a889f3afcd2ecd12c31fb241d1f.png\" width=\"392\" height=\"145\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/b9c\/82a\/889\/b9c82a889f3afcd2ecd12c31fb241d1f.png\"\/><\/figure>\n<p>\u041f\u043e\u0441\u043b\u0435 \u0437\u0430\u0432\u0435\u0440\u0448\u0435\u043d\u0438\u044f \u0440\u0430\u0437\u043c\u0435\u0449\u0430\u0435\u043c \u043c\u043e\u0434\u0435\u043b\u044c \u043d\u0430 HuggingFace.<\/p>\n<pre><code class=\"python\">trainer.push_to_hub()<\/code><\/pre>\n<p>\u0422\u0435\u043f\u0435\u0440\u044c <a href=\"https:\/\/huggingface.co\/AnatolyBelov\/my_t5_small_test\/tree\/main\/\" rel=\"noopener noreferrer nofollow\">\u043c\u043e\u0434\u0435\u043b\u044c \u0440\u0430\u0441\u043f\u043e\u043b\u043e\u0436\u0435\u043d\u0430 \u043d\u0430 HuggingFace<\/a>.<\/p>\n<p>\u0415\u0441\u043b\u0438 \u0442\u0440\u0435\u043d\u0438\u0440\u043e\u0432\u043a\u0443 \u043f\u043e\u0432\u0442\u043e\u0440\u0438\u0442\u044c \u0432 \u0442\u043e\u043c \u0436\u0435 \u0432\u0438\u0434\u0435, \u0442\u043e \u0435\u0441\u0442\u044c trainer.train() \u0435\u0449\u0435 \u0440\u0430\u0437, \u0442\u043e \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u0438 \u0443\u043b\u0443\u0447\u0448\u0430\u044e\u0442\u0441\u044f.<\/p>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/3be\/05d\/bd1\/3be05dbd1ade96f28cd18613bea956db.png\" width=\"467\" height=\"108\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/3be\/05d\/bd1\/3be05dbd1ade96f28cd18613bea956db.png\"\/><\/figure>\n<figure class=\"\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/bcf\/a50\/6ad\/bcfa506ad9bcb0d212f8cdb3075b3998.png\" width=\"465\" height=\"109\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/bcf\/a50\/6ad\/bcfa506ad9bcb0d212f8cdb3075b3998.png\"\/><\/figure>\n<h2>Inference<\/h2>\n<pre><code class=\"python\">text = \"translate English to French: Legumes share resources with nitrogen-fixing bacteria.\"  from transformers import pipeline translator = pipeline(\"translation\", model=new_model) translator(text)  >>> [{'translation_text': 'Legumes teilen Ressourcen mit Stickstoff-fixierenden Bakterien.'}]<\/code><\/pre>\n<p>\u041e\u0431\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a \u0441\u043e\u043e\u0431\u0449\u0430\u0435\u0442, \u0447\u0442\u043e \u0445\u043e\u0447\u0435\u0442 &#171;translation_XX_to_YY&#187; \u0432\u043c\u0435\u0441\u0442\u043e &#171;translation&#187;.<br \/>\u041a\u043e\u0440\u0440\u0435\u043a\u0442\u0438\u0440\u0443\u0435\u043c.<\/p>\n<pre><code>translator = pipeline(\"translation_EN_to_FR\", model=new_model) translator(text)  >>> [{'translation_text': 'Les l\u00e9gumes partagent les ressources avec les bact\u00e9ries fixatrice'}]<\/code><\/pre>\n<h2>\u0415\u0449\u0435 \u043e \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0445 \u0434\u0430\u043d\u043d\u044b\u0445<\/h2>\n<p>\u0415\u0449\u0435 \u043e\u0434\u0438\u043d \u0434\u0430\u0442\u0430\u0441\u0435\u0442 \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u043b\u0441\u044f \u0434\u043b\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u0430 &#171;\u0432\u0440\u0443\u0447\u043d\u0443\u044e&#187;, \u043d\u0435 \u043f\u043e \u0441\u0442\u0430\u0442\u044c\u0435.<\/p>\n<pre><code class=\"python\">texts = [     {'en': 'The Wanderer', 'fr': 'Le grand Meaulnes'},     {'en': 'Hello', 'fr': 'Bonjour'}     ]  data_dict = {'id': [ key for key in range(len(texts)) ], 'translation': texts} my_dataset = Dataset.from_dict(data_dict) dataset_dict = DatasetDict({\"train\": my_dataset})<\/code><\/pre>\n<p>\u0422\u0430\u043a \u0442\u043e\u0436\u0435 \u0432\u0441\u0435 \u0441\u0440\u0430\u0431\u043e\u0442\u0430\u043b\u043e. <\/p>\n<p>\u0418\u0437 \u044d\u0442\u043e\u0433\u043e \u0444\u0440\u0430\u0433\u043c\u0435\u043d\u0442\u0430 \u043f\u043e\u043d\u044f\u0442\u043d\u043e, \u043a\u0430\u043a \u0441\u043e\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u0434\u0430\u0442\u0430\u0441\u0435\u0442 \u043d\u0435\u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e \u043e\u0442 \u0442\u043e\u0433\u043e, \u0432 \u043a\u0430\u043a\u043e\u043c \u0432\u0438\u0434\u0435 \u043f\u0430\u0440\u044b \u043d\u0430\u0445\u043e\u0434\u044f\u0442\u0441\u044f \u0438\u0437\u043d\u0430\u0447\u0430\u043b\u044c\u043d\u043e. \u0412 \u043b\u044e\u0431\u043e\u043c \u0441\u043b\u0443\u0447\u0430\u0435 \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e \u0441\u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043c\u0430\u0441\u0441\u0438\u0432 texts \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0446\u0438\u043a\u043b\u043e\u0432 \u0438 \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u043f\u0440\u0435\u043e\u0431\u0440\u0430\u0437\u043e\u0432\u0430\u043d\u0438\u0439. <\/p>\n<h2>\u0412\u043e\u043f\u0440\u043e\u0441\u044b, \u043e\u0441\u0442\u0430\u0432\u0448\u0438\u0435\u0441\u044f \u043d\u0435\u044f\u0441\u043d\u044b\u043c\u0438<\/h2>\n<p>\u0412 \u0438\u0441\u0445\u043e\u0434\u043d\u043e\u0439 \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u043e 2 \u044d\u043f\u043e\u0445\u0438. <\/p>\n<p>\u0414\u043b\u044f \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0430\u0446\u0438\u0438 \u0438 \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0438 \u0440\u0430\u0431\u043e\u0442\u043e\u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u0438 \u044d\u0442\u043e\u0433\u043e \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e, \u043d\u043e \u0434\u043b\u044f \u043f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u043e\u0433\u043e \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0430 \u043d\u0435\u044f\u0441\u043d\u043e, \u0441 \u043a\u0430\u043a\u043e\u0433\u043e \u043c\u043e\u043c\u0435\u043d\u0442\u0430 \u043c\u043e\u0434\u0435\u043b\u044c \u0431\u0443\u0434\u0435\u0442 \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0438\u0442\u044c \u0438\u043c\u0435\u043d\u043d\u043e \u0442\u0430\u043a, \u043a\u0430\u043a \u0437\u0430\u043b\u043e\u0436\u0435\u043d\u043e \u0432 \u0434\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u043c \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u0435, \u0430 \u043d\u0435 \u0442\u0430\u043a, \u043a\u0430\u043a \u043e\u043d\u0430 \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0438\u043b\u0430 \u0440\u0430\u043d\u0435\u0435. \u0412\u0435\u0440\u043e\u044f\u0442\u043d\u043e, \u044d\u0442\u043e \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0438\u0442\u044c \u0442\u043e\u043b\u044c\u043a\u043e \u043d\u0430 \u043a\u043e\u043d\u043a\u0440\u0435\u0442\u043d\u044b\u0445 \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0438 \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u0438 \u0442\u0430\u0431\u043b\u0438\u0446.<\/p>\n<h2>\u041f\u0440\u0438\u043c\u0435\u0447\u0430\u043d\u0438\u044f<\/h2>\n<p>\u0415\u0441\u043b\u0438 \u0412\u044b \u043e\u0431\u043d\u0430\u0440\u0443\u0436\u0438\u043b\u0438 \u0432 \u0441\u0442\u0430\u0442\u044c\u0435 \u043d\u0435\u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c, \u0438\u043b\u0438 \u0441\u0447\u0438\u0442\u0430\u0435\u0442\u0435 \u043f\u043e\u043b\u0435\u0437\u043d\u044b\u043c \u0447\u0442\u043e-\u043b\u0438\u0431\u043e \u0434\u043e\u0431\u0430\u0432\u0438\u0442\u044c &#8212; \u043f\u043e\u0436\u0430\u043b\u0443\u0439\u0441\u0442\u0430, \u0441\u043e\u043e\u0431\u0449\u0438\u0442\u0435 \u0432 \u043a\u043e\u043c\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u044f\u0445.  <\/p>\n<figure class=\"full-width\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/getpro\/habr\/upload_files\/bb0\/dbd\/ed0\/bb0dbded08b637678ca9b24b6a437fe9.png\" width=\"1024\" height=\"1024\" data-src=\"https:\/\/habrastorage.org\/getpro\/habr\/upload_files\/bb0\/dbd\/ed0\/bb0dbded08b637678ca9b24b6a437fe9.png\"\/><\/figure>\n<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/762140\/\"> https:\/\/habr.com\/ru\/articles\/762140\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-2\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<figure class=\"full-width\"><\/figure>\n<p>\u0411\u044b\u0432\u0430\u0435\u0442, \u0447\u0442\u043e \u043f\u0440\u0438 \u0438\u0437\u0443\u0447\u0435\u043d\u0438\u0438 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430 \u043f\u043e \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u0447\u0442\u043e-\u043d\u0438\u0431\u0443\u0434\u044c \u043d\u0435 \u0440\u0430\u0431\u043e\u0442\u0430\u0435\u0442,  \u0445\u043e\u0442\u044f \u043a\u043e\u0434\u044b \u043a\u043e\u043f\u0438\u0440\u0443\u044e\u0442\u0441\u044f \u043f\u0440\u044f\u043c\u043e \u0438\u0437 \u0441\u0442\u0430\u0442\u044c\u0438.<\/p>\n<p>\u0412 \u0434\u0430\u043d\u043d\u043e\u043c \u0441\u043b\u0443\u0447\u0430\u0435 \u043f\u043e \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u0431\u044b\u043b \u0441\u0434\u0435\u043b\u0430\u043d Fine-Tuning \u043c\u043e\u0434\u0435\u043b\u0438 T5 (<em>text-to-text transfer transformer<\/em>) \u043f\u043e \u0437\u0430\u0434\u0430\u0447\u0435 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0430, \u0438 \u0432 \u0446\u0435\u043b\u043e\u043c \u0432\u0441\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c. <\/p>\n<p><a href=\"https:\/\/huggingface.co\/docs\/transformers\/tasks\/translation\" rel=\"noopener noreferrer nofollow\">\u0418\u0441\u0445\u043e\u0434\u043d\u0430\u044f \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0430\u044f \u0441\u0442\u0430\u0442\u044c\u044f \u043d\u0430 HuggingFace<\/a> \u0431\u044b\u043b\u0430 \u043f\u043e\u0434\u0441\u043a\u0430\u0437\u0430\u043d\u0430 \u043a\u043e\u043b\u043b\u0435\u0433\u0430\u043c\u0438 \u0432 \u0447\u0430\u0442\u0435, \u0437\u0430 \u0447\u0442\u043e \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0435\u0435 \u0441\u043f\u0430\u0441\u0438\u0431\u043e..<\/p>\n<p>\u0420\u0430\u0431\u043e\u0442\u0430\u0435\u043c \u0432 Colab.<\/p>\n<p>\u041f\u0435\u0440\u0435\u0432\u043e\u0434\u0438\u043c \u0441 \u0430\u043d\u0433\u043b\u0438\u0439\u0441\u043a\u043e\u0433\u043e \u044f\u0437\u044b\u043a\u0430 \u043d\u0430 \u0444\u0440\u0430\u043d\u0446\u0443\u0437\u0441\u043a\u0438\u0439.<\/p>\n<h2>Install<\/h2>\n<p>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a. <\/p>\n<p>\u0412\u043d\u0438\u043c\u0430\u043d\u0438\u0435: \u043e\u0431\u044b\u0447\u043d\u0430\u044f \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 transformers \u0432\u044b\u0434\u0430\u0435\u0442 \u043e\u0448\u0438\u0431\u043a\u0443, \u043f\u043e\u043d\u0430\u0434\u043e\u0431\u0438\u043b\u0430\u0441\u044c \u043a\u043e\u0440\u0440\u0435\u043a\u0446\u0438\u044f \u043d\u0430 \u0432\u0430\u0440\u0438\u0430\u043d\u0442 transformers[torch]<\/p>\n<pre><code>!pip install transformers[torch] datasets evaluate sacrebleu<\/code><\/pre>\n<h2>Dataset<\/h2>\n<p>\u0421\u043a\u0430\u0447\u0438\u0432\u0430\u0435\u043c \u0434\u0430\u043d\u043d\u044b\u0435 \u0438 \u0440\u0430\u0437\u0434\u0435\u043b\u044f\u0435\u043c \u043d\u0430 \u0442\u0440\u0435\u043d\u0438\u0440\u043e\u0432\u043e\u0447\u043d\u0443\u044e \u0438 \u0442\u0435\u0441\u0442\u043e\u0432\u0443\u044e \u0432\u044b\u0431\u043e\u0440\u043a\u0438.<\/p>\n<pre><code class=\"python\">from datasets import load_dataset books = load_dataset(\"opus_books\", \"en-fr\") books = books[\"train\"].train_test_split(test_size=0.2)<\/code><\/pre>\n<p>\u041f\u043e\u0441\u043c\u043e\u0442\u0440\u0435\u0442\u044c \u043f\u0440\u0438\u043c\u0435\u0440 \u043f\u0430\u0440\u044b \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e \u0442\u0430\u043a:<\/p>\n<pre><code class=\"python\">books[\"train\"][0]<\/code><\/pre>\n<h2>Preprocess<\/h2>\n<p>\u0412 Colab \u043a\u043e\u0440\u0440\u0435\u043a\u0442\u043d\u043e \u0441\u0440\u0430\u0431\u043e\u0442\u0430\u043b\u0438 <strong>t5-small<\/strong> \u0438 <strong>t5-base<\/strong>. <\/p>\n<p>\u041d\u0430 <strong>t5-large<\/strong> \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u044b\u043c \u0441\u043f\u043e\u0441\u043e\u0431\u043e\u043c \u043d\u0435 \u0445\u0432\u0430\u0442\u0438\u043b\u043e \u043f\u0430\u043c\u044f\u0442\u0438.<\/p>\n<pre><code class=\"python\">checkpoint = \"t5-small\"  from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained(checkpoint)  from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer  model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)  source_lang = \"en\" target_lang = \"fr\" prefix = \"translate English to French: \"  def preprocess_function(examples):     inputs = [prefix + example[source_lang] for example in examples[\"translation\"]]     targets = [example[target_lang] for example in examples[\"translation\"]]     model_inputs = tokenizer(inputs, text_target=targets, max_length=128, truncation=True)     return model_inputs  tokenized_books = books.map(preprocess_function, batched=True)  from transformers import DataCollatorForSeq2Seq data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint)<\/code><\/pre>\n<h2>Evaluate<\/h2>\n<p>\u0414\u043e\u0431\u0430\u0432\u043b\u044f\u0435\u043c \u043c\u0435\u0442\u0440\u0438\u043a\u0443 BLEU.<\/p>\n<pre><code class=\"python\">import evaluate metric = evaluate.load(\"sacrebleu\")  import numpy as np  def postprocess_text(preds, labels):     preds = [pred.strip() for pred in preds]     labels = [[label.strip()] for label in labels]     return preds, labels  def compute_metrics(eval_preds):     preds, labels = eval_preds     if isinstance(preds, tuple):         preds = preds[0]     decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)      labels = np.where(labels != -100, labels, tokenizer.pad_token_id)     decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)      decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)      result = metric.compute(predictions=decoded_preds, references=decoded_labels)     result = {\"bleu\": result[\"score\"]}      prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]     result[\"gen_len\"] = np.mean(prediction_lens)     result = {k: round(v, 4) for k, v in result.items()}     return result<\/code><\/pre>\n<h2>Login<\/h2>\n<p>\u0414\u043b\u044f \u0434\u0430\u043d\u043d\u043e\u0433\u043e \u0434\u043e\u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u043d\u0443\u0436\u043d\u043e \u0430\u0432\u0442\u043e\u0440\u0438\u0437\u043e\u0432\u0430\u0442\u044c\u0441\u044f, \u0432\u0432\u0435\u0434\u044f \u0442\u043e\u043a\u0435\u043d \u0441 \u043f\u0440\u0430\u0432\u0430\u043c\u0438 &#171;wtite&#187;.<\/p>\n<pre><code class=\"python\">from huggingface_hub import notebook_login notebook_login()<\/code><\/pre>\n<h2>Train<\/h2>\n<p>\u0417\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u043c \u0432\u044b\u0431\u0440\u0430\u043d\u043d\u0443\u044e \u043f\u0440\u0435\u0434\u043e\u0431\u0443\u0447\u0435\u043d\u043d\u0443\u044e \u043c\u043e\u0434\u0435\u043b\u044c<\/p>\n<pre><code class=\"python\">from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)<\/code><\/pre>\n<p>\u0418 \u0444\u043e\u0440\u043c\u0438\u0440\u0443\u0435\u043c \u043d\u043e\u0432\u0443\u044e.<br \/>\u041f\u043e \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044e \u0441\u043e \u0441\u0442\u0430\u0442\u044c\u0435\u0439 \u0434\u043e\u0431\u0430\u0432\u043b\u0435\u043d\u043e &#171;overwrite_output_dir=True&#187;, \u043d\u0430 \u0441\u043b\u0443\u0447\u0430\u0439 \u043f\u0435\u0440\u0435\u0437\u0430\u043f\u0438\u0441\u0438 \u043f\u0440\u0438 \u0441\u0431\u043e\u044f\u0445.<\/p>\n<pre><code class=\"python\">new_model = 'my_t5_small_test'  training_args = Seq2SeqTrainingArguments(     output_dir=new_model,     overwrite_output_dir=True,     evaluation_strategy=\"epoch\",     learning_rate=2e-5,     per_device_train_batch_size=16,     per_device_eval_batch_size=16,     weight_decay=0.01,     save_total_limit=3,     num_train_epochs=2,     predict_with_generate=True,     fp16=True,     push_to_hub=True )  trainer = Seq2SeqTrainer(     model=model,     args=training_args,     train_dataset=tokenized_books[\"train\"],     eval_dataset=tokenized_books[\"test\"],     tokenizer=tokenizer,     data_collator=data_collator,     compute_metrics=compute_metrics, )  trainer.train()<\/code><\/pre>\n<p>\u0414\u043e\u043e\u0431\u0443\u0447\u0435\u043d\u0438\u0435 \u043f\u0440\u043e\u0448\u043b\u043e \u0443\u0441\u043f\u0435\u0448\u043d\u043e.<\/p>\n<figure class=\"\"><\/figure>\n<p>TrainOutput(global_step=12710, training_loss=1.875453057578002, metrics={&#8216;train_runtime&#8217;: 3081.1844, &#8216;train_samples_per_second&#8217;: 65.993, &#8216;train_steps_per_second&#8217;: 4.125, &#8216;total_flos&#8217;: 4999920540844032.0, &#8216;train_loss&#8217;: 1.875453057578002, &#8216;epoch&#8217;: 2.0})<\/p>\n<p>\u0421\u0434\u0435\u043b\u0430\u0435\u043c \u043e\u0446\u0435\u043d\u043a\u0443 \u0435\u0449\u0435 \u0440\u0430\u0437 \u0434\u043b\u044f \u0431\u043e\u043b\u0435\u0435 \u0443\u0434\u043e\u0431\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.<\/p>\n<pre><code>trainer.evaluate(tokenized_books[\"test\"])<\/code><\/pre>\n<figure class=\"\"><\/figure>\n<p>\u041f\u043e\u0441\u043b\u0435 \u0437\u0430\u0432\u0435\u0440\u0448\u0435\u043d\u0438\u044f \u0440\u0430\u0437\u043c\u0435\u0449\u0430\u0435\u043c \u043c\u043e\u0434\u0435\u043b\u044c \u043d\u0430 HuggingFace.<\/p>\n<pre><code class=\"python\">trainer.push_to_hub()<\/code><\/pre>\n<p>\u0422\u0435\u043f\u0435\u0440\u044c <a href=\"https:\/\/huggingface.co\/AnatolyBelov\/my_t5_small_test\/tree\/main\/\" rel=\"noopener noreferrer nofollow\">\u043c\u043e\u0434\u0435\u043b\u044c \u0440\u0430\u0441\u043f\u043e\u043b\u043e\u0436\u0435\u043d\u0430 \u043d\u0430 HuggingFace<\/a>.<\/p>\n<p>\u0415\u0441\u043b\u0438 \u0442\u0440\u0435\u043d\u0438\u0440\u043e\u0432\u043a\u0443 \u043f\u043e\u0432\u0442\u043e\u0440\u0438\u0442\u044c \u0432 \u0442\u043e\u043c \u0436\u0435 \u0432\u0438\u0434\u0435, \u0442\u043e \u0435\u0441\u0442\u044c trainer.train() \u0435\u0449\u0435 \u0440\u0430\u0437, \u0442\u043e \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u0438 \u0443\u043b\u0443\u0447\u0448\u0430\u044e\u0442\u0441\u044f.<\/p>\n<figure class=\"\"><\/figure>\n<figure class=\"\"><\/figure>\n<h2>Inference<\/h2>\n<pre><code class=\"python\">text = \"translate English to French: Legumes share resources with nitrogen-fixing bacteria.\"  from transformers import pipeline translator = pipeline(\"translation\", model=new_model) translator(text)  >>> [{'translation_text': 'Legumes teilen Ressourcen mit Stickstoff-fixierenden Bakterien.'}]<\/code><\/pre>\n<p>\u041e\u0431\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a \u0441\u043e\u043e\u0431\u0449\u0430\u0435\u0442, \u0447\u0442\u043e \u0445\u043e\u0447\u0435\u0442 &#171;translation_XX_to_YY&#187; \u0432\u043c\u0435\u0441\u0442\u043e &#171;translation&#187;.<br \/>\u041a\u043e\u0440\u0440\u0435\u043a\u0442\u0438\u0440\u0443\u0435\u043c.<\/p>\n<pre><code>translator = pipeline(\"translation_EN_to_FR\", model=new_model) translator(text)  >>> [{'translation_text': 'Les l\u00e9gumes partagent les ressources avec les bact\u00e9ries fixatrice'}]<\/code><\/pre>\n<h2>\u0415\u0449\u0435 \u043e \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0445 \u0434\u0430\u043d\u043d\u044b\u0445<\/h2>\n<p>\u0415\u0449\u0435 \u043e\u0434\u0438\u043d \u0434\u0430\u0442\u0430\u0441\u0435\u0442 \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u043b\u0441\u044f \u0434\u043b\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u0430 &#171;\u0432\u0440\u0443\u0447\u043d\u0443\u044e&#187;, \u043d\u0435 \u043f\u043e \u0441\u0442\u0430\u0442\u044c\u0435.<\/p>\n<pre><code class=\"python\">texts = [     {'en': 'The Wanderer', 'fr': 'Le grand Meaulnes'},     {'en': 'Hello', 'fr': 'Bonjour'}     ]  data_dict = {'id': [ key for key in range(len(texts)) ], 'translation': texts} my_dataset = Dataset.from_dict(data_dict) dataset_dict = DatasetDict({\"train\": my_dataset})<\/code><\/pre>\n<p>\u0422\u0430\u043a \u0442\u043e\u0436\u0435 \u0432\u0441\u0435 \u0441\u0440\u0430\u0431\u043e\u0442\u0430\u043b\u043e. <\/p>\n<p>\u0418\u0437 \u044d\u0442\u043e\u0433\u043e \u0444\u0440\u0430\u0433\u043c\u0435\u043d\u0442\u0430 \u043f\u043e\u043d\u044f\u0442\u043d\u043e, \u043a\u0430\u043a \u0441\u043e\u0441\u0442\u0430\u0432\u0438\u0442\u044c \u0434\u0430\u0442\u0430\u0441\u0435\u0442 \u043d\u0435\u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e \u043e\u0442 \u0442\u043e\u0433\u043e, \u0432 \u043a\u0430\u043a\u043e\u043c \u0432\u0438\u0434\u0435 \u043f\u0430\u0440\u044b \u043d\u0430\u0445\u043e\u0434\u044f\u0442\u0441\u044f \u0438\u0437\u043d\u0430\u0447\u0430\u043b\u044c\u043d\u043e. \u0412 \u043b\u044e\u0431\u043e\u043c \u0441\u043b\u0443\u0447\u0430\u0435 \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e \u0441\u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043c\u0430\u0441\u0441\u0438\u0432 texts \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0446\u0438\u043a\u043b\u043e\u0432 \u0438 \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u043f\u0440\u0435\u043e\u0431\u0440\u0430\u0437\u043e\u0432\u0430\u043d\u0438\u0439. <\/p>\n<h2>\u0412\u043e\u043f\u0440\u043e\u0441\u044b, \u043e\u0441\u0442\u0430\u0432\u0448\u0438\u0435\u0441\u044f \u043d\u0435\u044f\u0441\u043d\u044b\u043c\u0438<\/h2>\n<p>\u0412 \u0438\u0441\u0445\u043e\u0434\u043d\u043e\u0439 \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043f\u0440\u0435\u0434\u043b\u043e\u0436\u0435\u043d\u043e 2 \u044d\u043f\u043e\u0445\u0438. <\/p>\n<p>\u0414\u043b\u044f \u0434\u0435\u043c\u043e\u043d\u0441\u0442\u0440\u0430\u0446\u0438\u0438 \u0438 \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0438 \u0440\u0430\u0431\u043e\u0442\u043e\u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u0438 \u044d\u0442\u043e\u0433\u043e \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e, \u043d\u043e \u0434\u043b\u044f \u043f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u043e\u0433\u043e \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0430 \u043d\u0435\u044f\u0441\u043d\u043e, \u0441 \u043a\u0430\u043a\u043e\u0433\u043e \u043c\u043e\u043c\u0435\u043d\u0442\u0430 \u043c\u043e\u0434\u0435\u043b\u044c \u0431\u0443\u0434\u0435\u0442 \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0438\u0442\u044c \u0438\u043c\u0435\u043d\u043d\u043e \u0442\u0430\u043a, \u043a\u0430\u043a \u0437\u0430\u043b\u043e\u0436\u0435\u043d\u043e \u0432 \u0434\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u043c \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u0435, \u0430 \u043d\u0435 \u0442\u0430\u043a, \u043a\u0430\u043a \u043e\u043d\u0430 \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0438\u043b\u0430 \u0440\u0430\u043d\u0435\u0435. \u0412\u0435\u0440\u043e\u044f\u0442\u043d\u043e, \u044d\u0442\u043e \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0438\u0442\u044c \u0442\u043e\u043b\u044c\u043a\u043e \u043d\u0430 \u043a\u043e\u043d\u043a\u0440\u0435\u0442\u043d\u044b\u0445 \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0438 \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u0438 \u0442\u0430\u0431\u043b\u0438\u0446.<\/p>\n<h2>\u041f\u0440\u0438\u043c\u0435\u0447\u0430\u043d\u0438\u044f<\/h2>\n<p>\u0415\u0441\u043b\u0438 \u0412\u044b \u043e\u0431\u043d\u0430\u0440\u0443\u0436\u0438\u043b\u0438 \u0432 \u0441\u0442\u0430\u0442\u044c\u0435 \u043d\u0435\u0442\u043e\u0447\u043d\u043e\u0441\u0442\u044c, \u0438\u043b\u0438 \u0441\u0447\u0438\u0442\u0430\u0435\u0442\u0435 \u043f\u043e\u043b\u0435\u0437\u043d\u044b\u043c \u0447\u0442\u043e-\u043b\u0438\u0431\u043e \u0434\u043e\u0431\u0430\u0432\u0438\u0442\u044c &#8212; \u043f\u043e\u0436\u0430\u043b\u0443\u0439\u0441\u0442\u0430, \u0441\u043e\u043e\u0431\u0449\u0438\u0442\u0435 \u0432 \u043a\u043e\u043c\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u044f\u0445.  <\/p>\n<figure class=\"full-width\"><\/figure>\n<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/762140\/\"> https:\/\/habr.com\/ru\/articles\/762140\/<\/a><br \/><\/br><\/br><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-355330","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/355330","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=355330"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/355330\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=355330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=355330"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=355330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}