{"id":414914,"date":"2024-06-30T00:02:46","date_gmt":"2024-06-30T00:02:46","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=414914"},"modified":"-0001-11-30T00:00:00","modified_gmt":"2024-06-30T00:02:46","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=414914","title":{"rendered":"<span>HDB++ TANGO Archiving System<\/span>"},"content":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-1\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<div style=\"text-align:center;\"><img decoding=\"async\" alt=\"main\" width=\"800\" src=\"https:\/\/habrastorage.org\/r\/w1560\/webt\/tu\/vr\/ec\/tuvrecn2a_yv1mikcfb39r9hlfa.png\" data-src=\"https:\/\/habrastorage.org\/webt\/tu\/vr\/ec\/tuvrecn2a_yv1mikcfb39r9hlfa.png\"\/><\/div>\n<p>  <\/p>\n<h2 id=\"what-is-hdbhttpswwwtango-controlsorgcommunityproject-docshdbplusplus\">What is <a href=\"https:\/\/www.tango-controls.org\/community\/project-docs\/hdbplusplus\/\" rel=\"nofollow noopener noreferrer\">HDB++<\/a>?<\/h2>\n<p>  <\/p>\n<p>This is a TANGO archiving system, allows you to save data received from devices in the TANGO system.<\/p>\n<p>  <\/p>\n<p>Working with Linux will be described here (<a href=\"https:\/\/s2innovation.sharepoint.com\/:f:\/s\/Developers\/EovD2IBwhppAp-ZLXtawQ6gB9F6aXPPs2msr2hgPGTO-FQ?e=Ii3tnr\" rel=\"nofollow noopener noreferrer\"><strong>TangoBox 9.3<\/strong><\/a> on base Ubuntu 18.04), this is a ready-made system where everything is configured.<\/p>\n<p>  <\/p>\n<h2 id=\"what-is-the-article-about\">What is the article about?<\/h2>\n<p>  <\/p>\n<ul>\n<li>System architecture.<\/li>\n<li>How to set up archiving.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>It took me ~ 2 weeks to understand the architecture and write my own scripts for <strong>python<\/strong> for this case.<\/p>\n<p>  <\/p>\n<h2 id=\"what-is-it-for\">What is it for?<\/h2>\n<p>  <\/p>\n<p>Allows you to store the history of the readings of your equipment.<\/p>\n<p>  <\/p>\n<ul>\n<li>You don&#8217;t need to think about how to store data in the database.<\/li>\n<li>You just need to specify which attributes to archive from which equipment.<\/li>\n<\/ul>\n<p><a name=\"habracut\"><\/a>  <\/p>\n<h2 id=\"where-to-get\">Where to get?<\/h2>\n<p>  <\/p>\n<ul>\n<li><a href=\"https:\/\/www.tango-controls.org\/community\/project-docs\/hdbplusplus\/hdbplusplus-downloads\/\" rel=\"nofollow noopener noreferrer\"><strong>deb &amp;&amp; sql<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/github.com\/tango-controls-hdbpp\" rel=\"nofollow noopener noreferrer\"><strong>source code<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.tango-controls.org\/community\/project-docs\/hdbplusplus\/hdbplusplus-doc\/\" rel=\"nofollow noopener noreferrer\"><strong>documentation<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/tango-controls.readthedocs.io\/en\/latest\/installation\/virtualmachine.html\" rel=\"nofollow noopener noreferrer\"><strong>image TangoBox on the Ubuntu base<\/strong><\/a><\/li>\n<\/ul>\n<p>  <\/p>\n<h2 id=\"architecture\">Architecture<\/h2>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w1560\/webt\/si\/9f\/sq\/si9fsq0jjolmzn2t2-ka9n_svnk.png\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/si\/9f\/sq\/si9fsq0jjolmzn2t2-ka9n_svnk.png\"\/><\/p>\n<p>  <\/p>\n<p>The two most important things here are <strong>Archiver<\/strong> and <strong>Archiving DB<\/strong>. <strong>HDB++ Configuration<\/strong> graphical utility for managing Archiver. <strong>HDB ++ Viewer<\/strong> Archiving DB viewer utility.<\/p>\n<p>  <\/p>\n<p><strong>Archiver<\/strong> interrogates our <strong>Device Server<\/strong> and writes history to the <strong>Archiving DB<\/strong>.<br \/>  Archiver is the same Device Server:<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/wx\/bw\/4c\/wxbw4cyo_occntl8jparsj2drbo.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/wx\/bw\/4c\/wxbw4cyo_occntl8jparsj2drbo.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p><strong>Archiving DB<\/strong> in our system is based on <strong> MySQL <\/strong>, it is in docker <strong>tangobox-hdbpp<\/strong>.<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/lx\/zx\/tf\/lxzxtfz04nofarfshjy4jdh3hhy.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/lx\/zx\/tf\/lxzxtfz04nofarfshjy4jdh3hhy.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<h2 id=\"archiver\">Archiver<\/h2>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/bc\/dd\/mw\/bcddmwci_lpsr7exia445op9nze.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/bc\/dd\/mw\/bcddmwci_lpsr7exia445op9nze.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>Let&#8217;s see the list of commands for this server.<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/e7\/qk\/ts\/e7qktshbfap5gkk3dm_de-uqyxg.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/e7\/qk\/ts\/e7qktshbfap5gkk3dm_de-uqyxg.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>The picture shows a list of commands for managing this Device Server.<\/p>\n<p>  <\/p>\n<p>Here we are interested in:<\/p>\n<p>  <\/p>\n<ul>\n<li><strong>AttibuteAdd<\/strong> \u2014 add attribute.<\/li>\n<li><strong>AttibutePause<\/strong> \u2014 pause archiving attribute.<\/li>\n<li><strong>AttibuteRemove<\/strong> \u2014 delete attribute.<\/li>\n<li><strong>AttibuteStart<\/strong> \u2014 start archiving.<\/li>\n<li><strong>AttibuteStatus<\/strong> \u2014 status attribute.<\/li>\n<li><strong>AttibuteStop<\/strong> \u2014 stop archiving.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>Let&#8217;s look through the <strong>Jive<\/strong> list of attributes that Archiver is watching:<\/p>\n<p>  <\/p>\n<pre><code class=\"bash\">jive<\/code><\/pre>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/wx\/bw\/4c\/wxbw4cyo_occntl8jparsj2drbo.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/wx\/bw\/4c\/wxbw4cyo_occntl8jparsj2drbo.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>Double click on the device.<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/gq\/li\/qa\/gqliqanb2roaz98yqhl0b4lxhkk.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/gq\/li\/qa\/gqliqanb2roaz98yqhl0b4lxhkk.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>This Device Server is controlled through the <strong>HDB ++ Configuration<\/strong> utility, this is a graphical utility that sends the above commands to <strong>archiving\/hdbpp\/eventsubscriber.1<\/strong>. Further it will be shown how to do this programmatically.<\/p>\n<p>  <\/p>\n<h2 id=\"hdb-configuration\">HDB++ Configuration<\/h2>\n<p>  <\/p>\n<pre><code class=\"bash\">hdbpp-configurator -configure<\/code><\/pre>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/sv\/nw\/bh\/svnwbh8yxo3l170gmibfly0gdzi.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/sv\/nw\/bh\/svnwbh8yxo3l170gmibfly0gdzi.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>It starts\/stops archiving and sets the archiving parameters. Double click on the attribute:<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/ve\/gq\/qt\/vegqqtoux5y-2cctdn1dugymqzw.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/ve\/gq\/qt\/vegqqtoux5y-2cctdn1dugymqzw.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<ul>\n<li><strong>TTL<\/strong> \u2014 how many days the history will be archived.<\/li>\n<li><strong>absolute change<\/strong> \u2014 changing the attribute in units.<\/li>\n<li><strong>relative change<\/strong> \u2014 attribute change in percentage.<\/li>\n<li><strong>event period<\/strong> \u2014 if the values have changed, then write every ms.<\/li>\n<li><strong>Attibute polling period<\/strong> \u2014 write every ms.<\/li>\n<\/ul>\n<p>  <\/p>\n<h2 id=\"archiving-db\">Archiving DB<\/h2>\n<p>  <\/p>\n<p>In it, we will be interested in the <strong>hdbpp<\/strong> database:<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/5e\/hh\/15\/5ehh15mfnxsnbr_di1sc_oqfhzk.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/5e\/hh\/15\/5ehh15mfnxsnbr_di1sc_oqfhzk.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>You will also need to configure access to the database, from which machine and who is allowed to connect (Because I was too lazy to study how the system was already configured, it is easier to set my user.):<\/p>\n<p>  <\/p>\n<pre><code class=\"sql\">GRANT ALL PRIVILEGES on *.* to 'root'@'172.18.0.1' IDENTIFIED BY 'tango'; FLUSH PRIVILEGES;<\/code><\/pre>\n<p>  <\/p>\n<p>Our main system is at <strong>172.18.0.1<\/strong>, the docker database is at <strong>172.18.0.7<\/strong>.<\/p>\n<p>  <\/p>\n<p>Now let&#8217;s move on to the database structure, here is the main table <strong>att_conf<\/strong>. It contains the attributes that have entered the archiving system:<\/p>\n<p>  <\/p>\n<p><img decoding=\"async\" src=\"https:\/\/habrastorage.org\/r\/w780q1\/webt\/p5\/qb\/jn\/p5qbjnakml0npautm0rhfiz8ixw.jpeg\" alt=\"image\" data-src=\"https:\/\/habrastorage.org\/webt\/p5\/qb\/jn\/p5qbjnakml0npautm0rhfiz8ixw.jpeg\" data-blurred=\"true\"\/><\/p>\n<p>  <\/p>\n<p>The important fields here are <strong>att_conf_id<\/strong> and <strong>att_conf_data_type_id<\/strong>. Using att_conf_data_type_id from the <strong>att_conf_data_type<\/strong> table, we get the data type of the attribute. For example <strong>scalar_devushort_ro<\/strong>, having received the data type, we find out the table in which the history is stored. The name of the table will be <strong>att_scalar_devushort_ro<\/strong>, from this table by att_conf_id we get the data archive of the attribute we are interested in.<\/p>\n<p>  <\/p>\n<h2 id=\"python\">Python<\/h2>\n<p>  <\/p>\n<p>Python mechanisms for working with HDB ++.<\/p>\n<p>  <\/p>\n<p>There is an official python<strong>2.7<\/strong> library for working with HDB++ <a href=\"https:\/\/github.com\/tango-controls\/PyTangoArchiving\" rel=\"nofollow noopener noreferrer\">PyTangoArchiving<\/a>. I managed to deal with it only when I wrote my library. There is not enough documentation on what is passed to the methods, what data types, what to pass in the arguments (<strong>This is my opinion<\/strong>).<\/p>\n<p>  <\/p>\n<p>My <a href=\"https:\/\/github.com\/dvjdjvu\/hdbpp\" rel=\"nofollow noopener noreferrer\"><strong>module<\/strong><\/a> is built for version >= <strong>3.7<\/strong>. Here, all the standard settings for working on <strong>TangoBox 9.3<\/strong> are set by default.<\/p>\n<p>  <\/p>\n<h3 id=\"installing-the-module\">Installing the module<\/h3>\n<p>  <\/p>\n<pre><code class=\"bash\">sudo pip3 install -r requirements.txt sudo python3 setup.py install<\/code><\/pre>\n<p>  <\/p>\n<p>Dependencies:<\/p>\n<p>  <\/p>\n<ul>\n<li><strong>mysql-connector<\/strong>>=2.2.9<\/li>\n<li><strong>pytango<\/strong>>=9.3.2<\/li>\n<li><strong>distribute<\/strong>>=0.7.3<\/li>\n<\/ul>\n<p>  <\/p>\n<h3 id=\"how-to-use\">How to use<\/h3>\n<p>  <\/p>\n<pre><code class=\"python\">from hdbpp import HDBPP  if __name__ == '__main__':     hdbpp = HDBPP()     # semantics:     # __init__(self, host=\"172.18.0.7\", user=\"root\", password=\"tango\", database=\"hdbpp\",      #    archive_server_name=\"archiving\/hdbpp\/eventsubscriber.1\",     #    server_default = \"tango:\/\/tangobox:10000\")      # Connect to the archive server and the database with archives.     if hdbpp.connect() == False :         exit(0)      # Get the history of an attribute for all time.     archive = hdbpp.get_archive('tango:\/\/tangobox:10000\/ECG\/ecg\/1\/Lead')     for a in archive:         print(a)      # The current archive status of the attribute.     ret = hdbpp.archiving_status('tango:\/\/tangobox:10000\/ECG\/ecg\/1\/Lead')     print(ret)      # Add Attribute to Archive Server.     hdbpp.archiving_add(['tango:\/\/tangobox:10000\/ECG\/ecg\/1\/Lead'])      # Start archiving the attribute.     hdbpp.archiving_start('tango:\/\/tangobox:10000\/ECG\/ecg\/1\/Lead', 10 * 60 * 1000, 5 * 60 * 1000, 2, 1)     # \u0433\u0434\u0435:     # 10 * 60 * 1000 - poll and archive the attribute every ms     # 5 * 60 * 1000 - archive the attribute every ms if the value exceeds the threshold     # 2 - attribute change threshold in units     # 1 - attribute change threshold in percent      # Stop archiving an attribute     hdbpp.archiving_stop('tango:\/\/tangobox:10000\/ECG\/ecg\/1\/Lead')      # Close connection     hdbpp.close()<\/code><\/pre>\n<p>  <\/p>\n<p>More detailed documentation in the code.<br \/>  It can be viewed like this:<\/p>\n<p>  <\/p>\n<pre><code class=\"bash\">pydoc3 hdbpp.HDBPP<\/code><\/pre>\n<p>  <\/p>\n<h2 id=\"links\">Links<\/h2>\n<p>  <\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/dvjdjvu\/hdbpp\" rel=\"nofollow noopener noreferrer\">Source code<\/a><\/li>\n<\/ul>\n<p>  <\/p>\n<p>Thank you for attention.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!----><!----><\/div>\n<p><!----><!----><br \/> \u0441\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b \u0441\u0442\u0430\u0442\u044c\u0438 <a href=\"https:\/\/habr.com\/ru\/articles\/544536\/\"> https:\/\/habr.com\/ru\/articles\/544536\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<div><!--[--><!--]--><\/div>\n<div id=\"post-content-body\">\n<div>\n<div class=\"article-formatted-body article-formatted-body article-formatted-body_version-1\">\n<div xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<div style=\"text-align:center;\"><img decoding=\"async\" alt=\"main\" width=\"800\" src=\"https:\/\/habrastorage.org\/r\/w1560\/webt\/tu\/vr\/ec\/tuvrecn2a_yv1mikcfb39r9hlfa.png\" data-src=\"https:\/\/habrastorage.org\/webt\/tu\/vr\/ec\/tuvrecn2a_yv1mikcfb39r9hlfa.png\"\/><\/div>\n<p>  <\/p>\n<h2 id=\"what-is-hdbhttpswwwtango-controlsorgcommunityproject-docshdbplusplus\">What is <a href=\"https:\/\/www.tango-controls.org\/community\/project-docs\/hdbplusplus\/\" rel=\"nofollow noopener noreferrer\">HDB++<\/a>?<\/h2>\n<p>  <\/p>\n<p>This is a TANGO archiving system, allows you to save data received from devices in the TANGO system.<\/p>\n<p>  <\/p>\n<p>Working with Linux will be described here (<a href=\"https:\/\/s2innovation.sharepoint.com\/:f:\/s\/Developers\/EovD2IBwhppAp-ZLXtawQ6gB9F6aXPPs2msr2hgPGTO-FQ?e=Ii3tnr\" rel=\"nofollow noopener noreferrer\"><strong>TangoBox 9.3<\/strong><\/a> on base Ubuntu 18.04), this is a ready-made system where everything is configured.<\/p>\n<p>  <\/p>\n<h2 id=\"what-is-the-article-about\">What is the article about?<\/h2>\n<p>  <\/p>\n<ul>\n<li>System architecture.<\/li>\n<li>How to set up archiving.<\/li>\n<\/ul>\n<p>  <\/p>\n<p>It took me ~ 2 weeks to understand the architecture and write my own scripts for <strong>python<\/strong> for this case.<\/p>\n<p>  <\/p>\n<h2 id=\"what-is-it-for\">What is it for?<\/h2>\n<p>  <\/p>\n<p>Allows you to store the history of the readings of your equipment.<\/p>\n<p>  <\/p>\n<ul>\n<li>You don&#8217;t need to think about how to store data in the database.<\/li>\n<li>You just need to specify which attributes to archive from which equipment.<\/li>\n<\/ul>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-414914","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/414914","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=414914"}],"version-history":[{"count":0,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=\/wp\/v2\/posts\/414914\/revisions"}],"wp:attachment":[{"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=414914"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=414914"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/savepearlharbor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=414914"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}