Организация регулярного дайджеста логов с помощью python и ansible на примере asterisk

Когда я создавал данный инструмент, я не был знаком с logwatch. Мне захотелось видеть ситуацию с логами на своих серверах в целом, и, так я сделал этот велосипед. Думаю, что данный механизм может помочь новичкам в понимании альтернативных возможностей ansible.

Используемые программные продукты:

• python 2.7.14
• ansible 2.3
• сервера asterisk на базе FreePBX 13

Механизм состоит из двух частей – python скрипт, который занимается обработкой лог файла и отправкой отчета на почту, и плейбука для сбора логов с серверов и передачи их скрипту для обработки.

Сам плейбук:

--- - name: parseastlogs   hosts: production_asterisks   vars:     date: "{{ lookup('pipe', 'date +%Y%m%d') }}"     ipaddr: "{{ ansible_default_ipv4.address }}"   tasks:    - debug: var=date    - debug: var=ipaddr    - fetch:	        src: /var/log/asterisk/full-{{ date }}       dest: /tmp/full-{{ date }}-{{ ipaddr }}       flat: yes     - local_action: "shell /etc/ansible/localscripts/astReporter.py {{ ipaddr }} full-{{ date }}-{{ ipaddr }}"  В плейбуке задаем переменные, ответственные за имя файла и забираем лог файлы с сервера модулем fetch. Так как у нас будут уникальные файлы - используем параметр flat, чтобы избежать длинных путей к лог файлам.  

Скрипт:

#!/usr/bin/python2.7 import re from collections import Counter import yagmail import sys import datetime import os  servername=sys.argv[1] yag=yagmail.SMTP(user='ansible@2.1',password=None,host='192.168.2.1',port='25',smtp_starttls=False,smtp_set_debuglevel=0,smtp_skip_login=True) recipients=['user@mydomain.local']   filename=sys.argv[2] workdir='/tmp/' log_file=open(workdir+filename,'r')#sys.argv[1] loglist=list() verbosity=['NOTICE','ERROR','WARNING'] regexes = ["Call from '.*' .* to extension '.*' rejected because extension not found in context '.*'.", "Identifier \d+, identifier_type \d+ not found in identifier list", "Invalid result identifier \d+ passed in aMYSQL_clear", "This function can only be used on SIP channels.", "fwrite() returned error: Broken pipe", "CDR requires a value \(CDR\(variable\)=value\)", "Received SIP subscribe for peer without mailbox: .*", "Removed interface '.*' from queue '.*'", "Peer '.*' is trying to register, but not configured as host=dynamic", "Registration from '.*' failed for '.*' - Peer is not supposed to register", "Unable to join queue '.*'", "Attempt to pause interface Local/@from-queue/n, not found", "PRESENCE_STATE unknown", "EXTENSION_STATE requires an extension", "Prodding channel '.*' failed", "Channel '.*' sent to invalid extension but no invalid handler: context,exten,priority=.*", "Can't send 10 type frames with PJSIP", "Attempt to pause interface .+, not found", "Attempt to unpause interface .+, not found", "no samples for ulawtolin", "Could not find matching INVITE transaction for CANCEL request", "Peer '.*' is now Reachable. \(.*\)", "Peer '.*' is now UNREACHABLE!  Last qualify: .*", "Registration from .* failed for '.*' - Wrong password", "Retransmission timeout reached on transmission .*", "no samples for alawtolin", "Peer '.*' is now Lagged. \(\d+ms / \d+ms\)", "Call completed to .*", "Invalid retrytime at line \d+ of .*", "Not accepting call completion offers from call-forward recipient .*", "No such context '.*' for macro '.*'\. Was called by .*", "[pP]ickup .* attempt by .*", "Call failed to go through, reason \(5\) Remote end is Busy", "Deprecated syntax found\..* ", "No digits dialed for atxfer.", "Unable to create channel of type 'SIP' \(cause \d+ - Subscriber absent\)", "'tls' is not a valid transport type when tlsenable=no\. If no other is specified, the defaults from general will be used\.", "'tcp' is not a valid transport type when tcpenable=no\. If no other is specified, the defaults from general will be used\.", "Queued call to .* expired without completion after \d+ attempts", "Re-invite to non-existing call leg on other UA. SIP dialog .*\. Giving up.", "Channel .* not found!  Variable 'BLKVM' not set to .*\.", "Remote host can't match request CANCEL to call .*\. Giving up\.", "Unable to execute query \[.*\]", "SQL Exec Direct failed \(-1\)!\[", "SQL Execute returned an error .*", "   -- Re-registration for .*", "Outbound Registration\: Expiry for .* is \d+ sec \(Scheduling reregistration in \d+ s\)", "Correct auth, but based on stale nonce received from '.*'", "Unable to write frametype: 2", "Received response: \"Forbidden\" from '\".*\" .*'", "Huh\?  Not an RDNIS SIP header .*", "Hanging up call .* - no reply to our critical packet .*", "Cancelling retransmit of OPTIONs \(call id .*\)  ", "Still have a QUALIFY dialog active, deleting", "The use of '_\.' for an extension is strongly discouraged and can have unexpected behavior.  Please use '_X\.' instead at line .*", "Context '.*' tries to include nonexistent context '(.*)'", "aMYSQL_query: mysql_query failed\. Error: Duplicate entry .*", "RTCP SR transmission error to .*, rtcp halted Operation not permitted", "Failed to write frame to '.*': Resource temporarily unavailable", "Unable to forward frametype: 2", "Timeout on .* on non-critical invite transaction.", "Unexpected control subclass '\d+'", "Context '.*' for macro '.*' lacks .*", "No response received from '.*' on registration attempt to '.*', retrying in '\d+'", "Unknown RTP codec 90 received from '.*'", "Invalid extension '.*', but no rule 'i' or 'e' in context '.*'", "Added interface '.*' to queue '.*'", "Exceptionally long voice queue length queuing to .*", "Request from '.*' failed for '.*' \(callid: .*\) - No matching endpoint found", "Junk at the beginning of frame \d+", "Unable to register extension at line .*", "Unable to register extension '.*' priority .* in '.*', already in use", "Unable to load config file .*", 'Purely numeric hostname \(\d+\), and not a peer--rejecting!', "Unknown directive .* at line \d+ .*", "Context '.*' already included in '.*' context on include at line \d+ of .*", "No closing parenthesis found\? '.*' at line \d+ of .*", "Can't use '.*' priority on the first entry at line \d+ of .*", "The .* options are deprecated. Please see UPGRADE.txt for information", "Can't use '.*' priority on the first entry at line \d+ of .*", "Call failed to go through, reason .*", "Unable to open .*", "Playback of message .* failed", "File .* does not exist in any format", "Playback failed on .* for .*", "Adding .* to .*", "Can't move channel. One or both is dead .*", "Unable to complete call pickup of .*", "Pickup .* failed by .*", "No entry in voicemail config file for .*", "   \-\- Registration for '.*' timed out, trying again \(Attempt #[0-9]+\)", "Disconnecting call .* for lack of RTP activity in [0-9]+ seconds", "Failed to initialize .* declining image stream", "Can't send 10 type frames with SIP write", "Set requires an '=' to be a valid assignment.", "Timeout but no rule 't' or 'e' in context .*", "RTCP RR transmission error to .*, rtcp halted Operation not permitted", "Unable to acquire target extension for attended transfer.", "Unterminated comment detected beginning on line"] notmatched=list() matchedregex=list()  combinedre="("+")|(".join(regexes) + ")"  # данный класс сделан для удобной работы со строками из лог файла. Функционал несколько избыточен и не все поля используются. class logitem:     ldatetime=None     ltype=None     lchan=None     lsource=None     lcontent=None     def __init__(self,ldate,ltype,lch,lsrc,lcont):         self.ldatetime=ldate         self.ltype=ltype         self.lchan=lch         self.lsource=lsrc         self.lcontent=lcont   # данная функция осуществляет работу с лог файлом def main():      i=0     dumpchanstart=False      for line in log_file: #игнорируем сообщения отладки Dumpchan     	if 'app_dumpchan.c' in line:             dumpchanstart = 0             for line in log_file:                 if dumpchanstart==2:                     break                  if "================================================================================" in line:                     dumpchanstart+=1 #обрабатываем строки со стандартными сообщениями, которые начинаются с [ 	if line[0]=="[":           buf=re.split('] ', line) # обрабатываем строки с заданной verbosity           if any(x in buf[1] for x in verbosity):                 buf1spl=buf[1].split('-')                 channum=''                 buf2spl=re.split('c: ',buf[2])                 if len(buf1spl)>1:                     channum=buf1spl[1]                 itemcheck=logitem(buf[0][1:],buf[1].split('[')[0],channum,buf2spl[0],buf2spl[1])                 filtration(itemcheck.lcontent)  #функция распределяет получившийся объект в список matchedregex, если он попал под шаблон и в notmatched, если нет.	 def filtration(logitem):      found = False     for regexitem in regexes:          if  re.match(regexitem,logitem):             found= True             matchedregex.append(regexitem)             continue      if found==False:         notmatched.append(logitem)   #функция генерации основного тела письма. В блоке if..elif производится цветовое выделение важных сообщений def genContents():     content=u"<html>"     content+=u"<head></head>"     content+=u"<body>"     content+=u"<h1>Log digest for " +servername+ u"</h1>"     content+=u"<table  style='width: 60%;' border = \"1\" cellpadding = \"1\"'>"     for key, val in Counter(matchedregex).most_common():         if u"rejected because extension not found in context" in key:             key=u"<b style='background-color:#e8edff'>"+key+u"</b>"         elif u"No matching endpoint found" in key:             key=u"<b style='background-color:#fd6161'>"+key+u"</b>"         elif u"No closing parenthesis " in key:             key = u"<b style='background-color:#98862a'>" + key + u"</b>"         elif u"Wrong password" in key:             key=u"<b style='background-color:#fd6161'>"+key+u"</b>"             elif u"Unterminated comment detected beginning on line" in key:             key=u"<b style='background-color:#fd6161'>"+key+u"</b>"             content += u"<tr>"         content+= u"<td>"+key+u"</td><td>" +str(val) +u"</td>"         content += u"</tr>"     content+=u"</table><br>"     content+=u"<h1>not matched logitems:</h1><br>"     for item in notmatched:         content+=item     content+=u"</body>\n"     content+=u"</html>\n"     return  content  main() cont=genContents() yag.send(recipients,subject='Log digest for '+servername,contents=cont) os.remove(workdiir+filename) 

запуск скрипта осуществляется командой: /usr/local/bin/ansible-playbook /etc/ansible/playbooks/parseastlogs.yml

Результатом выполнения команды будет некоторое количество писем(по одному письму на каждый сервер из группы production_asterisk) с примерно следующим содержанием:

Если есть вопросы или предложения, готов на них ответить.