• 欢迎访问蜷缩的蜗牛博客 蜷缩的蜗牛
  • 微信搜索: 蜷缩的蜗牛 | 联系站长 kbsonlong@qq.com
  • 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧

【转载】HAproxy增加日志记录功能和自定义日志输出内容、格式

Linux 蜷缩的蜗牛 1年前 (2017-07-28) 178次浏览 已收录 0个评论

一.增加 haproxy 日志记录功能
1.1 由于数据分析的需要,我们必须打开 haproxy 日志,记录相关信息。
  在配置前,我们先来了解一下日志的 level:local0~local7 16~23 保留为本地使用

emerg 0 系统不可用
     alert 1 必须马上采取行动的事件
     crit 2 关键的事件
     err 3 错误事件
     warning 4 警告事件
     notice 5 普通但重要的事件
     info 6 有用的信息
     debug 7 调试信息

1.2 实际配置:
########################################

vi haproxy.conf(在 default 处添加如下信息)
defaults
    log global
    option httplog
    log 127.0.0.1 local3

########################################

#######################################

vi /etc/rsyslog.conf(添加如下内容)
local3.* /data/logs/haproxy.log

#######################################

#######################################

vi /etc/sysconfig/rsyslog

把 SYSLOGD_OPTIONS="-m 0"
改成 SYSLOGD_OPTIONS="-r -m 0 -c 2"

#######################################

相关解释说明:
-r:打开接受外来日志消息的功能,其监控 514 UDP 端口;
-x:关闭自动解析对方日志服务器的 FQDN 信息,这能避免 DNS 不完整所带来的麻烦;
-m:修改 syslog 的内部 mark 消息写入间隔时间(0 为关闭),例如 240 为每隔 240 分钟写入一次”–MARK–“信息;
-h:默认情况下,syslog 不会发送从远端接受过来的消息到其他主机,而使用该选项,则把该开关打开,所有
[email protected]�过去
配置完毕后重启 rsyslogd 和 haproxy 即可.
日志内容如下:

Sep 18 15:48:11 localhost haproxy[12124]: Proxy html-server started.
Sep 18 15:48:11 localhost haproxy[12124]: Proxy html-server started.
Sep 18 15:48:29 localhost haproxy[12125]: 10.69.112.139:59244 [18/Sep/2016:15:48:29.027] http-in html-server/html-A 0/0/0/0/0 200 254 - - --NI 0/0/0/0/0 0/0 "GET / HTTP/1.1"
Sep 18 15:48:29 localhost haproxy[12125]: 10.69.112.139:59244 [18/Sep/2016:15:48:29.027] http-in html-server/html-A 0/0/0/0/0 200 254 - - --NI 0/0/0/0/0 0/0 "GET / HTTP/1.1"
Sep 18 15:48:29 localhost haproxy[12125]: 10.69.112.139:59244 [18/Sep/2016:15:48:29.027] http-in html-server/html-A 0/0/0/0/0 200 254 - - --NI 0/0/0/0/0 0/0 "GET / HTTP/1.1"

但日志可读性很差,我们需要显示更多的信息,但默认的都捕获不到,只能自定义需要显示的内容了。
二.定义 haproxy 日志输出内容和格式
2.1 自定义 haproxy 日志。直接上配置文件。

global
    log 127.0.0.1 local3 info
    maxconn 4096
    chroot /gomeo2o/haproxy
    uid haproxy
    gid haproxy
    daemon
    nbproc 1
    description haproxy
    pidfile /var/run/haproxy.pid
defaults
    log global
#    option httplog
    balance roundrobin
    mode http
    maxconn 4096
    retries 3
    option redispatch
    option forceclose
    option dontlognull
    option abortonclose
    stats uri /haproxy
frontend http-in
    bind 0.0.0.0:80
    mode http
    log global
    option httplog
    option httpclose
    acl html url_reg -i \.html$
    use_backend html-server if html
    default_backend html-server
#((   
    capture request header Host len 64
    capture request header User-Agent len 128
    capture request header X-Forwarded-For len 100
    capture request header Referer len 200
    capture response header Server len 40
    capture response header Server-ID len 40
    #capture 捕获信息
    log-format %ci:%cp\ %si:%sp\ %B\ %U\ %ST\ %r\ %b\ %f\ %bi\ %hrl\ %hsl\
        #log-forcat 定义日志需显示内容(变量)
        #利用 capture 捕获信息,log-forcat 定义变量
        #详解请见 2.1 和 2.2
#))
backend html-server
    mode http
    balance roundrobin
    option httpchk GET /index.html
    cookie SERVERID insert indirect nocache
    server html-A 10.69.112.150:80 weight 1 cookie 3 check inter 2000 rise 2 fall 5
    server html-B 10.69.112.148:80 weight 1 cookie 4 check inter 2000 rise 2 fall 5

其他参数在此不做过多解释,主要查看两个中括号之间的内容。
2.2 capture 捕获
haproxy 利用 capture 捕获请求(request)和响应(response)信息,具体需捕获内容可以做如下操作:
用浏览器访问一个网站(我用的 google),比如 www.gomeplus.com,按 F12,内容如下(注意我红色框起来的部分):
【转载】HAproxy 增加日志记录功能和自定义日志输出内容、格式
找到 response headers 项,这里就是你可以捕获的信息。比如捕获 server 和 server-id:

capture response header Server len 40
capture response header Server-ID len 40

备注:2.1 配置文件当中的 capture 基本的信息都能捕获到了,如有特别需求,请查阅官网(capture 到底能捕获多少信息,多详细,我也在查询当中)。
2.3 定义 log-format 变量
定义示例:

log-format %T\ %t\ Some\ Text
log-format %{+Q}o\ %t\ %s\ %{-Q}r

请参阅下面的表为定义的变量(转自官方文档):

+---+------+-----------------------------------------------+-------------+
  | R | var  | field name (8.2.2 and 8.2.3 for description)  | type        |
  +---+------+-----------------------------------------------+-------------+
  |   | %o   | special variable, apply flags on all next var |             |
  +---+------+-----------------------------------------------+-------------+
  |   | %B   | bytes_read           (from server to client)  | numeric     |
  | H | %CC  | captured_request_cookie                       | string      |
  | H | %CS  | captured_response_cookie                      | string      |
  |   | %H   | hostname                                      | string      |
  | H | %HM  | HTTP method (ex: POST)                        | string      |
  | H | %HP  | HTTP request URI without query string (path)  | string      |
  | H | %HQ  | HTTP request URI query string (ex: ?bar=baz)  | string      |
  | H | %HU  | HTTP request URI (ex: /foo?bar=baz)           | string      |
  | H | %HV  | HTTP version (ex: HTTP/1.0)                   | string      |
  |   | %ID  | unique-id                                     | string      |
  |   | %ST  | status_code                                   | numeric     |
  |   | %T   | gmt_date_time                                 | date        |
  |   | %Tc  | Tc                                            | numeric     |
  |   | %Td  | Td = Tt - (Tq + Tw + Tc + Tr)                 | numeric     |
  |   | %Tl  | local_date_time                               | date        |
  | H | %Tq  | Tq                                            | numeric     |
  | H | %Tr  | Tr                                            | numeric     |
  |   | %Ts  | timestamp                                     | numeric     |
  |   | %Tt  | Tt                                            | numeric     |
  |   | %Tw  | Tw                                            | numeric     |
  |   | %U   | bytes_uploaded       (from client to server)  | numeric     |
  |   | %ac  | actconn                                       | numeric     |
  |   | %b   | backend_name                                  | string      |
  |   | %bc  | beconn      (backend concurrent connections)  | numeric     |
  |   | %bi  | backend_source_ip       (connecting address)  | IP          |
  |   | %bp  | backend_source_port     (connecting address)  | numeric     |
  |   | %bq  | backend_queue                                 | numeric     |
  |   | %ci  | client_ip                 (accepted address)  | IP          |
  |   | %cp  | client_port               (accepted address)  | numeric     |
  |   | %f   | frontend_name                                 | string      |
  |   | %fc  | feconn     (frontend concurrent connections)  | numeric     |
  |   | %fi  | frontend_ip              (accepting address)  | IP          |
  |   | %fp  | frontend_port            (accepting address)  | numeric     |
  |   | %ft  | frontend_name_transport ('~' suffix for SSL)  | string      |
  |   | %lc  | frontend_log_counter                          | numeric     |
  |   | %hr  | captured_request_headers default style        | string      |
  |   | %hrl | captured_request_headers CLF style            | string list |
  |   | %hs  | captured_response_headers default style       | string      |
  |   | %hsl | captured_response_headers CLF style           | string list |
  |   | %ms  | accept date milliseconds (left-padded with 0) | numeric     |
  |   | %pid | PID                                           | numeric     |
  | H | %r   | http_request                                  | string      |
  |   | %rc  | retries                                       | numeric     |
  |   | %rt  | request_counter (HTTP req or TCP session)     | numeric     |
  |   | %s   | server_name                                   | string      |
  |   | %sc  | srv_conn     (server concurrent connections)  | numeric     |
  |   | %si  | server_IP                   (target address)  | IP          |
  |   | %sp  | server_port                 (target address)  | numeric     |
  |   | %sq  | srv_queue                                     | numeric     |
  | S | %sslc| ssl_ciphers (ex: AES-SHA)                     | string      |
  | S | %sslv| ssl_version (ex: TLSv1)                       | string      |
  |   | %t   | date_time      (with millisecond resolution)  | date        |
  |   | %ts  | termination_state                             | string      |
  | H | %tsc | termination_state with cookie status          | string      |
  +---+------+-----------------------------------------------+-------------+

最后查看定义之后的日志内容输出:

Sep 20 17:06:12 localhost haproxy[16372]: 10.69.112.139:39957 10.69.112.148:80 2614 179 200 GET / HTTP/1.1 html-server http-in 10.69.112.151www.qiang999.com curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 - - nginx/1.9.8 -\
Sep 20 17:06:12 localhost haproxy[16372]: 10.69.112.139:39957 10.69.112.148:80 2614 179 200 GET / HTTP/1.1 html-server http-in 10.69.112.151www.qiang999.com curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 - - nginx/1.9.8 -\
Sep 20 17:06:12 localhost haproxy[16372]: 10.69.112.139:39966 10.69.112.150:80 254 179 200 GET / HTTP/1.1 html-server http-in 10.69.112.151www.qiang999.com curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 - - nginx/1.9.8 -\
Sep 20 17:06:12 localhost haproxy[16372]: 10.69.112.139:39966 10.69.112.150:80 254 179 200 GET / HTTP/1.1 html-server http-in 10.69.112.151www.qiang999.com curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 - - nginx/1.9.8 -\
Sep 20 17:06:13 localhost haproxy[16372]: 10.69.112.139:39974 10.69.112.148:80 2614 179 200 GET / HTTP/1.1 html-server http-in 10.69.112.151www.qiang999.com curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 - - nginx/1.9.8 -\

蜷缩的蜗牛 , 版权所有丨如未注明 , 均为原创丨 转载请注明【转载】HAproxy 增加日志记录功能和自定义日志输出内容、格式
喜欢 (0)
[]
分享 (0)

您必须 登录 才能发表评论!