安装Graylog

前置条件

  • Java (>= 8)
  • Elasticsearch (>= 5.x)
  • MongoDB (>= 3.6)

JAVA安装省略

MongoDB

直接通过yum安装,添加yum源

vim /etc/yum.repos.d/mongodb-org-4.0.repo

[mongodb-org-4.0]
name=MongoDB Repository
#baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.0/x86_64/
baseurl=https://mirrors.aliyun.com/mongodb/yum/redhat/7Server/mongodb-org/4.0/x86_64/
gpgcheck=0
enabled=1
#gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc

安装最新版MongoDB yum install -y mongodb-org

修改配置文件:/etc/mongod.conf

主要是修改data目录,防止文件过大

systemLog:
destination: file
logAppend: true
path: /data/log/mongodb/mongod.log

# Where and how to store data.
storage:
#dbPath: /var/lib/mongo
dbPath: /data/graylog_data/mongo
journal:
 enabled: true

启动:systemctl start mongod

安装Elasticsearch

因为网络问题,放弃yum安装,直接下载rpm包

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.0.rpm
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.0.rpm.sha512
shasum -a 512 -c elasticsearch-6.7.0.rpm.sha512 
sudo rpm --install elasticsearch-6.7.0.rpm

同样修改data目录

配置文件:/etc/elasticsearch/elasticsearch.yml

path.data: /data/graylog_data/elasticsearch
 #
 # Path to log files:
 #
 path.logs: /data/log/elasticsearch

配置文件:/etc/elasticsearch/jvm.options

# has sufficient space
 -XX:HeapDumpPath=/data/graylog_data/elasticsearch

 # specify an alternative path for JVM fatal error logs
 -XX:ErrorFile=/data/log/elasticsearch/hs_err_pid%p.log

  ## JDK 8 GC logging

 8:-XX:+PrintGCDetails
 8:-XX:+PrintGCDateStamps
 8:-XX:+PrintTenuringDistribution
 8:-XX:+PrintGCApplicationStoppedTime
 8:-Xloggc:/data/log/elasticsearch/gc.log
 8:-XX:+UseGCLogFileRotation
 8:-XX:NumberOfGCLogFiles=32
 8:-XX:GCLogFileSize=64m

 # JDK 9+ GC logging
 9-:-Xlog:gc*,gc+age=trace,safepoint:file=/data/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m

启动:systemctl start elasticsearch.service

安装graplog服务端

下载rpm包

rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.0-repository_latest.rpm
yum install graylog-server
  • 创建密码

    [root@fabu ~]# echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
    Enter Password: xxxxxxx
    2b6767c88dcd3ac0ce94600e6817e78928c962ffbfa07fcff5d2073b7b65c7f2
    [root@fabu server]# pwgen -N 1 -s 96  UYUOdMverfRKgDTgPgLVMMLFWKwILeZVdshOKrHPUrDZUCeKmAf7r5IMliYk2wxSMg6hLKNxf5ezu20IiwAAzfSbNCxB3F11
    
  • 修改配置

    vim /etc/graylog/server/server.conf

    http_bind_address = 0.0.0.0:9000
      http_external_uri = http://log.xxxxxx.com/
      password_secret = 
      UYUOdMverfRKgDTgPgLVMMLFWKwILeZVdshOKrHPUrDZUCeKmAf7r5IMliYk2wxSMg6hLKNxf5ezu20IiwAAzfSbNCxB3F11
      root_password_sha2 =                                      2b6767c88dcd3ac0ce94600e6817e78928c962ffbfa07fcff5d2073b7b65c7f2
      root_timezone = Asia/Shanghai
  • 启动服务 systemctl start graylog-server

nginx配置

cat /opt/nginx/conf/conf.d/graylog.conf

server {
    listen       80;
    server_name log.xxxxx.com;

    access_log  logs/graylog.access.log;
    error_log  logs/graylog.error.log;

    location / {
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Graylog-Server-URL http://$server_name/;
        proxy_pass       http://127.0.0.1:9000;
   }
}

收集linux系统服务日志

客户端主动发送日志到服务器,有多种方式,这里介绍sidecar+filebeat 具体收集哪些日志的规则,由服务端下发和更新。

安装filebeat

下载地址:https://www.elastic.co/cn/downloads/beats/filebeat

安装:rpm -i filebeat-6.7.0-x86_64.rpm

安装graylog-sidecar

下载地址:https://github.com/Graylog2/collector-sidecar/releases

安装:rpm -i graylog-sidecar-1.0.0-1.x86_64.rpm

修改配置文件 vim /etc/graylog/sidecar/sidecar.yml

server_url: "http://192.168.180.144:9000/api/"
server_api_token: "186j21g67d9jjdibn78lsjaopalates92ir9rthovftumioleoi9"

注意:这里的server_api_token需要服务端的web页面生成。

启动服务

graylog-sidecar -service install
systemctl start graylog-sidecar
systemctl enable graylog-sidecar