==============
Java 1.8+Elasticsearch 2.0+Kibana 4.2.0(失敗)
後降版至Kibana 3 成功
之後再補上筆記
==============
參考網址：
http://www.chenshake.com/centos-install-7-x-elk-elasticsearchlogstashkibana/
https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html
https://www.digitalocean.com/community/tutorial_series/centralized-logging-with-logstash-and-kibana-on-centos-7
http://zettadata.blogspot.tw/2014/09/elkelastic-searchlogstashkibanalog.html
http://icodding.blogspot.tw/2015/09/centos-elkelastic-searchlogstashkibana.html?showComment=1446104855572
看了許多文章和測試後目前自己設計的實驗架構如下：
Java 1.8
sudo yum install java-1.8.0-openjdk httpd unzip
Elasticsearch 2.0
安裝
sudo vi /etc/yum.repos.d/elasticsearch.repo
Java 1.8
yum install java-1.8.0-openjdk httpd unzip
Elastic Search
Logstash
Kibana
https://download.elastic.co/kibana/kibana/kibana-4.2.0-linux-x64.tar.gz
安裝Elastic Search
下載公鑰
rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch
在 /etc/yum.repos.d/ 目錄下新增檔案，例如 elasticsearch.repo，內容如下
vi /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-2.0]
name=Elasticsearch repository for 2.0.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
安裝Elastersearch
yum install elasticsearch
設定Elasticsearch
編輯 /etc/elasticsearch/elasticsearch.yml
sudo vi /etc/elasticsearch/elasticsearch.yml
cluster.name: "LogCluster"
node.name: "LogMaster"
node.master: true
node.data: true
path.conf: /etc/elasticsearch
path.data: /datapool/data1
path.work: /datapool/work
path.logs: /datapool/log
mkdir /datapool
mkdir /datapool/data1
mkdir /datapool/work
mkdir /datapool/log
將服務加入系統
sudo chkconfig --add elasticsearch
啟動服務
sudo /etc/init.d/elasticsearch start
測試啟動是否成功
curl localhost:9200/_nodes/process?pretty
防火牆開啟
sudo firewall-cmd --permanent --zone=public --add-port=80/tcp
sudo firewall-cmd --permanent --zone=public --add-port=5403/tcp
sudo firewall-cmd --permanent --zone=public --add-port=5601/tcp
sudo firewall-cmd --permanent --zone=public --add-port=9200/tcp
sudo firewall-cmd --permanent --zone=public --add-port=9300/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-all
安裝Logstash
安裝金鑰
rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch
在 /etc/yum.repos.d/ 目錄下新增檔案，例如 logstash.repo，內容如下
sudo vi /etc/yum.repos.d/logstash.repo
[logstash-2.0]
name=logstash repository for 2.0.x packages
baseurl=http://packages.elasticsearch.org/logstash/2.0/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
安裝Logstash
sudo yum install logstash
配置logstash，如下是Logstash的配置文件
==
sudo vi /etc/logstash/conf.d/logstasg_access.conf
==
sudo vi /etc/logstash/conf.d/01_apache_logstasg_input.conf
input {
file {
path => "/var/log/httpd/access_log"
type => "apache" # a type to identify those logs (will need this later)
}
}
filter {
if [type] == "apache" { # this is where we use the type from the input section
grok {
match => [ "message", "%{COMBINEDAPACHELOG}" ]
}
date {
# Try to pull the timestamp from the 'timestamp' field (parsed above with
# grok). The apache time format looks like: "18/Aug/2011:05:44:34 -0700"
match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
}
if [user-agent] != "-" and [user-agent] != "" {
useragent {
add_tag => [ "UA" ]
source => "user-agent"
}
}
if "UA" in [tags] {
if [device] == "Other" { mutate { remove_field => "device" } }
if [name] == "Other" { mutate { remove_field => "name" } }
if [os] == "Other" { mutate { remove_field => "os" } }
}
}
}
sudo vi /etc/logstash/conf.d/02_apache_logstasg_output.conf
output {
elasticsearch {
host => "lsap.adtc.com.tw"
cluster => "elasticsearch"
node_name => "Elektra Natchios"
}
}
啟動 logstash 服務
cd /opt/logstash/bin/
./logstash
安裝Kibana 4.2.0(結構改變 需再研究)
sudo wget https://download.elastic.co/kibana/kibana/kibana-4.2.0-linux-x64.tar.gz && gunzip kibana-4.2.0-linux-x64.tar.gz && tar -xvf kibana-4.2.0-linux-x64.tar && mv kibana-4.2.0-linux-x64 kibana && sudo mv kibana /var/www/html/
安裝Kibana 3.1.2
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.zip && unzip kibana-3.1.2.zip && mv kibana-3.1.2 kibana && mv kibana /var/www/html/
設定apache
vi /etc/httpd/conf/httpd.conf
<VirtualHost *:80>
ServerAdmin edwin.chen@adtc.com.tw
DocumentRoot /var/www/html/kibana
ServerName lsap.adtc.com.tw
ErrorLog logs/kibana.opsnotes.net-error_log
CustomLog logs/kibana.opsnotes.net-access_log common
</VirtualHost>
設定防火牆
#
# 開啟HTTP 80 port
#
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
#
# 設定ElasticSearch使用的port 9200-9300
#
iptables -A INPUT -p tcp --dport 9200:9300 -j ACCEPT
重啟 apache 服務
service httpd restart
sudo vim /etc/init.d/logstash
LS_USER=root
LS_GROUP=root
※為讀取LOG需改變權限為root
(應該可以透過SU更改)
==================
20161216 5.Xtest
#===JAVA=====
yum -y install net-tools
yum -y install java-1.8.0-openjdk.x86_64
yum -y install wget vim
echo $JAVA_HOME
/usr/sbin/alternatives --config java
java -version
vim /etc/profile
find /usr/lib/jvm/java-1.x.x-openjdk
export JAVA_HOME="path that you found"
export PATH=$JAVA_HOME/bin:$PATH
vim /root/.bash_profile
echo $JAVA_HOME
#===JAVA=====
#===elasticsearch
vim /etc/hosts
nmtui
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/2.4.3/elasticsearch-2.4.3.rpm
rpm -ivh elasticsearch-2.1.1.rpm
vim /etc/elasticsearch/elasticsearch.yml
vim /etc/sysconfig/elasticsearch
vim /usr/lib/systemd/system/elasticsearch.service
vim /etc/sysconfig/elasticsearch
firewall-cmd --permanent --add-port={9200/tcp,9300/tcp}
firewall-cmd --reload
sudo systemctl start elasticsearch.service
sudo systemctl status elasticsearch.service
cd /usr/share/elasticsearch/bin
./plugin install lmenezes/elasticsearch-kopf
hostname -F /etc/hostname
#kibana-5.x
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
echo "[kibana-5.x]
name=Kibana repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md" | tee /etc/yum.repos.d/kibana.repo
yum install -y kibana
firewall-cmd --permanent --add-port=5601/tcp
firewall-cmd --reload
systemctl daemon-reload
systemctl enable kibana.service
systemctl start kibana.service
vim config/kibana.yml
elasticsearch.url
