Read the first item in this Table of Contents if you haven't been here before.
Table of Contents
- The ELKBeats Stack: Sounds Like a Good Idea ...
- The ELKBeats Stack: the Ground Work
- The ELKBeats Stack: L is for Logstash
- The ELKBeats Stack: E is for Elasticsearch
- The ELKBeats Stack: K is for Kibana
- The ELKBeats Stack: Getting E, L, and K to play nice together
- The ELK Stack with Beats: Feeding Logstash with Beats (Insecure - so far)
- The ELK Stack with Beats: Securing the Beats-to-Logstash Connection
Installing Elasticsearch
As this product is also from elastic.co, much of this will sound familiar (the prerequisites):
# apt-get update ; apt-get install elasticsearch
This is also a large package (30+MB), but not so large as logstash.
Your configuration file is /etc/elasticsearch/elasticsearch.yaml, and in this case they do supply an example file. The settings recommended by the linode guide:
node.name: myserver cluster.name: mytest discovery.zen.ping.multicast.enabled: false index.number_of_replicas: 0
Elasticsearch thinks big: it assumes it's going to be a node in a big cluster of Elasticsearch servers. It also likes to go find local nodes to join. So give your node a name, and name the cluster something unique so it won't join someone else's node with the default name.
Their reasons are based on more knowledge than I have, but at the same time the first setting is the only one already present in the example file, and the third setting doesn't even appear to be in the current documentation.
To run by hand:
# /usr/share/elasticsearch/bin/elasticsearch
Seriously?! You put your binaries under /usr/share/? Possibly the only package in history to do that. At least logstash put itself in /opt/ - much saner.
Running the binary on console gets an error dump about logging not being set up correctly, and the program immediately exits. So let's try running it as a service - it's a package, that's the way you're supposed to do it:
# systemctl start elasticsearch
Given that it's Java-based, I'd give it a few seconds to start before trying this test:
$ wget localhost:9200
...
Saving to: ‘index.html’
...
Note that by default this URL is only available locally - you can't get it from a remote machine. The response page that wget saved to index.html is actually JSON:
{
"name" : "myserver",
"cluster_name" : "mytest",
"version" : {
"number" : "2.2.0",
"build_hash" : "8ff36d139e16f8720f2947ef62c8167a888992fe",
"build_timestamp" : "2016-01-27T13:32:39Z",
"build_snapshot" : false,
"lucene_version" : "5.4.1"
},
"tagline" : "You Know, for Search"
}
If you see this, you're up and running. Make it a permanent service:
# systemctl enable elasticsearch
This ensures elasticsearch will be running after the machine is rebooted.
Elasticsearch logs to the /var/log/elasticsearch/ folder.
Continue to The ELKBeats Stack: K is for Kibana, the next article in this series.
Bibliography
(This is the same Bibliography for all of the "ELKBeats Stack" articles.)
- https://en.wikipedia.org/wiki/Elasticsearch
- https://en.wikipedia.org/wiki/Kibana
- https://www.linode.com/docs/databases/elasticsearch/webserver-logs-with-elk-stack ... this is an excellent set of instructions that's significantly out-of-date (old URLs/addresses), which was nevertheless my main source of information
- https://www.elastic.co/guide/en/logstash/current/config-examples.html
- http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html (with the caveat that as of 2016-03, my instructions are more accurate than theirs ...)
- https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html
- Getting Kibana Up and Running
- Elasticsearch Getting Started
- Elasticsearch Reference >> Installation
- Elasticsearch Repositories (at elastic.co)
- Getting Started with Logstash
- https://www.elastic.co/guide/en/beats/libbeat/1.1/elasticsearch-installation.html
- Logstash Repositories (at elastic.co)
- How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on Ubuntu 14.04, Digital Ocean's uneven guide to this same subject, occasionally helpful but big on "install this" and short on "understand"
- http://main.justinflowers.ca/web/wordpress/?p=19