Setting up Logstash with Amazon EC2 and Route 53

Setting up Logstash with Amazon EC2 and Route 53

Introduction

Logstash is software used to perform centralized log collection, normalization and enrichment. Used in conjunction with Elasticsearch and Kibana it becomes a powerful log management tool, also known as ELK stack. As a good start, we are going to deploy Logstash 1.5 on Ubuntu 14.04 at EC2, create DNS entries with Route 53 and generate SSL certificates for Logstash.

Initial setup – install Java and Logstash

  1. In order to run Logstash we need Java. It is recommended to use Oracle Java 8 (generally it runs on version 7 but sometimes there are issues). It should also work with OpenJDK.

  1. Install logstash

 

Add DNS entry for Logstash server and set hostname

  1. Go to Route 53 service in AWS console.

001

  1. Go to “Hosted zones” and click Create hosted zone. We are going to use Private Hosted zone for Amazon VPC type. Name it as you like, for example myzone.local and select your VPC.

002

  1. Go to your hosted zone and click Create record set. Put your hostname and private IP address of your server.

003

  1. After A record is created, change your hostname by editing /etc/hostname and /etc/hosts files. Now it should be, for example your-name.your-domain.local.
  2. Reboot your server or type the command below to apply new hostname

sudo hostname your-name.your-domain.local

Generate certificates for Logstash

We are going to use these certificates later, to transmit our logs encrypted. That’s all for now! In the next part of this tutorial we’ll setup Elasticsearch node at AWS.

References

https://www.elastic.co/products/logstash
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-1-7-logstash-1-5-and-kibana-4-1-elk-stack-on-ubuntu-14-04

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.