More than six months away, I've put both Splunk and Logstash (plus ElasticSearch and Kibana) to trial. Since the beginning of my experiment, the difference between both products was pretty clear. I was hoping that ELK would be a nice solution to aggregate GBs of logs every day and more importantly to allow me and my team to dig into those logs.
Unfortunately, ELK failed hard on me. Mostly because of ElasticSearch: java threads on the loose eating my CPU, weird issue where a restart would fail and ES would become unavailable, no real storage tiering solution, and so on.
ElastickSearch looks like a very nice developer playground to me. It's quite badly - if at all - documented on the sysadmin side, meaning if you're not a developer and don't have weeks to spend reading API documentation you will have a hard time figuring out how to simply use/tune the product.
Also, this data storage backend has absolutely no security features (at the time I was testing Logstash). Just imagine you put valuable data into a remotely available database/storage/whatever and you're told that you are the one that should create a security layer around the storage. Imagine a product like SMB, NFS, Oracle DB, Postgres, etc. with zero access control feature, no role. Anyone who can display a pie chart in Kibana can gain full control of your ElasticSearch cluster and wipe all your data.
Logstash too has important issues: almost every setting changes require an application restart. Writing grok pattern to extract data is an horrendous process, and a new pattern won't be applied on past data. Splunk on the other hand will happily use a brand new pattern to extract values from past logs, no restart required.
Logstash misses also pipes, functions, triggers… Search in Kibana is not great. The syntax is a bit weird and you can easily find nothing just because you've forgot to enclose your search string with quotes… or is it because you put those quotes? And of course there is this strange limit that prevent you from searching in more than seven or eight months of data…
I've tried, hard. I've registered to not-so-helpful official mailing lists for Logstash and ElasticSearch and simply reading them will show you how far those products are from "production approval". I've used it alongside with Splunk, it boils down to this statement: Kibana is pretty, you can put together many fancy pie charts, tables, maps and Splunk is useful, reliable, efficient.
Yes, I'm moving to Splunk, @work and @home. My own needs are very easily covered by a Free license, and we are buying a 10 GB/day license for our +350 servers/switches/firewalls.
You might say that Splunk is expensive. That's true. But it's way less expensive to me than paying a full time experienced Java developer to dive into ElasticSearch and Logstash for at least a year. Splunk has proper ACL that will allow me to abide by regulations, a good documentation, and a large community. It support natively storage tiering, too.
ELK is a fast moving beast, it grows, evolves, but it's way behind Splunk in terms of maturity. You can't just deploy ELK like you would do for MySQL, Apache, Postfix, Oracle DB, or Splunk. Lets wait few years to see how it gets better.
So, nearly after 4 years, do you still have the same opinion? please share with us!
Well, I'm a happy Splunk admin :)
My opinion about Splunk is very good and I don't see any reason to change, for now. And more importantly I don't have an up-to-date opinion about ELK: it's very time consuming to test live (real data, real problems, real work scenarios) and my time costs more than a Splunk license.
It would be nice to compare again though…