I’ve learned a lot since my last post. One of those things is that I was wrong… setting up Logstash on your Redis nodes isn’t such a bad idea. Another thing that I have learned is that fluentd / td-agent is not as great as I thought it was. My revised plan as depicted in the updated design below is to use Logstash Forwarder on my non-Windows nodes and send that to a Logstash instance that does nothing but stick things into a local Redis instance. Doing this also eliminates the need for my custom receiver named Sawyer. The last change noted below is that I have upped my number of Elasticsearch data nodes and Logstash indexers to 3 each. This was a direct result of load. I also like the improved distribution of shards by having more than 2 nodes in a 5×2 shard setup. » Read more

 genebean        

I’ve been working on a new logging system based around Elasticsearch, Logstash, and Kibana. One of my biggest challenges was that all the recommended designs I found said that logs should go from a shipper to Redis. The problems with this are twofold: Logstash doesn’t seem like a good fit for Windows. The biggest issues are that it relies on Java which isn’t something that is very sellable to any Windows admin that I know. The other is that it simply didn’t work reliably in my testing. The 1.4.x series had performance issues and the copy of 1.5.1 I just tried on Windows 7 is throwing Windows Event Log error: Invoke of: NextEvent<br></br> Source: SWbemEventSource<br></br> Description: Timed out errors under even the simplest of tests. Unlike other tools it also » Read more

 genebean