There are a million and one log-management tools running in networks all over the world. Some are simple file-searching utilities. Some dig a little deeper. But all are focused on a specific log file from a specific service or network device. They don't always talk to one another, however, which creates havoc for admins.
LogLogic aims to bring all these disparate log files together in a palatable, sustainable way -- and they largely succeed with the Series 3 appliances. LogLogic's solution is a collection of two appliances: the LX series serves as the initial log-file destination and the console for reporting, while the ST series handles log-file archiving.
Both appliance families are built on standard 2U and 3U rack-mount server hardware platforms, leveraging a Linux core, MySQL, Apache Tomcat, and Java to deliver the whole solution. The kernel is a recent 2.6.12 version, and LogLogic does not prohibit root-level SSH log-ins to the appliances. The hardware itself is quite robust, with both appliances sporting redundant power supplies and a 3ware 8500-series SATA RAID controller.
The LX 2000 I had in the lab was designed only to hold data for 90 days in a high-volume environment, so it had roughly 500GB of storage in a RAID 5 array of four 160GB drives, 4GB of RAM, and two 2.8GHz Intel Xeon CPUs.
The ST 3000 provides storage for the LX's archived log data. It's equipped with 2GB of RAM, two 2.8GHz Intel Xeon CPUs, and roughly 2TB of storage across a RAID 5 array of nine 250GB drives, with a mirrored set of boot drives. Both appliances have a significant amount of internal disk, but can be integrated into SAN and NAS environments for more archival storage.
Set 'em up, knock 'em down
Setting up the appliances is straightforward, with a serial console connection necessary to configure IP information. Further configuration and admin tasks are done via the Web interface. The Web UI is reasonably fast and well-rendered, but occasionally has a touch of that well-known Java sluggishness.
You can quickly access high-level info using LogLogic's status dashboard, which shows the current mps (messages per second) rate, existing alerts, system performance, and total message counters. Another click shows all defined syslog sources and their individual message counters, and a Java-based real-time syslog viewer applet shows the current syslog message streams as they flow into the box.
One of the appliances' nicer features is their log source autodetection capability. To configure a device to log to the LogLogic appliance, you need only to point the device's syslog output to the appliance's IP address. When the appliance sees traffic from that source, it is automatically classified according to the determined data type.
LogLogic supports quite a few log formats, from routers, IDSes, and firewalls to more specific formats such as Microsoft Exchange and IAS and even Squid proxy logs. Log files can be imported directly into the LX 2000 via HTTP, HTTPS, SCP, FTP, or SFTP, although log-type restrictions can reduce this feature's usefulness. However, the direct importing is hugely important when dealing with non-syslog services such as Exchange, as this process can be cumbersome without it.
In the lab, I configured an LX 2000 and an ST 3000 for log-file archiving and began blasting syslog messages at the LX 2000 from a variety of simulated sources. Using a homegrown syslog injection script, I achieved and verified the top-end 3,000 mps rate on the LX 2000. The promised 150 percent burst rate was almost spot-on as well: I maxed out at 4,450 mps, and sustained that level indefinitely. The ST 3000's top-end rate is purported to be 50,000 mps; while I couldn't generate enough traffic to approach that level, I did get it up to just under 10,000 mps.
In production, the LX 2000 acts as the syslog server for all local devices and periodically archives data by compressing the log data and transmitting it to the ST 3000 via TCP. This setup means you can place LX 2000s at remote sites to collect log data generated there, significantly reducing the traffic that must flow back across the WAN to the main data centre.