Packs provide a way to one-click setup monitoring. As of today we have packs for Base Operating System Monitoring, Elastic Search, IIS, Memcache, Mongo DB, MS SQL, Redis and Riak. Packs are really simple and quick to create so we’ll likely balloon up in number over the next few weeks.
Behind the scenes what packs do is automate the installation of plugins, dashboards and alert rules which are then all managed as a single ‘pack’ entity. This makes it easy to install, uninstall and upgrade with just a single click.
Let’s say you want to monitor your ElasticSearch cluster. This can now be done in 3 simple steps:
1. Install the Dataloop Agent on your ElasticSearch servers
2. Install the Elastic Search pack from the Packs Store
3. Tag your Elastic Search servers with the ElasticSearch tag
You’ll now have a dashboard setup so you can view how well your cluster is performing in addition to some rules that will alert if anything goes wrong.
The easiest way to get involved is to request a pack for something. We’re happy to create them on demand or you can even create them yourself following these simple instructions:
1. Fork our Packs repository on Github
2. Create a new directory for your Pack and create subdirectories for dashboards, plugins and rules
3. Create a package.yaml in the root of your Pack that describes it and put your plugins, dashboards and rules into the relevant folders
5. Submit a pull request
Packs are usually developed inside Dataloop by creating a plugin, a dashboard and some rules in the browser. When they are perfect you can export by copying and pasting the plugin and pressing the export buttons to save the dashboard and rules as yaml files.
A couple of caveats: We only support Python plugins for now as they run inside a little agent sandbox and we only bundle a Python 2.7 interpreter and list of libraries. Also, the Pack names need to be a single word with no spaces.
This launch is just the initial iteration of our central pack store. We intend to make it easier for people to create packs from within Dataloop via a little pack creation Wizard so you don’t need to do all of the downloading and copying and pasting.
We also believe companies should start to build up their own locally developed packs that encapsulate the monitoring knowledge for each of their own micro services. So we’ll be letting you manage your own Packs inside Dataloop in future in addition to the ones we provide.
Some people have multiple environments or clusters which makes installing a single pack less than ideal.
In future we’ll be adding Tag level variables so that a single pack can be used to collect data from every server by setting different user names, passwords and url’s for each. For now the workaround is to update the plugin to read config from disk.
We’ll also be adding the ability to re-scope dashboards so that each dashboard installed by a pack can be switched via drop down to show a different cross section of your servers. We’ll also allow favourites on these so it’s quick to switch. The workaround in the meantime is to export, find and replace on tag: <tagname> and import. You can do this on dashboards and rules to quickly duplicate dashboards for different environments.
Finally, our aim is to automate everything. You should be able to deploy our Dataloop agent via configuration management to all of your servers and then watch the magic happen and packs bring us one step closer to this. Next year we’ll be adding the final piece: our auto discovery engine. This will dynamically tag agents based on what software we detect they are running and then install the relevant pack.
If you have any feedback on any of the above please drop us a line at firstname.lastname@example.org