Automate.Insights Part 1; General Overview

What if instead of having to know the intricacies of every server and software in your stack, you could simply pick a type of software, and let your tools do the rest? 

This was a question proposed recently by Thirdwave Insights and their topology deployment tool. I have spent a few days getting to know the technology, and wanted to provide a little feedback. 

Note: I wrote this in plans to be a 3 part blog. Feel free to skip if you are looking for the more technical tracks. 

First, What is Automate.Insights? 

Automate.Insights is a visualization and formatting tool for the creation of application stacks. Working in tandem with Chef and Automate.Insights custom plugin (Knife-Topo), it is designed to create software stacks on multiple nodes at once rather than one at a time. Secondly, with it's human friendly ui interface, it allow a casual Chef user to customize servers to their use case and task. This allows us to think and manage an application from a global perspective, rather than a single node at a time. This is in many ways more natural to our way of thinking than a list of cookbooks. 

Clarification of Terms

After testing the Automate.Inisghts tool it was apparent that it was a well thought out system. However I did want to clarify some terms to avoid initial confusion. Basic understanding of Chef is recommended as well. 

Business System: A large enterprise typically has multiple applications supporting the work of different business units:

  • customer data analytics
  • internal employee systems
  • R&D application
  • company site/marketing

Each application contains unique, often proprietary software that needs to be managed. Understanding each of these as a whole business system the best way to ensure the business unit's operational requirements are met. On a Chef server, separation between business systems might be achieved using organizations; or one organization might manage multiple business systems, using tags to distinguish them.

Topology: The topology of nodes and their configuration information for a deployment of the business system. Using knife-topo, you can map out the topology of a cluster of nodes and ensure that they are properly configured.

Let's take an example from one of our business systems, the company website. 

A typical company site could consist of

  • a webserver
  • a dbserver
  • a load balancer

These are all separate and can be managed as individual cookbooks, but together they are one functioning "topology" of nodes. By this definition, it is a topology. 

Blueprint: This is like a topology, and also not which can be slightly confusing. Essentially, a blueprint is an outline or template for a topology that can have various tweaks done to it for adjusting to certain use cases. 

For example, in our company site information, traditional setup for this is to have a dev, stage, and prod server setup for the site. The original topology calls for

  • an app server
  • a database server
  • a load balancer. 

However, in the dev and staging server, it isn't necessary to have a load balancer. You dont HAVE to route traffic, so why incur the cost of running one needlessly 24/7? That's where the blueprint comes into play. It allows you to make adjustments to the overall topology, override attributes, then push that blueprint to the Chef server for provisioning. In doing so, the staging and dev blueprints will look more like

  • an appserver
  • a database server

This is somewhat simplified, but honestly not by much. It becomes more simple to think of things from a holistic perspective. More importantly, with knife-topo, you can provision all the machines at once. It truly becomes a one click proccess. 

Things I really liked

  • Leverage existing Infrastructure: Those of you who have put in your time with cookbook development know the learning curve involved with Chef. Being able to leverage existing cookbooks rather than abandon it for yet another config management tool is a welcome alternative. It means less time porting infrastructure, and more time working with what already works well.  
  • True one click deployment: Running through the basic training module and the topology.json file, I was able to provision a three node cluster with a one line command. Not one line that wraps over 4 in your terminal, but one line. This simplification is a great improvement over the status quo. 
  • Allows for holistic viewpoint: Automate.Insights encourages you to think about what should be consistent across a system, and gives you views so it's easier to line up how your software stacks are configured.

Possible Concerns

  • You need to know Chef to use it: I am a more advanced Chef user, so after a couple hours I was more than fine with the product and the benefits it was offering.

    If you are unfamiliar with Chef however you would be better served starting here getting an understanding  of the basics of Chef before coming back to this blog. The Automate.Insights tool was designed to simplify your existing cookbook infrastructure, so if your organization has existing cookbooks you may be fine. 

  • Chef-provisioning also sets up topologies: In the last year much buzz has been made about the use of provisioning, which uses cookbook resources to build machines. There is some overlap between this and what knife-topo and Automate.Insights does. In fact, speaking with the A.I team, it is their intention to integrate with Chef Provisioning. For those of us who have been waiting for provisioning to mature, Automate.Insights might be a better option to get started with deploying and configuring topologies. 
  • Is it really better than the status quo?: This is the key question. Rather than try to explain, I decided to show it with a more practical application.  

    Click here for my practical demo of the Automate.Insights tool.