Why do we need automation?

Let's think now why we need to automate processes. One instant point that comes to my mind is to save time or hasten processes.

Actually, when we were working on the waterfall model, we had to iterate with development life cycle stages one by one, and modules were tightly coupled. Due to this, the development was a very long-term, complex process.

Now we move on to the agile development process, where each iteration between phases is very fast. It works in continuous integration (CI) and continuous development mode.

During this mode, we have a repeated set of tasks to complete. If we do all of these manually, it will take more time and there will also be a chance of human error. So, we automate the repetitive set of tasks to hasten the process and save some time.

As well as repetitive tasks, we also have a few critical and complex tasks that do not occur frequently, but need more man hours to be completed. As it is manual, there is a higher chance of getting errors. For example, data migration—we need consistency to perform migration work.

Here, if we automate the process, we will be able to remove the human dependencies and manual errors, which will result in improved consistency. Likewise, if we use a Cisco CloudCenter type tool where we model the app once, we can deploy and migrate it anywhere without human intervention.

With automation, we gain speed and momentum in our delivery process. For example, an infrastructure team is given the responsibility of creating a thousand VMs, which all need an operation system to be installed, security patching, and other configuration to be done. If we do manual provisioning, this task will probably take weeks to complete, and there will be at least a 10% chance of error unless we have a team of perfectionists. 

Now, suppose that I am using Ansible to perform this task; it will take just a few hours to complete all these tasks and deliver the VMs. In Ansible, we just need to write a playbook to create a VM with all the configuration details along with security patches. Whenever we need a VM, we just need to run the Ansible playbook and it will create a complete infrastructure, ready for delivery.

Automation also helps us perform scheduled tasks; for example, we have to make a backup of all the VMs, applications, and databases. For the sake of minimal interference with application performance, we mostly schedule backup for odd hours.

If it is a manual job, we need a human who will be working these odd hours. Again, here, there is a chance for errors or mistakes. There is also the possibility that we might miss the backup for a few apps.

However, if we have automated the backup task using bash, or some other automation tool, it will start the process on time and complete it with positive results. If there are any errors, you will get a notification, and then you can take appropriate action.

Automation makes life easier for system administrators.

Let's summarize the benefits of automation:

• Time and cost saving
• Faster deployment and delivery
• Elimination of the chance of human error
• Improved collaboration
• Easy to implement security

To adopt the DevOps process, we have to complete a few prerequisites. As said earlier, DevOps is a practice, and to adopt it, we have to adhere to the process of DevOps.

So, to adopt the DevOps culture, we need to have the following things:

• CI: In CI, process development teams store their code in a central repository from which automatic build and test processes run. CI processing helps us find bugs quickly and also lets you validate updates and deliver them fast. For example, we use Jenkins, which provides a complete CI/CD process.
• Continuous delivery (CD): In this process, all the developed code is automatically built, tested, and ready for release and production. Once CD is successfully implemented, a developer will get all the production-ready artifacts/packages available for rollout.
• Microservices: Its process is to build an application into a set of small services. Each service runs its own processes and communicates with other services using a well-defined framework or API. For example, we either use Docker Swarm or Kubernetes, which run microservices in containers. 
• Infrastructure as Code: The best part of DevOps is that we defined Infrastructure as Code, and it is idempotent in nature. Infrastructure is coded and also stored similarly to the applications that have version control and a CI process. We know that clouds give us API-level access to all the components and services. We use APIs to code the infrastructure instead of provisioning it manually. In the AWS environment, we have CloudFormation to code complete infrastructure into a simple JSON code. Infrastructure as Code also has the following two subprocesses:
  • Configuration management: In the configuration management process, we code all the configuration-level activity such as patch management, OS update and upgrade, and the security hardening process. Because configuration management is a repetitive process, it consumes too much time for system or operation teams. For this, we use tools such as Ansible, Chef, and Puppet, where we code configuration management stuff once and run it when it is required. In the following code, its Ansible playbook code snippet installs the IIS service on Windows. It's easy and simple. We write it once and can run it on Windows Server:

---
- name: Install IIS
  hosts: all
  gather_facts: false
  tasks:
    - name: Install IIS
      win_feature:
        name: "Web-Server"
        state: present
        restart: yes
        include_sub_features: yes
        include_management_tools: yes

• • Policy as code: Once we have coded our infrastructure, it's time to code our policy as well, which will include security and compliance. There are different cloud vendor-specific tools that let you manage the security and compliance, and also send you alerts and notifications. In AWS, we have AWS configuration service. We will study AWS configuration in more detail in the next chapter.

• Monitoring and logging: This is an important section for the whole organization. Here, we get the details of what is happening in the backend infrastructure and application and, at the same time, we also get to know the status using events and alarms. Here, we use monitoring and logging tools to automate the monitoring and log management. We have the Elasticsearch, Logstash, and Kibana (ELK) stack, New Relic, Nagios, and so on, which perform monitoring processes and send you alerts on an event basis.
• Collaboration and communication: As we know, DevOps is a practice where the development and operation teams work in collaboration. Here, we define the workflow for top to bottom-level communication. We also use tools that can be used for code tracking; communication between the team using chat also can give detail about bugs.