2019年 3月 の投稿一覧

Automating security updates on AWS and pushing the output to Slack


Introduction

Hello there! My name is Adam, an aspiring DevOps engineer who joined the Infrastructure team of Linkbal this year in February. My main focuses are optimizing the AWS environments and the server middleware. Occasionally I will make something cool and useful, most likely involving AWS, which I would like to share with you.

Today, I will be detailing how to schedule a security update script to be run on an EC2 Linux instance on a time schedule, then push the output logs to a Slack channel for everyone to see.

Objectives

First, let’s consider what we want to achieve, so as to determine what the end result should be:

  1. Automatically run security updates on an EC2 Linux instance on a pre-determined schedule.
  2. Be notified of the update and see the output in a Slack channel.
  3. Be able to change the time schedule or turn on/off without having to access the instance.

The 3rd point is important as it adds a layer of convenience to the method. If it ever becomes necessary to stop the updates, we want to be able to do so quickly and easily, without having to remove what we put in place.

Pre-requisites

  • AWS account with privileges to create a CloudWatch rule and use Systems Manager.
  • EC2 Linux-based instance with AWS CLI installed.
  • IAM role attached to the EC2 instance. 
  • Permission to access the EC2 instance via SSH connection and use root credentials.
  • A Slack account with a channel for receiving the output logs and the permission to create a webhook.

Plan and script

The source code originally came from here: Slack notification via curl
The diagram was drawn using a free account in LucidChart.
#!/bin/bash

# This script runs a security update then pushes the output to Slack.

VAR1=$(sudo yum update --security -y)
IP=$(curl ifconfig.me)

function post_to_slack () {
    SLACK_MESSAGE="\`\`\`$1\`\`\`"
    SLACK_URL={Slack web hook} # Replace this with your own web hook, without the braces

    case "$2" in
      INFO)
        SLACK_ICON=':slack:'
        ;;
      WARNING)
        SLACK_ICON=':warning:'
        ;;
      ERROR)
        SLACK_ICON=':bangbang:'
        ;;
      *)
        SLACK_ICON=':slack:'
        ;;
    esac

curl -X POST --data "payload={\"text\": \"${SLACK_ICON} ${SLACK_MESSAGE}\"}" ${SLACK_URL}
}
post_to_slack "Result of security update for $IP: $VAR1" "INFO"
exit 0

Procedure

  1. Create a Slack web hook for the channel you want to receive the log output in. Paste the web hook into the script as the value for SLACK_URL (without the {} braces).
  2. Upload script to the instance (take a note of the path). This can be done from the command line as such:
    $ scp -i /path/to/key.pem ec2-user@{IPv4 DNS}:path/to/file
    Or you can SSH connect to the instance, make a new file and paste the code in there:
    $ ssh -i /path/to/key.pem ec2-user@{IPv4 DNS}
    $ vi /path/to/file/update_security.sh
    It doesn’t matter were the file goes, so long as you know where it is.
    Be sure to change the permissions on the script so that it can be executed.
    $ chmod 775 /path/to/file/update_security.sh
    Install the AWS SSM agent: (installed by default on AWS Linux 1 AMIs since 2017.9 and on all AWS Linux 2 AMIs)[Manually install SSM agent on Amazon EC2 Linux instances]
    Add the policy “AmazonEC2RoleforSSM” policy to the role being use by the EC2 instance.
  3. Go to CloudWatch, select “Rule” on the left toolbar and then “Create rule”. Set the time expression by either a fixed rate or a cron expression. Note that the time will be in UTC, so depending on your timezone you will need to adjust the hour and possible day values. As an example, to update at 2am UTC Monday-Friday, the expression would be:
    0 2 ? * MON-FRI *
    For help with cron expressions in AWS, please see: AWS CloudWatch – How to schedule events
    Select “SSM Run Command”.
    Select the document “AWS-RunShellScript (Linux)”
    Target key = InstanceIds (can choose a different target, if you prefer)
    Target values = {Instance ID} i.e. i-a1b2c3d4e5
    Click on the “Configure parameters” drop down.
    Commands = ./update_security.sh
    WorkingDirectory = /path/to/file/
    ExecutionTimeout = 100
    Select “Create a new role for this specific resource”
    Click on “Configure details”. Give it a name and a description.
  4. The CloudWatch Rule uses AWS Systems Manager to call the SSM Agent which runs the script.  The script will run the update, then pass the output to a post function which uses the Slack web hook to post the output in your channel.
    Even if there are no updates due, the message will still be sent.
  5. Success! You can customize the name and image of the web hook in the Slack settings menu.

 

Conclusion

We have now configured for security updates to run on the AWS EC2 Linux instance on a specific time schedule, with the result of the update being sent to a channel in Slack. The script can be turned on/off easily from the AWS console (or by the AWS CLI) and we can change the time schedule there as well.
Let us now consider the advantages and disadvantages of this method.

Advantages

  • Quick and simple to set up.
  • The script is reusable; can be used on most if not all Linux instances on AWS. Only the Slack web hook would have to be changed if a different channel was required, otherwise no need to edit the script.
  • Only a basic understanding of AWS is required.
  • Virtually zero cost.

Disadvantages

  • CloudWatch rule must be stopped manually. Could set up a trigger with SMS and Lambda to stop the rule without the need for human intervention.
  • Only works for Linux instances. Windows would require something different.
  • Root privileges are required to set up (possibly avoidable with AWS Systems Manager).
  • Would be easier to understand if more details concerning the instance were put into the script, such as the name tag. (Can use ec2-metadata to obtain a lot of this information, but tags are not included in this list. Could use AWS CLI to get the instance name tag.)

Overall we have achieved what we set out to do. There are ways we could improve the script, but for now this is more than suitable.

I hope you have enjoyed this post, it’s a simple start with many more things to come. See you next time!

References

  1. Source code: http://blog.pragbits.com/it/2015/02/09/slack-notifications-via-curl/
  2. LucidChart: https://www.lucidchart.com/
  3. AWS CloudWatch – How to schedule events: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html
  4. AWS EC2 – Manually install SSM Agent on AWS EC2 Linux instance: https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-manual-agent-install.html
  5. Manually install SSM agent on Amazon EC2 Linux instances : https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-manual-agent-install.html

 

 

 

 

 

 

 

 


Progressive Web Apps – Part 1


Introduction

Hi! I’m Dennis and I’ve been working at Linkbal as a software engineer since October 2017.

This is the first part in a series of articles about Progressive Web Apps or PWAs. I am currently taking the following course at Udemy and I am writing these articles to help me consolidate the earned knowledge: Progressive Web Apps (PWA) – The Complete Guide

What are Progressive Web Apps

As you can notice by reading articles such as Seriously, though. What is a progressive web app?, it is very difficult to describe what a PWA actually is, but I will give it a try.

Progressive Web Apps (PWAs) are web applications that behaves like native mobile applications. They do so by using a set of techniques and technologies to provide end-users with features they usually find on native apps, such as:

  • Background synchronization
  • Push notifications
  • Offline accessibility
  • Location based behavior
  • Device camera access
  • Application shortcut in the home screen

It’s difficult to define PWAs as there is not a single specific criteria to consider an app a PWA, so it is even more difficult (if not impossible) to call an app a “perfect” PWA. Because some of the recommended enhancements rely on modern web browser capabilities and new capabilities will surely create new recommended enhancements in the future, I believe it is not absurd to consider the goal of a “perfect” PWA unattainable. Not only that, but depending on the nature of your application, implementing some of the enhancements proposed for PWAs may not even make sense.

That said, as written in the article mentioned previously, there are three baseline criteria to qualify an app as a PWA:

  1. It must run under HTTPS.
  2. It must include a Web App Manifest.
  3. It must implement a service worker.

The first criteria is quite straightforward and we will cover the other two later.

I’m not sure if it’s completely accurate to consider an app a PWA once these three criterias are satisfied, but I like to think so, as it gives us an objective way to make this judgement. Even after we can call our app a Progressie Web App, we will always be striving to reach the goals proposed for PWAs, such as making our apps more reliable, faster and more engaging [1].

Core Building Blocks

Now that we have an idea of what Progressive Web Apps are, let’s take a look at their core building blocks, or the core techniques and technologies we need to use to build a PWA.

Service Workers

Service workers are basically JavaScript code that runs in a background process, separately from the main browser thread, even if the application is not open in the browser. It is a technology well supported by modern web browsers 2 and a building block that provides a lot of the features that PWAs can provide.

Service workers can intercept network requests and cache those requests to improve performance. This cache can then also be used to provide offline access when the network is not available [3].

Because they run in the background, they are capable of providing some features that make web applications work like native apps, such as:

  • Background Sync
  • Push Notifications

Other Characteristics

There are a couple more characteristics to service workers that are consequences of their role in a PWA and the fact that they are independent from the application they are associated with:

  • Because a service worker is not blocking, synchronous XHR and localStorage cannot be used.
  • Service workers can’t access the DOM directly, so they need to use a messaging system to communicate with the page.
  • As they can intercept network requests and modify responses, service workers only run over HTTPS to prevent “man-in-the-middle” attacks.
  • Service workers become idle when not in use and restart when needed.

Application Manifests

An application manifest makes it possible for a web application to be added to mobile device homescreens. It is a simple JSON file that also describes how it should behave when “installed”. Among other things, it specifies the icon to be used, the URL to access when the app is launched and the background color of the splash screen displayed while the app is loading.

Responsive Design

At this point, responsive web design is a topic that does not need introduction, but it can be considered a core bulding block of PWAs because one of the goals is to create an application that fits any form factor [4].

PWAs and SPAs (Single Page Applications)

There’s one important confusion that is frequently seen regarding SPAs and PWAs, that the techniques and technologies to progressively enhance web applications to turn them into PWAs can only be applied to single page applications. The reality is that any web application can become a progressive web app no matter it is a SPA or a traditional multiple pages application.

Application Manifests

An application manifest is a JSON file that describes how a web app should behave once it is “installed” in the user’s device. For example, the browser uses that information to determine the icon to display when the application is added to the device’s homescreen.

Adding a Manifest File

Adding a manifest file to a project is as simple as creating a file named manifest.json and linking it in the “ of all the pages of the web app:

<link rel="manifest" href="/manifest.json">

Content of the Manifest File

Below, there’s an example content for an application manifest file:

{
  "name": "PWApp - Progressively Enhanced App",
  "short_name": "PWApp",
  "start_url": "index.html",
  "scope": ".",
  "display": "standalone",
  "background": "#5cb4f7",
  "theme_color": "#5cb4f7",
  "description": "A web app progressively enhanced to become a PWA",
  "dir": "ltr",
  "lang": "en-US",
  "orientation": "portrait",
  "icons": [
    {
      "src": "/assets/images/icons/icon-36.png",
      "type": "image/png",
      "sizes": "36x36"
    },
    {
      "src": "/assets/images/icons/icon-48.png",
      "type": "image/png",
      "sizes": "48x48"
    }
  ],
  "related_applications": [
    {
      "platform": "play",
      "url": "https://play.google.com/store/apps/details?id=com.example.app1",
      "id": "com.example.app1"
    },
    {
      "platform": "itunes",
      "url": "https://itunes.apple.com/app/example-app1/id123456789"
    }
  ]
}
  • name: specifies a long name for the app, displayed in the app install prompt[5], for example.
  • short_name: short name for the app that may be used, for example, in a device homescreen where there is insufficient space to display the full name.
  • start_url: URL that is loaded when the app is launched through the icon added to the homescreen.
  • scope: determines the pages the application manifest applies to. “.” applies to all pages.
  • display: defines the preferred display mode for the website [6]. Using standalone will make the app look and feel like a standalone app, for example.

A PWA launched browser mode (left), standalone mode (middle) and fullscreen mode (right). Image: New in Chrome 58

  • background_color: sets the background color used in the splashscreen displayed while the app is loading.
  • theme_color: this color is shown, for example, in the browser toolbar and in the top bar on Android’s task switcher, what increases the sense of OS integration even more.
  • description: this field is used whenever the browser needs some description for an app.
  • dir: specifies the text direction for name, short_name and description.
  • lang: together with lang, it helps to display right-to-left languages.
  • orientation: sets the default orientation, but bear in mind users prefer selecting the orientation themselves.


Portrait and landscape orientation options for the app manifest. Image:The Web App Manifest

  • icons: icons to be displayed in the homescreen. Multiple icons with different sizes can be provided in the array and the browser will determine the “best” icon depending on the device.
  • related_applications: array of native applications that can be installed through the applications stores of the respective platforms.

Conclusion

In this first part of the series about the course on PWAs I am taking on Udemy, I tried to define what Progressive Web apps are, presented their building blocks and an introduction to application manifests. In the next part, we are going to go deeper into the subject of application manifests and start talking about service workers.

References

  1. Progressive Web Apps – Google Developers
  2. Service Workers – Can I use
  3. Introduction to Service Worker – Google Developers
  4. Your First Progressive Web App
  5. The Web App Manifest – Google Developers
  6. Web App Manifest #display – MDN