I am a senior software developer with <a href="https://www.altusgroup.com" target="_blank">Altus Group</a>. My professional interests include cloud computing, web development, web APIs, software architecture, and machine learning. I has achieved the AWS Solutions Architect - Associate certification. I also earned multiple certifications in Data Science from Coursera.
I hold a BS in Computer Engineering and a Masters in Management in Science & Technology.

Madhu Lakshmikanthan

<base target="_blank">

## Introduction
One of the use cases for Lambda I am intrigued about is report and chart generation. For example one could think of a scenario where groups within an organization familiar with Python and other reporting tools may want to generate reports and charts for the executive audience. If they can be provided some training on developing the python code as Lambda and pushing it to the repository tied to the CI/CD pipeline, then it frees them up to focus on what they do best and reduce dependency on the IT team to deploy artifacts that provide critical business insights. IT or the web team can just provide the cloud locations for these reports and the Lambda developer will ensure that the code they write is configured with that target location. The ultimate destination like the web site or portal will just render the reports from these locations within the web layout already familiar to the target audience.

This blog post shows how to implement a non-trivial python lambda that generates a chart from a public data source. Since the chart library dependencies caused unzipped deployment package size to exceed the file size limit of 250Mb, I had to implement it as a python lambda container. The python code in question is the same as my [previous post](https://www.know2drive.com/blog/charting/) showcasing some of python's charting capabilities.
It is fairly straightforward to take python code written using Jupyter notebook and modify it to work as a AWS Lambda function. But the unfamiliar nuances for a developer are all around packaging and deployment. This blog will also walk you through the Terraform script to build and deploy the Lambda as a container to AWS. If you implement this code in your AWS account you may incur some minimal charges when you run the Lambda. 
If you are also interested in other cloud serverless implementations like Azure Function and Google Cloud Function, your learning exercise can be to take this code and modify it to work on those platforms as well.  

You will need your test AWS account and have programmatic access enabled from your local development environment in order to deploy the code described in this post.
If you do not have an AWS account, please create a free account using [this](https://portal.aws.amazon.com/billing/signup?nc2=h_mobile&redirect_url=https%3A%2F%2Faws.amazon.com%2Fregistration-confirmation#/start/email). Next, get programmatic access following the steps in [here](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).

## Implementing existing python code as AWS Lambda
I added the **def handler(event, context)** function to the code you see in my [previous post](https://www.know2drive.com/blog/charting/) to implement the Lambda as you can see in [here](https://github.com/madhuvanesh/python_lambda_container/blob/master/app/app.py).
The lambda will be triggered when the source CSV file is uploaded to a S3 location. This S3 location is specified as a resource **inputlocation** in the [Terraform script](https://github.com/madhuvanesh/python_lambda_container/blob/master/terraform/lambda.tf).
The Lambda will process the CSV and generate the chart in an output location which is specified as **outputlocation** in terraform/lambda.tf. The chart is a .png file defined in **locals.outputfilename** in the [Terraform script](https://github.com/madhuvanesh/python_lambda_container/blob/master/terraform/lambda.tf).
See the code on [Github](https://github.com/madhuvanesh/python_lambda_container) for more details.

## The Docker file that describes what goes into the container
Here is what the [Docker](https://github.com/madhuvanesh/python_lambda_container/blob/master/app/Dockerfile) file describes:
1. Copy the requirements.txt with the list of dependent libraries onto the container's current directory.
2. Install the libraries on the root folder as defined by the AWS reserved variable LAMBDA_TASK_ROOT.
3. The matplotlib library uses a folder defined in the environment variable MPLCONFIGDIR to store some caches to improve performance. The Lambda execution environment provides a file system for our code to use at /tmp. So the Docker file has commands to create a /tmp/matplotlib folder and set the environment variable to it.  
4. Finally the CMD is set to app.handler function - The handler() function in the app.py file.

## The Terraform code to package the Lambda and deploy it
Install [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli#install-cli).
The [Terraform script](https://github.com/madhuvanesh/python_lambda_container/blob/master/terraform/lambda.tf) does the following:
1. Creates the input and output s3 locations.
2. Creates the ECR repository such that it can be deleted when terraform destroy is called.
4. Creates the docker image and pushes the same to the ECR repository. 
5. Creates the AWS IAM role that allows the AWS Lambda service to execute the Lambda function.
6. Creates the IAM policy and assigns it to the role. The IAM policy will allow the creation of Cloudwatch log group and log stream, 
provide read access on the input s3 location and write access on the output s3 location.
7. Creates the lambda function and assigns the AWS IAM role to it. Also, adds the environment variable for the output s3 bucket and the output filename.
8. Finally creates the trigger for the Lambda which is upload of a file to the input s3 location.

The terraform script has been tested on Linux subsystem in Windows. Also, this was tested using Docker Desktop on Windows. 
So if you are on Windows:
1. Install Docker desktop from [here](https://www.docker.com/products/docker-desktop/).
2. Run Docker Desktop.
2. Please enable the Linux subsystem. See these [instructions](https://learn.microsoft.com/en-us/windows/wsl/install).
3. Launch the bash terminal and execute these:
### cd \<location where you cloned the repository from Github\>/terraform
### terraform init
### terraform plan -out=./plan.txt
### terraform apply ./plan.txt
If you see no errors from the above commands , you should see the Lambda deployed in your AWS account as 
**reportgen-<randomId>-function**
Upload the CSV file [cost_of_living_v2.csv](https://github.com/madhuvanesh/python_lambda_container/blob/main/test_input/cost-of-living_v2.csv). You will see the output .png file generated in the output S3 location.

Once you have tested, cleanup using:
### terraform destroy
so that you do not incur charges.

Hope this post was useful to you. You can leave your comments in the **Enter comments here..** box at end of this page and hit **Submit Reply**.


This blog post shows how to implement a non-trivial python lambda that generates a chart from a public data source. Since the chart library dependencies caused unzipped deployment package size to exceed the file size limit of 250Mb, I had to implement it as a python lambda container.

Developing an AWS Lambda container in Python and deploying via Terraform

Why I wrote this post
Python has powerful data analysis and charting feature via the pandas and matplotlib libraries as any data science professional will attest. I wanted to target this blog primarily to developers who are not yet into machine learning and want to strengthen their data analysis skills in python.

Charting using python, pandas, matplotlib and a public dataset


A common paradigm in web development is the deployment of a static website to Amazon S3 instead of hosting in a web server. Web applications relying on server-side rendering frameworks like ASP&#46;NET, Spring, and Ruby on rails to name a few, are NOT candidates for deployment to S3. But even with these server-side rendering-based web applications, static content can still be stored in S3 to leverage CDN capabilities provided by AWS Cloudfront. Authentication is not available "out of the box" for S3-based web applications.

In this post, I will cover how AWS Cloudfront, Lambda@Edge, and Cognito can be used to implement authentication for a static React website published onto S3. We will cover authentication via Cognito. 

To follow the steps in this article and implement the authorization you will need to use your AWS account or create a test account. Please do NOT use your work AWS account. See <a href="https://aws.amazon.com/free/?trk=b53e5260-30f6-45cf-9610-30be63969231&sc_channel=ps&s_kwcid=AL!4422!3!610000101357!p!!g!!aws%20cloud&ef_id=Cj0KCQjwmdGYBhDRARIsABmSEeNTT51cmO0lzvTPMogJTFO2_En70m1S_FXZ2lk4S9MU-m4qPpXtJ34aAvhZEALw_wcB:G:s&s_kwcid=AL!4422!3!610000101357!p!!g!!aws%20cloud&all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all" target="_blank">AWS Free Tier</a> to create free tier account.

You will also need to sign up for a Core Weather free account on <a href="https://developer.accuweather.com/" target="_blank">developer.accuweather.com</a> since the my-weather React website uses the AccuWeather API.

## Clone and test the static React site locally

The demo application for this post is a React application that fetches the next 12 hours' weather for the user's location. It leverages the browser's geolocation capability and uses AccuWeather's API to fetch the weather. Clone the code for this from <a href="https://github.com/madhuvanesh/my-weather.git" target="_blank">Github</a>.

Update **\<location of the React app files>\src\config.json\>->accuweatherappkey** with the app key from your AccuWeather account.

In the command prompt:

 - cd \<location of the React app files\>
 - npm start

Open <a href="http://localhost:3000" target="_blank">http://localhost:3000</a> to view it in your browser. 

Once the app comes up, wait a few seconds for the location to show up in the "Click here…" button. Once it does, click on the button and wait for a few seconds for the next 12 hours' temperatures to get populated. You should see a screen like this:
![my-weather](../../images/blog/my-weather.jpg)
3.  'npm run build' - to build the app for production to the `build` folder.
4.  The contents of the 'build' folder will need to be copied to the s3 location belonging to the web application as described in the [Replace s3 location with the React app files section](#replace-the-s3-location-with-the-react-app-files) below.

## Deploy and test the sample application for cookies-based authentication

The inspiration and main reference for this blog post is the Amazon article: <a href="https://aws.amazon.com/blogs/networking-and-content-delivery/authorizationedge-using-cookies-protect-your-amazon-cloudfront-content-from-being-downloaded-by-unauthenticated-users/" target="_blank">Authorization@Edge using cookies: Protect your Amazon CloudFront content from being downloaded by unauthenticated users</a>. It will be easier to review the Amazon article in its entirety after you complete the steps in this current post. From the "Deploying the Sample solution" section in the Amazon article, launch <a href="https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:520945424137:applications/cloudfront-authorization-at-edge" target="_blank">AWS Serverless Application Repository</a>. Keep the defaults in the Application Settings page that comes up except HttpHeaders. You set HttpHeaders to:
```
{"Content-Security-Policy": "default-src 'none'; img-src 'self'; script-src 'self' https://code.jquery.com https://stackpath.bootstrapcdn.com; style-src 'self' 'unsafe-inline' https://stackpath.bootstrapcdn.com; object-src 'none'; connect-src 'self' https://*.amazonaws.com https://*.amazoncognito.com https://intense-hollows-72278.herokuapp.com https://dataservice.accuweather.com",
"Strict-Transport-Security": "max-age=31536000; includeSubdomains; preload",
"Referrer-Policy": "same-origin",   
"X-XSS-Protection": "1; mode=block",   
"X-Frame-Options": "DENY",   
"X-Content-Type-Options": "nosniff"}
```
These URLs in the value for "Content-Security-Policy" above:

 - <a href="https://intense-hollows-72278.herokuapp.com" target="_blank">https://intense-hollows-72278.herokuapp.com</a>
 - <a href="https://dataservice.accuweather.com" target="_blank">https://dataservice.accuweather.com</a>

are the ones needed by the my-weather website to make the API calls to fetch the weather information without Cross-Origin Resource Sharing (CORS) errors.

In the Application Settings page that comes up when launching  <a href="https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:520945424137:applications/cloudfront-authorization-at-edge" target="blank">AWS Serverless Application Repository</a>,
check on:
 - [ ] I acknowledge that this app creates custom IAM roles and resource policies.

Click the **Deploy** button.

Now, check AWS Console-> CloudFormation ->Stacks and make sure the serverlessrepo-cloudfront-authorization-at-edge comes up with status=CREATE_COMPLETE like this:
![CloudFormation Stack](../../images/blog/cloudformation_success.jpg)

If you see errors in the stack created under CloudFormation, it is most likely because of incomplete cleanup after previous attempts at running <a href="https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:520945424137:applications/cloudfront-authorization-at-edge" target="blank">AWS Serverless Application Repository</a>. 
In AWS Console, disable the last distribution created in **Cloudfront->Distribution**. Once the status shows Disabled, delete the distribution. Delete **CloudFormation->Stacks->serverlessrepo-cloudfront-authorization-at-edge**. Wait for the stack with DELETE_IN_PROGRESS status to disappear in CloudFormation->Stacks. If the deletion of the CloudFormation stack results in errors, you will need to delete all of the Lambdas created by the Cloudformation stack creation attempt and then reattempt the deletion of the CloudFormation stack. s3
Next, clean up any buckets you do not recognize in S3. After that reattempt creating the stack following the steps above.

Once the deployment is complete, in your <a href="https://aws.amazon.com/console/" target="_blank">AWS console </a> window, search for Cognito and then click on **Manage User Pools**. You should see a user pool created with the name **serverlessrepo-cloudfront-authorization-at-edge**. There will be no users created if you followed the defaults in <a href="https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:520945424137:applications/cloudfront-authorization-at-edge" target="_blank">AWS Serverless Application Repository</a>.
In the user pool, create a user as described in <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/signing-up-users-in-your-app.html" target="_blank">Signing up and confirming user accounts - Amazon Cognito</a>. You only need to follow the steps from **Verifying contact information** section. Other information like Callback URL(s) and Sign out URL(s) will have been created already by the cloud formation that you completed earlier.

Now, in your AWS Console window in the browser, go to CloudFront and invoke the distribution that was just created. Copy the **Distribution domain name** and paste it into a browser tab. You will see a Cognito login prompt:
![login](../../images/blog/cognito_signin.jpg)

Login and you will see the landing page. Click on the Signout link on the page and you will need to log in again. You have now, tested the sample application.
In this blog post, you have seen a demonstration of serverless in action. The magic behind the authentication flow is implemented by AWS Lambda@Edge. Review **The sample solution, step by step** section in <a href="https://aws.amazon.com/blogs/networking-and-content-delivery/authorizationedge-using-cookies-protect-your-amazon-cloudfront-content-from-being-downloaded-by-unauthenticated-users/" target="_blank">Authorization@Edge using cookies: Protect your Amazon CloudFront content from being downloaded by unauthenticated users</a> to understand how the different Lambda@Edge functions for each of the authentication flow steps are implemented.

## Replace the s3 location with the React app files
In your AWS Console browser window, go to the S3 location as shown in the **CloudFront > Distributions > \<Id of the just created 
distribution\> > Origins tab > protected-origin**.
Upload to this S3 location, the React files from **\<location of the React app files\>/build** and the **static** subfolder as well.

Retest the Cloudfront distribution by refreshing the browser tab with the URL you had pasted to test the sample application. You will now be prompted to log in and after you log in, you will see the **my-weather** website. Click on Allow when this you see this pop-up:
![Know your location](../../images/blog/allow_know_ur_location.jpg)

You can click on the **Logout** button to be redirected to the login prompt.

## Extra points
Now that you have successfully implemented a static React website on S3 with Cognito authentication and cookies-based authorization using Lambda@Edge, you can try implementing social sign-in with your favorite identity provider: Facebook, Google, Amazon, or Apple following the steps described in <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-social-idp.html" target="blank">Adding social identity providers to a user pool - Amazon Cognito</a>.

Hope this post was useful to you. You can leave your comments in the **Enter comments here..** box at end of this page and hit **Submit Reply**.


A common paradigm in web development is the deployment of a static website to Amazon S3 instead of hosting in a web server. Web applications relying on server-side rendering frameworks like ASP&#46;NET, Spring, and Ruby on rails to name a few, are NOT candidates for deployment to S3. But even with these server-side rendering-based web applications, static content can still be stored in S3 to leverage CDN capabilities provided by AWS Cloudfront. Authentication is not available 'out of the box' for S3-based web applications.In this post, I will cover how AWS Cloudfront, Lambda@Edge, and Cognito can be used to implement authentication for a static React website published onto S3. We will cover authentication via Cognito

All posts

Developing an AWS Lambda container in Python and deploying via Terraform

Charting using python, pandas, matplotlib and a public dataset

Serverless implementation of authentication for a static React site deployed onto S3