In this blog post, we will take a step-by-step look at how we can create a project on AWS from our local computer and how we can link this project we created to a GitHub repository. During this process, we will closely examine the use of AWS SAM (Serverless Application Model) and AWS CodeBuild services. This guide aims to provide practical information to accelerate your application development process on AWS and to simplify GitHub integration.

In today's world, developing cloud-based applications, particularly using serverless architecture, has become key to providing fast, cost-effective, and scalable solutions. AWS SAM stands out as a framework that allows you to easily define and deploy your serverless applications. AWS CodeBuild accelerates your development process by automating continuous integration and delivery processes. In this article, we will thoroughly explore how to effectively use these two powerful tools, how to transfer your project from local to AWS, and finally, how to integrate this project with GitHub. Thus, we aim to create a useful resource for both newcomers to application development on AWS and for more experienced developers.

Before starting the process of creating a project on AWS and linking it to a GitHub repository, as explained on our blog page, there are some basic requirements that need to be met. To facilitate these procedures for you, below we simply present the necessary tools and conditions in a list format:

• Creating an AWS account
• Creating User in AWS
• AWS CLI (Command Line Interface) Installation
• AWS SAM CLI Installation
• Creating a GitHub Account and GitHub Repository
• Your permissions required to access AWS services
• Python (version < 3.11)

Creating a Project on AWS with AWS SAM from Your Local Computer

In this section, we will examine step by step how you can effectively and efficiently create a project from your local computer to AWS using the AWS Serverless Application Model (SAM). AWS SAM is a powerful tool that allows you to define and rapidly deploy your serverless applications. This process makes it possible to develop, test, and easily publish your application on AWS.

As the first step of our project, it is necessary to create the 'template.yaml' file, which is the cornerstone of AWS SAM. This file contains the configuration of your project for SAM to understand and interpret. Here, you can define the AWS resources, functions, permissions, and other settings of your application. The proper configuration of the 'template.yaml' file determines how your project will behave on AWS and which resources it will use. Below, we will show you the structure of an example 'template.yaml' file and explain the details of this file step by step.

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: Örnek SAM Uygulaması

Resources:
  SunumApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      Name: SunumApi


  LambdaUpload:
    Type: 'AWS::Serverless::Function'
    Properties:
      FunctionName: Sunum-upload
      Handler: lambda_function.lambda_handler
      Runtime: python3.11
      CodeUri: ./LambdaUpload/
      MemorySize: 128
      Timeout: 30
      Layers:
        - arn:aws:lambda:eu-central-1:************:layer:samlast:1
      Role: arn:aws:iam::************:role/cagdas-lambda
      Events:
        ApiEvent2:
          Type: Api
          Properties:
            Path: /Sunum-upload
            Method: any
            RestApiId: !Ref SunumApi


  LambdaWebhook:
    Type: 'AWS::Serverless::Function'
    Properties:
      FunctionName: Sunum-webhook-handler
      Handler: lambda_function.lambda_handler
      Runtime: python3.11
      CodeUri: ./LambdaWebhook/
      MemorySize: 128
      Timeout: 30
      Layers:
        - arn:aws:lambda:eu-central-1:************:layer:samlast:1
      Role: arn:aws:iam::************:role/cagdas-lambda
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /Sunum-webhook-handler
            Method: any
            RestApiId: !Ref SunumApi

AWSTemplateFormatVersion ve Transform:

• AWSTemplateFormatVersion: '2010-09-09'This line specifies the version of the AWS CloudFormation template format for the file.
• Transform: 'AWS::Serverless-2016-10-31'This is a special CloudFormation directive that specifies that AWS SAM will be used

Description:

• Sample SAM ApplicationThis contains a general description of your template.

Resources – SunumApi:

• Resources: The section where AWS resources are defined.
• SunumApi: API Gateway definition. This section specifies how to configure API Gateway
• Type: AWS::Serverless::ApiType of source
• PropertiesFeatures of the source
  • StageName: Prod:Phase of the API
  • Name: SunumApi: Name of the API

Resources – Lambda Fonksiyonları:

• LambdaUpload and LambdaWebhook: Two Lambda function definitions.
• Type: 'AWS::Serverless::Function': The type of the function.
• Properties: Properties of the function
  • FunctionName: Name of the function.
  • Handler: Entry point of the lambda function.
  • Runtime: Runtime of the lambda function (e.g. python3.11).
  • CodeUri: Where the function code is located.
  • MemorySize: The maximum amount of memory the function will use.
  • Timeout: Maximum running time (seconds) of the function.
  • Layers: Lambda layers.
  • Role: ARN of the IAM role that the function will use.
• Events: Events that will trigger the function
  • Type: Event type (for example, Api).
  • Properties: Features of the event.

With this template.yaml file, we have created two key components in a serverless architecture using AWS SAM: two Lambda functions named LambdaUpload and LambdaWebhook, and an API Gateway named SunumApi to trigger these functions. The LambdaUpload function responds to requests coming through the /Sunum-upload path, while the LambdaWebhook function is designed to process requests on the /Sunum-webhook-handler path. Both Lambda functions are configured to run in Python 3.11 and are set up to use specific AWS resources. By bringing together these functions with SunumApi, we enabled our application to respond dynamically and effectively to requests from the outside world.

The first step in running our project on AWS is to compile our project using the 'sam build' command in our local development environment. This command compiles our application according to the structure defined in the template.yaml file and creates a package for uploading to AWS. During this compilation process, SAM automatically installs all necessary dependencies and prepares a runtime environment for your Lambda functions. Once the compilation process is complete, we are ready to deploy our project to AWS.

The distribution process begins with the command sam deploy --guided. This command provides an interactive guide to deploy your project on AWS and create the necessary resources. When you run the command, SAM will ask you for various configuration information (such as the project's name, deployment region, permissions for resources, etc.). This information determines how your project will be configured on AWS and which resources will be used. After completing the configuration steps, SAM automatically creates the resources defined in your template.yaml file and runs your application in the specified AWS environment. This process includes all the necessary steps to ensure your project runs smoothly on AWS.

NOTE: After successfully deploying the project with AWS SAM, you can monitor the created resources and configurations in the AWS CloudFormation console. You can find the project in the CloudFormation console under the project name you specified during deployment.

Integration with GitHub Repo: Automatically Configure and Deploy Project with AWS CodeBuild

In this section, we will step by step cover how to connect the project we created on AWS to a GitHub repository using the AWS CodeBuild service. AWS CodeBuild is a service that automatically compiles, tests, and deploys your code, and with these features, it facilitates continuous integration and continuous deployment (CI/CD) processes. Integration with GitHub ensures that any changes to your project are automatically reflected on AWS. This integration makes your development process more efficient, allowing for quick and seamless transfer of code changes to the live environment.

The first step in matching our project with a GitHub repository is to create a 'Build Project' in the AWS CodeBuild console. To initiate this process, we click on the 'Create build project' button located in the console. Here, we define the name of our project and other basic settings. It's critical to select 'source' as GitHub, because this step will integrate our project with the GitHub repository. After selecting 'source', we need to choose the repository we want to integrate. By typing 'main' into the 'Source version – optional' box, we can ensure that only changes in the 'main' branch will affect our project. This allows the project to be continuously updated from a specific branch, and every commit made to this branch will be automatically compiled. in Figure-1 demonstrates how to select GitHub as the CI/CD option while creating a Build project.

As we continue with our Build Project settings, we reach the 'Primary source webhook events' section. Here, we need to select the 'Rebuild every time a code change is pushed to this repository' option. This ensures that our project is automatically rebuilt whenever any code changes are made in our GitHub repository. Additionally, by selecting the 'PUSH' and 'PULL_REQUEST_MERGED' options under 'Event type', we indicate our desire to trigger these types of events. These settings allow our code to be automatically updated during push and merge operations in the main branch on GitHub. In this way, we further automate our continuous integration process and ensure that our project is always up-to-date. in Figure-2 illustrates how and in which situations the project will receive code updates using a Webhook.

As we continue to set up our Build Project settings, we reach the 'Environment' section. In this part, we configure the build environment of our project. First, under the 'Compute' option, we choose the 'Lambda' option indicating that our project will be run on AWS Lambda. Then, in the 'Runtime' section, we select the 'python' option which specifies the programming language in which our project is written. Additionally, in the 'Image' option, we choose the 'x86_64' option which specifies the type of virtual machine to be used for the build process. Finally, in the 'Service role' section, we mark the 'New service role' option allowing AWS CodeBuild to manage resources on behalf of our project. These steps include the necessary configurations for our project to be properly compiled and run in the AWS CodeBuild environment. The requirements that need to be met when creating a CodeBuild service are shown in Figure-3.

After completing the configuration of the Environment section, we move on to the last step. At this stage, we create the 'Build Project' in WS CodeBuild by pressing the 'Create build project' button.

After successfully creating the 'Build Project' service and integrating it with the GitHub repository, the next step is to add the information obtained from our AWS services to the role assigned to this project. This particularly includes the ARN (Amazon Resource Name) information of services like AWS Lambda. We need to make updates in the permissions of the IAM (Identity and Access Management) role assigned to the 'Build Project' service, to allow access to the resources of these services.

For the updates we are going to make, we first need to go to the AWS CodeBuild console and click on the 'Build Project' we created earlier. This step will take us to the details of the project. Once we reach the details of our project, we should open the 'Build details' tab in order to make the necessary updates. Figure-4 shows the ‘Build details’ button to be used to access the service role that will be updated.

After switching to the 'Build details' tab, we need to access the 'Service role' option located in the 'Environment' section. If we need to add new permissions to this role, we can perform this addition at this stage. Particularly, focusing on permissions starting with 'CodeBuildBasePolicy', we should enter these permissions and use the 'Edit' option. By doing this, we can add the ARN (Amazon Resource Name) information of the AWS services we want to grant access permission to our project in the 'Resource' section. This process allows CodeBuild to access the specified AWS services and perform the necessary operations on these services. Figure-5 illustrates the area that needs to be clicked to access the service role that requires updating.

To modify the permission starting with 'CodeBuildBasePolicy', first, we need to click on this permission to open the relevant page. Once the page is opened, we need to click the 'Edit' button to edit the permission details. This step is necessary to make additions or changes to the relevant policy or modify existing permissions.

After clicking the 'Edit' button, an important modification needs to be made in the JSON file that appears. In this file, we need to locate the 'Resource' section and add the ARN (Amazon Resource Name) information of the AWS services and functions we want to add in sequence to this section. This process ensures that the CodeBuild project has access permissions to the AWS resources we have specified and is necessary for it to perform the required operations on these resources. Figure-7 shows the ARN information of the Lambda services to be updated that have been added to the Permissions section.

We have now successfully completed the AWS-side settings of our integration with GitHub using AWS CodeBuild. From this point forward, we need to focus on our project's GitHub repository. AWS CodeBuild requires a configuration file named buildspec.yml to fetch and compile code from the GitHub repository and run tests. This file tells AWS CodeBuild how to compile your project and how to execute the tests. It includes information such as which commands will be run, which files will be considered, and what stages will be followed during the compilation. The buildspec.yml file located in the root directory of our project serves as a guide for AWS CodeBuild on what to do with the code fetched from GitHub. The presence and proper configuration of this file ensure the smooth compilation and deployment of our project by AWS CodeBuild.

Considering the importance and functionality of the buildspec.yml file mentioned above, we will create an example buildspec.yml file. In this example, it will include basic commands and configurations that will be used by AWS CodeBuild. Our example will demonstrate how to configure the build process, which commands will be executed, and which files will be considered during the build. Through this example file, we will step by step examine the structure of the buildspec.yml file and how each section contributes to the build and deployment process of your project. This way, you will have a clear guide on how to prepare the buildspec.yml file for your own project.

version: 0.2
phases:
  install:
    runtime-versions:
      python: 3.11
    commands:
      - echo "Python environment setup"

  build:
    commands:
      - echo "Zipping deployment package..."
      - cd LambdaUpload
      - zip -r9 LambdaUpload.zip lambda_function.py
      - cd ..

      - cd LambdaWebhook
      - zip -r9 LambdaWebhook.zip lambda_function.py
      - cd ..

      - echo "Deployment package ready"

  post_build:
    commands:
      - echo "Updating lambda Function..."

      - aws lambda update-function-code --function-name App-upload --zip-file fileb://LambdaUpload/LambdaUpload.zip
      - aws lambda update-function-code --function-name App-webhook-handler --zip-file fileb://LambdaWebhook/LambdaWebhook.zip


      - echo "Lambda function configuration updated"
      - echo "DONE!!"

The buildspec.yml file featured on our blog page is a configuration file that specifies how our project will be compiled and how the tests will be conducted by AWS CodeBuild. Each section of this file manages different stages of the compilation process. For example:

• version: This section indicates which version of the buildspec file we are using.
• phases: It describes the different stages of the compilation process.
  • install: Used to install necessary tools and dependencies.
  • pre_build: Used for commands that need to be performed before compilation, for example configuration settings or dependency checking.
  • build: The part where the actual compilation process is carried out. Compiling source code, running tests, etc. operations are defined at this stage
  • post_build: Actions that need to be performed after the compilation process is completed, for example, uploading the compilation outputs to a specific storage area.

The buildspec.yml file example mentioned above is used to update the Lambda functions, App-upload, and App-webhook-handler, located in the LambdaUpload and LambdaWebhook folders in the GitHub repository, using the AWS Lambda service. This configuration enables AWS CodeBuild to automatically detect changes in GitHub and reflect these changes to the relevant functions in the AWS Lambda service. For example, a change made in the lambda_function.py file in the LambdaUpload folder updates the App-upload Lambda function in AWS according to the definitions in the buildspec.yml file. Similarly, changes in the LambdaWebhook folder are also used to update the App-webhook-handler Lambda function.

With the completion of these processes, a full integration is achieved between the GitHub repository and AWS services. Now, any changes made in the GitHub repository will automatically update the relevant functions and code in AWS services. Thanks to this integration, our development process becomes more efficient and seamless; code changes are deployed to the production environment quickly and effectively.

Conclusion

In this blog post, we have thoroughly discussed the steps to successfully transfer a project from your local computer to AWS and integrate it with GitHub. Learning to use key services like AWS SAM and AWS CodeBuild will simplify and accelerate your serverless application development process. We hope this guide offers practical and useful information for both beginners in cloud-based application development and more experienced developers.

Çağdaş Yılmaz
cagdas.yilmaz@golive.com.tr