Overview

The onboarding process, to setup our Logs Analysis service with an AWS ALB in the customer account, involves executing a CloudFormation Stack to be executed on the customer environment.

Requirements for CloudFormation Stack Execution

To execute this stack a role with enough permissions and the following trust relationship to the CloudFormation service is required to create the resources.

Create a role with the following trust policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Attach the following permissions policy to the role

The role should be able to have the following permissions :

Service	Actions

Service	Actions
IAM	`ListRolePolicies`, `ListAttachedRolePolicies`, `GetRolePolicy`, `GetRole`, `CreateRole`, `PutRolePolicy`.
Lambda	`GetFunction`, `GetRuntimeManagementConfig`, `GetFunctionCodeSigningConfig`, `CreateFunction`.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"iam:CreateRole",
				"iam:GetRole",
				"iam:GetRolePolicy",
				"iam:ListAttachedRolePolicies",
				"iam:ListRolePolicies",
				"iam:PutRolePolicy"
			],
			"Resource": "arn:aws:iam::${Account}:role/${RoleNameWithPath}"
		},
		{
			"Effect": "Allow",
			"Action": [
				"lambda:CreateFunction",
				"lambda:GetFunction",
				"lambda:GetFunctionCodeSigningConfig",
				"lambda:GetRuntimeManagementConfig"
			],
			"Resource": "arn:aws:lambda:${Region}:${Account}:function:${FunctionName}"
		}
	]
}

Resources created by the CloudFormation Stack

Logical ID	Type	Description

Logical ID	Type	Description
DTLogsAnalysisLambdaExecutionRole	IAM	Role to be assumed by the Lambda to execute the actions
DTLogsAnalysisLambdaFunction	Lambda	Lambda with Python code using boto3 to create resources necessary for logs to flow from the Loadbalancer to Data Theorem
InvokeLambdaFunction	Custom	Custom resource to invoke the Lambda with a Payload (i.e passing some params from the Stack such as ARN of the LB) to execute the logic

Lambda Execution Role Privileges (DTLogsAnalysisLambdaExecutionRole)

The execution role assigned to the Lambda, generated through CloudFormation, has the the below policy attached for necessary privileges:

Version: '2012-10-17'
Statement:
  - Effect: Allow
    Action:
      - 'logs:CreateLogGroup'
      - 'logs:CreateLogStream'
      - 'logs:PutLogEvents'
    Resource: 'arn:aws:logs:*:*:*'
  - Effect: Allow
    Action:
      - 'elasticloadbalancing:DescribeLoadBalancers'
      - 'elasticloadbalancing:ModifyLoadBalancerAttributes'
      - 'elasticloadbalancing:DescribeLoadBalancerAttributes'
      - 's3:*'
      - 'sns:CreateTopic'
      - 'sns:Subscribe'
    Resource: '*'

Resources created/modified by the Lambda (DTLogsAnalysisLambdaFunction)

The Lambda function, upon execution, may create or configure the following resources :

An S3 bucket for ALB raw log storage (if not already configured)
A bucket policy for read-only access by Data Theorem AWS account
An SNS topic and subscription for log event notifications to our Logs Analysis service

Example CloudFormation Template To Illustrate Resource Modifications

Do not use this template. The purpose of this template is to show the created SNS resources and the created/modified S3 bucket polices.

Parameters:
  S3LoggingBucketName:
    Type: String
  S3LoggingBucketPrefix:
    Type: String
  DataTheoremApiKey:
    Type: String
Resources:
  DataTheoremAWSALBLogSNSTopic:
    Type: AWS::SNS::Topic
  DataTheoremAWSALBLogSNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: DataTheoremSNSS3Policy
        Version: "2012-10-17"
        Statement:
          - Sid: datatheorem-logs-s3-bucket-to-sns-topic
            Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: SNS:Publish
            Resource: !Ref DataTheoremAWSALBLogSNSTopic
            Condition:
              StringEquals:
                - aws:SourceAccount: !Sub ${AWS::AccountId}
              ArnLike:
                - aws:SourceArn: !Sub arn:aws:s3:::${S3LoggingBucket}
      Topics:
        - !Ref DataTheoremAWSALBLogSNSTopic
  DataTheoremAWSALBLogSNSSubscription:
    Type: AWS::SNS::Subscription
    Properties:
      Protocol: https
      TopicArn: !Ref DataTheoremAWSALBLogSNSTopic
      Endpoint: !Sub https://token:${DataTheoremApiKey}@api-protect-api.securetheorem.com/logs/v1/ingest/aws_alb
      DeliveryPolicy:
        healthyRetryPolicy:
          numRetries: 3
          numNoDelayRetries: 0
          minDelayTarget: 20
          maxDelayTarget: 20
          numMinDelayRetries: 0
          numMaxDelayRetries: 0
          backoffFunction: linear
        requestPolicy:
          headerContentType: application/json
  S3LoggingBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub ${S3LoggingBucketName}
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: aws:kms
              KMSMasterKeyID: alias/aws/s3
      PublicAccessBlockConfiguration:
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      NotificationConfiguration:
        TopicConfigurations:
          - Event: s3:ObjectCreated:*
            Topic: !Ref DataTheoremAWSALBLogSNSTopic
  S3LoggingBucketBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref S3LoggingBucket
      PolicyDocument:
        Id: RequireEncryptionInTransit
        Version: "2012-10-17"
        Statement:
          - Sid: Allow AWS Log Puts
            Effect: Allow
            Principal:
              AWS: "arn:aws:iam::797873946194:root"
            Action: s3:PutObject
            Resource": !Sub arn:aws:s3:::${S3LoggingBucketName}/${S3LoggingBucketPrefix}/AWSLogs/${AWS::AccountId}/*"
          - Sid: GiveS3AccessToDT
            Effect: Allow
            Principal:
              AWS: arn:aws:iam::263348581401:user/prod-api-protect-api-logs-ingest
            Action: s3:GetObject
            Resource: !Sub arn:aws:s3:::${S3LoggingBucketName}/S3LoggingBucketPrefix}/*

Public Knowledge Base

Logs Analysis : AWS ALB Integration