Page Comparison

...

This can for example be used as part of a CI/CD pipeline in order to verify that the terraform file won’t create any resource with some urgent policy violation. This way customers can prevent the deployment of a cloud resource can be prevented to be deployed in production.

The feature was deployed on 2021/05/26. At that time we are only checking for urgent policy violations in S3 buckets. Alban (Unlicensed) Thomas Sileo and Marc Tranzer (Unlicensed) will have to prioritize what potential terraform issues should be checked first.

See https://www.terraform.io/docs/language/index.html for an explanation on terraform Terraform files.

Running a terraform scan

Data Theorem’s API can be used to run a terraform scan against a specific terraform configuration file

Retrieve an API Key that has the permission “API Security Results API” enabled; API keys are available in the Data Theorem portal at

...

API Key.

A terraform file scan can then be run using the following CURL command:

Code Block

curl -X POST 'https://api.securetheorem.com/apis/devops/v1/iac_scans' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: APIKey ABCACBA=' \
--form 'file=@"terraform_example_configuration:file.tf"' \
--form 'scan_type="TERRAFORM"'

3. Check the output for any issues in the files.

...

Code Block

export terraform_issues_count=$(curl -X POST 'https://api.securetheorem.com/apis/devops/v1/iac_scans' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: APIKey ABCACBA=' \
--form 'file=@"terraform_example_configuration:file.tf"' \
--form 'scan_type="TERRAFORM"'| jq -r ".issues_count")

// Then deploy your file only if terraform_issues_count is equal to 0

Use Public API to see all recent IAC scans

TODO: Alban (Unlicensed) Spencer James (Unlicensed) We should design a portal page to see recent IAC Scans results

Code Block

curl -X POST 'https:///prod-horizon.appspot.com//public/v2/iac_scans' \
--header 'Authorization: Session ABCACBA=' \

response:
class IacScansResponse(BasePaginatedResponse):
    iac_scans: List[IacScanSummaryField]
  
class IacScanSummaryField:
    id: UUID
    date_created: datetime
    status: IacScanStatusEnum
    scan_type: IacScanTypeEnum
    scanned_files_name: List[str]
    issue_count: intif [ $TERRAFORM_ISSUES_COUNT == 0 ]; then
  echo "Deploying file: terraform_example_configuration"
else
  exit 1

Integrating into a CI/CD pipeline

GitHub Actions

For terraform files hosted on GitHub, a GitHub Actions workflow can be configured. The workflow will perform terraform scans every time the repository is tagged with a new version.

To setup this workflow:

Create a new secret containing your API key (called DATATHEOREM_API_RESULT_API_KEY in the below example).
Create a new workflow by creating a file at .github/workflows/datatheorem.yaml with the following content:

Code Block

name: Data Theorem Terraform Scans
on:
  push:
    tags:
       - '*'
jobs:
  datatheorem-terraform-scan:
    runs-on: ubuntu-latest
    steps:
      - env:
          WEB_APPLICATION_ID: 14df3c48-ff3a-4324-b4a6-13d2904aa785
          DATATHEOREM_WEB_SECURE_SCANS_API_KEY: ${{ secrets.DATATHEOREM_API_RESULT_API_KEY }}
        run: |
          curl -X POST 'https://api.securetheorem.com/apis/devops/v1/iac_scans' \
              --header 'Content-Type: multipart/form-data' \
              --header 'Authorization: APIKey $DATATHEOREM_API_RESULT_API_KEY' \
              --form 'file=@"terraform_example_configuration:file.tf"' \
              --form 'scan_type="TERRAFORM"'

Bitbucket Pipelines

For terraform files hosted on Bitbucket, a similar workflow can be configured in Bitbucket Pipelines:

Create a new secure “Repository variables” in Pipelines configuration (from the repository setting).
Create a Pipeline by creating a .bitbucket-pipelines.yml within your repository with the following content:

Code Block

breakoutMode	wide

pipelines:
  tags:
    '*':
      - step:
        script:
            - apt-get update
            - apt-get install -y jq
            - export FILEPATH="$BITBUCKET_CLONE_DIR/my_terraform_file.tf"
            - if [ -f "$FILEPATH" ]; then echo "File  script:exists" ; else exit 1; fi
            - |
                export TERRAFORM_DATA_THEOREM_RESPONSE=$(curl -X POST 'https://api.securetheorem.com/apis/devops/v1/iac_scans' \
                                --header 'Content-Type: multipart/form-data' \
                                --header '"Authorization: APIKey $DATATHEOREM_APITERRAFORM_RESULT_API_KEY'" \
                                --form 'file=@"terraform_example_configuration:file.tf'"$FILEPATH"'"' \
                                --form 'scan_type="TERRAFORM"')
                export TERRAFORM_ISSUES_COUNT=$(echo $TERRAFORM_DATA_THEOREM_RESPONSE | tr '\r\n' ' '  | jq -r ".issues_count")
                export MARKDOWN_RESULT=$(echo $TERRAFORM_DATA_THEOREM_RESPONSE | tr '\r\n' ' ' | jq -r ".result_as_markdown")
                if [ $TERRAFORM_ISSUES_COUNT == 0 ]; then
                  echo "Deploying file: terraform_example_configuration"
                else
                  echo "Terraform file contains $TERRAFORM_ISSUES_COUNT issues, abort deployment..."
                  echo "Terraform file issues report: $MARKDOWN_RESULT"
                  exit 1
                fi

Versions Compared

Old Version 2

New Version Current

Key

Running a terraform scan

Use Public API to see all recent IAC scans

Integrating into a CI/CD pipeline

GitHub Actions

Bitbucket Pipelines