Data Theorem allows you to run our source code SAST analyzer directly in your environment , and on your hardware. This gives you full control over the scanning infrastructure: the scanning machine could be your own on-prem hardware, or it could be a CI runner (for example, from Github, Bitbucket, Gitlab, or Azure DevOps).

Data Theorem’s on-prem scanner allows you to leverage Data Theorem’s SAST scanning without sending any off your source code off-site. The security scan results will still be uploaded to the Data Theorem portal. However, this approach comes with a couple of important limitations:

Data Theorem’s SAST analyzer won’t be able to post source code annotations directly in the Github / Bitbucket / Gitlab UI. Security scan results will only be consumable within the Data Theorem portal, or via our Security Scan Results API.

If you don’t need to abide these specific requirementsprefer not to be limited by the above, we recommend utilising utilizing our dedicated Github / Bitbucket / Gitlab integrations, which are built around Data Theorem’s Cloud infrastructure and provide the most polished developer experience (see onboarding instructions at DevSecOps > SAST Code Analysis).

Table of Contents

minLevel	1
maxLevel	2
outline	false
type	list
printable	false

Requirements

The machine running the scanner must have docker installed
The machine running the scanner must have internet access
We can recommend a base of 8G memory / 4 CPUs to run the scans, but note that scan time is proportional to the code base Here are our base spec recommendations for running the on-prem scanner

Repository Size	CPUs	RAM	Disk Size (SSD)
0-5 GB	4 CPUs	8 GB	16 GB
5-10 GB	8 CPUs	16 GB	32 GB
10-20 GB	16 CPUs	32 GB	64 GB

Note: Scan time is relative to the repository size so the specs that fit your needs may vary based on the size of your

...

repository.

Step 1: Generate a SAST Security Results API Key

...

DT_SAST_SCAN_HEAD_REF: git ref of the head to scan
DT_SAST_SCAN_TARGET_REF: git ref of the target to scan

[Optional] If you want the process to

set DT_SAST_FAIL_MODE=true if set, the process will return a non

...

-zero status when issues are found

...

. This can be used to make Data Theorem SAST a blocking step of your workflow

...

set DT_SAST_FAIL_MODE=true.
set DT_SAST_NO_FORWARD_MODE=true if you want to skip forwarding scan results/metadata to Data Theorem, note that this will mean that no scan results will be visible from the Data Theorem Portal

Local Scanning Example

...

set DT_SAST_INCLUDE_CODE_SNIPPETS=false if you want to hide code snippets from the printed scan result in the output (you will still see the issue location in the code from the file path and line)

Local Scanning example

The Data Theorem on-prem scanner can run from your local machine.

From the root of the git repository you wish to scan, run the following command

Code Block

docker run -it \
 -e DT_SAST_API_KEY=$DT_SAST_API_KEY \
 -e DT_SAST_REPOSITORY_NAME="<my_org>/<my_repo>" \
 -e DT_SAST_NO_FORWARD_MODE=true \
 --mount type=bind,source="$(pwd)"/,target=/target \
 us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast \
 data_theorem_sast_analyzer scan /target

Sample output:

Code Block

Scanning completed in 15.65 seconds
Scan results: 1 issues on commit=f719d004ef98254b46187c53ef1b3ed2f8643082
Total Issues: 1
Issues per types:
- First Party Code: 1
- SCA: 1
Issues per severity:
- High Severity: 1
- Medium Severity: 1
[
  {
    "issue_title": "Unauthenticated Route Found for Flask API",
    "issue_description": "The security of this code is compromised due to the presence of unauthenticated access to specific routes within the Flask API. This vulnerability poses a significant risk as it can potentially expose sensitive data or allow unauthorized actions to be performed. To mitigate this risk, it is crucial to implement robust authentication mechanisms that ensure only authorized users can access the protected routes.\n\nBy allowing unauthenticated access, the code fails to validate the identity of users before granting them access to certain routes. This lack of authentication opens the door for malicious actors to exploit the system and gain unauthorized access to sensitive information or perform actions that they should not be able to.\n\nTo address this issue, it is recommended to implement a secure authentication process that verifies the identity of users before granting them access to protected routes. This can be achieved through various methods such as username/password authentication, token-based authentication, or integration with third-party authentication providers.\n\nAdditionally, it is important to consider implementing other security measures such as encryption of sensitive data, input validation to prevent injection attacks, and proper error handling to avoid leaking sensitive information.\n\nBy implementing these security measures, the code can ensure that only authenticated and authorized users can access the protected routes, significantly reducing the risk of unauthorized access or data breaches. It is essential to prioritize security in the development process to safeguard sensitive data and protect the integrity of the system.",
    "issue_type": "FIRST_PARTY_CODE",
    "severity": "HIGH",
    "detected_in_file_path": "sample_code/bad_python.py",
    "detected_on_line": 7,
    "issue_code_snippet": "@app.route(\"/\")\ndef index():\n    cmd = request.args.get(\"cmd\", \"\")\n    exec(cmd)\n    return \"\""
  },
  {
    "issue_title": "jinja2 version 3.1.2 contains a known vulnerability (via PyPI dependency): Jinja vulnerable to HTML attribute injection when passing user input as keys to xmlattr filter",
    "issue_description": "Jinja vulnerable to HTML attribute injection when passing user input as keys to xmlattr filter",
    "issue_type": "SCA",
    "severity": "MEDIUM",
    "detected_in_file_path": "sample_code/requirements.txt",
    "detected_on_line": 1,
    "issue_code_snippet": "jinja2==3.1.2\n"
  }
]
Visit https://www.securetheorem.com/api/v2/security/sast for more details

Github Actions examples

Set the Data Theorem API Key as a secret variable

Go to your repository > Settings > Security > Secrets and variables > Actions> Secrets

Click on New repository secret and create a secret variable named DT_SAST_API_KEY with the value retrieved in step 1

Scans on pushes

Code Block

name: Data Theorem SAST

on:
  push:
    branches: [ "main" ]
  workflow_dispatch:

jobs:
  scan:
    runs-on: ubuntu-latest
    container:
      image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
      env:
        DT_SAST_API_KEY: ${{ secrets.DT_RESULTS_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - name: Start Data Theorem SAST Scan
        run: data_theorem_sast_analyzer scan --name=$GITHUB_REPOSITORY --repo-platform=GITHUB --repo-id=$GITHUB_REPOSITORY_ID --repo-html-url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY" --repo-default-branch-name=${{ github.event.repository.default_branch }}  --output-dir=$PWD
      # Optional step to make scan results available as a Github artifact
      - uses: actions/upload-artifact@v4
        with:
          name: dt-sast-scan-result
          path: ./scan-results-sarif.json

Scans on pull requests

...

Example with inputs to forward scan results to the [Data Theorem Portal](https://www.securetheorem.com/api/v2/security/sast )

Code Block

docker run -it \
 -e DT_SAST_API_KEY=$DT_SAST_API_KEY \
 -e DT_SAST_REPOSITORY_NAME="<my_org>/<my_repo>" \
 -e DT_SAST_REPOSITORY_PLATFORM=BITBUCKET \
 -e DT_SAST_REPOSITORY_ID={1e734a1b-8d0e-4787-a205-aba048c00a89} \
 -e DT_SAST_REPOSITORY_HTML_URL="https://bitbucket.org/<my_org>/<my_repo>" \
 -e DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME="main" \
 -e DT_SAST_SCANNED_BRANCH="main" \
 --mount type=bind,source="$(pwd)"/,target=/target \
 us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast \
 data_theorem_sast_analyzer scan /target

Sample output:

Code Block

Scanning completed in 15.65 seconds
Scan results: 1 issues on commit=f719d004ef98254b46187c53ef1b3ed2f8643082
Total Issues: 1
Issues per types:
- First Party Code: 1
- SCA: 1
Issues per severity:
- High Severity: 1
- Medium Severity: 1
[
  {
    "issue_title": "Unauthenticated Route Found for Flask API",
    "issue_description": "The security of this code is compromised due to the presence of unauthenticated access to specific routes within the Flask API. This vulnerability poses a significant risk as it can potentially expose sensitive data or allow unauthorized actions to be performed. To mitigate this risk, it is crucial to implement robust authentication mechanisms that ensure only authorized users can access the protected routes.\n\nBy allowing unauthenticated access, the code fails to validate the identity of users before granting them access to certain routes. This lack of authentication opens the door for malicious actors to exploit the system and gain unauthorized access to sensitive information or perform actions that they should not be able to.\n\nTo address this issue, it is recommended to implement a secure authentication process that verifies the identity of users before granting them access to protected routes. This can be achieved through various methods such as username/password authentication, token-based authentication, or integration with third-party authentication providers.\n\nAdditionally, it is important to consider implementing other security measures such as encryption of sensitive data, input validation to prevent injection attacks, and proper error handling to avoid leaking sensitive information.\n\nBy implementing these security measures, the code can ensure that only authenticated and authorized users can access the protected routes, significantly reducing the risk of unauthorized access or data breaches. It is essential to prioritize security in the development process to safeguard sensitive data and protect the integrity of the system.",
    "issue_type": "FIRST_PARTY_CODE",
    "severity": "HIGH",
    "detected_in_file_path": "sample_code/bad_python.py",
    "detected_on_line": 7,
    "issue_code_snippet": "@app.route(\"/\")\ndef index():\n    cmd = request.args.get(\"cmd\", \"\")\n    exec(cmd)\n    return \"\""
  },
  {
    "issue_title": "jinja2 version 3.1.2 contains a known vulnerability (via PyPI dependency): Jinja vulnerable to HTML attribute injection when passing user input as keys to xmlattr filter",
    "issue_description": "Jinja vulnerable to HTML attribute injection when passing user input as keys to xmlattr filter",
    "issue_type": "SCA",
    "severity": "MEDIUM",
    "detected_in_file_path": "sample_code/requirements.txt",
    "detected_on_line": 1,
    "issue_code_snippet": "jinja2==3.1.2\n"
  }
]
Visit https://www.securetheorem.com/api/v2/security/sast for more details

GitHub Actions example

Set the Data Theorem API Key as a secret variable

Go to your repository > Settings > Security > Secrets and variables > Actions> Secrets

Click on New Repository Secret and create a secret variable named DT_SAST_API_KEY with the value retrieved in Step 1

Scans on pushes

Code Block

name: Data Theorem SAST

# Controls when the workflow will run, adapt to your own needs
on:
  # Triggers the workflow on push or pull request events but only for the "main" branch
  # Adapt triggers to your own needs
  push:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  scan:
    continue-on-error: true
    timeout-minutes: 30
    runs-on: ubuntu-latest
    container:
      image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
      env:
        DT_SAST_API_KEY: ${{ secrets.DT_SAST_API_KEY }}
        DT_SAST_REPOSITORY_NAME: ${{ github.event.repository.full_name }}
        DT_SAST_REPOSITORY_PLATFORM: GITHUB
        DT_SAST_APIREPOSITORY_KEYID: ${{ secrets.DT_SAST_API_KEYgithub.event.repository.id }}
        DT_SAST_REPOSITORY_HTML_NAMEURL: ${{ github.event.repository.fullhtml_nameurl }}
        DT_SAST_REPOSITORY_PLATFORM: GITHUB
        DT_SAST_REPOSITORY_IDDEFAULT_BRANCH_NAME: ${{ github.event.repository.iddefault_branch }}
        DT_SAST_REPOSITORYOUTPUT_HTML_URLDIR: ${{ github.event.repository.html_url }}./
    steps:
    DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME: ${{ github.event.repository.default_branch }}
 - uses: actions/checkout@v4
       DT_SAST_SCAN_HEAD_REF: "refs/remotes/origin/${{ github.head_ref }}"
- name: Start Data Theorem SAST Scan
       DT_SAST_SCAN_TARGET_REF: "refs/remotes/origin/${{ github.base_ref }}"
        DT_SAST_FAIL_MODE: true
 run: data_theorem_sast_analyzer scan ./
   steps:       - uses: actions/checkout@v4upload-artifact@v4
        with:
          fetch-depthname: 0  # IMPORTANT: Needed because by default, actions/checkout@v4 doesn't load the full git history/refs
      - name: Startdt-sast-scan-result
          path: ./scan-results-sarif.json

Scans on pull requests

Code Block
name: Data Theorem SAST Scan # Controls when the workflow will run, adapt to run: data_theorem_sast_analyzer scan ./b

Bitbucket pipeline example

Set the Data Theorem API Key as a secret variable

Go to your repository > Repository Settings > Repository Variables

Add a variable named DT_SAST_API_KEY with the value retrieved in step 1 and make sure the Secured option is checked

Code Block

image: atlassian/default-image:3

pipelines:
  branches:
    main:
      - step:
          name: 'Data Theorem SAST'
          image: us-your own needs
on:
  # Triggers the workflow on push or pull request events but only for the "main" branch
  # Adapt triggers to your own needs
  pull_request

jobs:
  scan:
    continue-on-error: true
    timeout-minutes: 30
    runs-on: ubuntu-latest
    container:
      image: us-central1-docker.pkg.dev/devprod-scandal-us/datatheorem-sast-dev/datatheorem-sast-dev:latest
      env: 
  script:      DT_SAST_API_KEY: ${{ secrets.DT_SAST_API_KEY }}
   - echo "Your security scan goes here..." DT_SAST_REPOSITORY_NAME: ${{ github.event.repository.full_name }}
        DT_SAST_REPOSITORY_PLATFORM: GITHUB
      - export DT_SAST_API_KEY=$DT_SAST_API_KEY
  REPOSITORY_ID: ${{ github.event.repository.id }}
         - export DT_SAST_REPOSITORY_NAME=$BITBUCKET_REPO_FULL_NAME
    _HTML_URL: ${{ github.event.repository.html_url }}
       - export DT_SAST_REPOSITORY_PLATFORM=BITBUCKET
  _DEFAULT_BRANCH_NAME: ${{ github.event.repository.default_branch }}
         - export DT_SAST_REPOSITORYSCAN_ID=$BITBUCKET_REPO_UUID
   HEAD_REF: "refs/remotes/origin/${{ github.head_ref }}"
        - export DT_SAST_REPOSITORYSCAN_HTML_URL=$BITBUCKET_GIT_HTTP_ORIGIN
  TARGET_REF: "refs/remotes/origin/${{ github.base_ref }}"
         - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME="main"FAIL_MODE: true
    steps:
      - uses: actions/checkout@v4
- data_theorem_sast_analyzer scan ./     pull-requestswith:
    "**":       fetch- stepdepth: 0  # IMPORTANT: Needed because by default,   name: 'Data Theorem SAST'
  actions/checkout@v4 doesn't load the full git history/refs
      - image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
   name: Start Data Theorem SAST Scan
      script:             - echo "Your security scan goes here..."
            - export DT_SAST_API_KEY=$DT_SAST_API_KEY
            - export DT_SAST_REPOSITORY_NAME=$BITBUCKET_REPO_FULL_NAME
     run: data_theorem_sast_analyzer scan ./

Bitbucket pipeline example

Set the Data Theorem API Key as a secret variable

Go to your repository > Repository Settings > Repository Variables

Add a variable named DT_SAST_API_KEY with the value retrieved in step 1 and make sure the Secured option is checked

Code Block

image: atlassian/default-image:3

pipelines:
  # Triggers the pipeline on push events but only for the "main" branch
  # Adapt triggers to your own needs
  branches:
    main:
      - export DT_SAST_REPOSITORY_PLATFORM=BITBUCKETstep:
          name: 'Data Theorem SAST'
          image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem- export DT_SAST_REPOSITORY_ID=$BITBUCKET_REPO_UUIDsast/datatheorem-sast:latest
          script:
            - export echo "Your security scan goes here..."
            - export DT_SAST_API_KEY=$DT_SAST_API_KEY
            - export DT_SAST_REPOSITORY_HTML_URL=$BITBUCKET_GIT_HTTP_ORIGIN
            - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME="main"
            - export DT_SAST_SCAN_HEAD_REF=$BITBUCKET_COMMIT
            - export DT_SAST_SCAN_TARGET_REF=$BITBUCKET_PR_DESTINATION_COMMIT
            - export DT_SAST_FAIL_MODE=true
            - data_theorem_sast_analyzer scan ./NAME=$BITBUCKET_REPO_FULL_NAME
            - export DT_SAST_REPOSITORY_PLATFORM=BITBUCKET
            - export DT_SAST_REPOSITORY_ID=$BITBUCKET_REPO_UUID
            - export DT_SAST_REPOSITORY_HTML_URL=$BITBUCKET_GIT_HTTP_ORIGIN
            - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME="main"
            - data_theorem_sast_analyzer scan ./


  pull-requests:
    # Triggers the pipeline on pull request events
    # Adapt triggers to your own needs
    "**":
      - step:
          name: 'Data Theorem SAST'
          image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
          script:
            - echo "Your security scan goes here..."
            - export DT_SAST_API_KEY=$DT_SAST_API_KEY
            - export DT_SAST_REPOSITORY_NAME=$BITBUCKET_REPO_FULL_NAME
            - export DT_SAST_REPOSITORY_PLATFORM=BITBUCKET
            - export DT_SAST_REPOSITORY_ID=$BITBUCKET_REPO_UUID
            - export DT_SAST_REPOSITORY_HTML_URL=$BITBUCKET_GIT_HTTP_ORIGIN
            - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME="main"
            - export DT_SAST_SCAN_HEAD_REF=$BITBUCKET_COMMIT
            - export DT_SAST_SCAN_TARGET_REF=$BITBUCKET_PR_DESTINATION_COMMIT
            - export DT_SAST_FAIL_MODE=true
            - data_theorem_sast_analyzer scan ./

Gitlab pipeline example

Set the Data Theorem API Key as a secret variable

Go to your project > Settings > CI/CD > Variables

Add a variable named DT_SAST_API_KEY with the value retrieved in step 1 and make sure the Masked option is checked

Note: the Gitlab pipeline must run the Data Theorem SAST step on an executor that supports the image feature.
See https://docs.gitlab.com/runner/executors/#compatibility-chart for more information on compatible executors

Code Block

stages:
  - security-scan

datatheorem-sast-scan-branch-job:
  only:
    - main  # Trigger on default branch push, replace 'main' with the name of your default branch
  tags:
    - gitlab-runner-docker # Needs to be an executor compatible with the`image` feature
  stage: security-scan
  image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
  script:
    - export DT_SAST_API_KEY=$DT_SAST_API_KEY
    - export DT_SAST_REPOSITORY_NAME=$CI_PROJECT_PATH
    - export DT_SAST_REPOSITORY_PLATFORM="GITLAB_ON_PREM"
    - export DT_SAST_REPOSITORY_ID=$CI_PROJECT_ID
    - export DT_SAST_REPOSITORY_HTML_URL=$CI_PROJECT_URL
    - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME=$CI_DEFAULT_BRANCH
    - export DT_SAST_SCAN_HEAD_REF=$CI_COMMIT_REF_NAME
    - data_theorem_sast_analyzer scan ./

datatheorem-sast-scan-merge-request-job:
  only:
    - merge_requests
  tags:
    - gitlab-runner-docker # Needs to be an executor compatible with the`image` feature
  stage: security-scan
  image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
  script:
    - export DT_SAST_API_KEY=$DT_SAST_API_KEY
    - export DT_SAST_REPOSITORY_NAME=$CI_PROJECT_PATH
    - export DT_SAST_REPOSITORY_PLATFORM="GITLAB_ON_PREM"
    - export DT_SAST_REPOSITORY_ID=$CI_PROJECT_ID
    - export DT_SAST_REPOSITORY_HTML_URL=$CI_PROJECT_URL
    - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME=$CI_DEFAULT_BRANCH
    - export DT_SAST_SCAN_TARGET_REF=$CI_MERGE_REQUEST_TARGET_BRANCH_NAME
    - data_theorem_sast_analyzer scan ./

Azure DevOps Pipeline Example

Create a new Azure DevOps Pipeline

Add a variable named DT_SAST_API_KEY with the value retrieved in step 1 and make sure the Keep this value secret option is checked. (See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-secret-variables?view=azure-devops&tabs=yaml%2Cbash )

The Azure Pipeline definition should look like this:

Code Block

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    docker run \
    -e DT_SAST_API_KEY='$(DT_SAST_API_KEY)' \
    -e DT_SAST_REPOSITORY_NAME=$(Build.Repository.Name) \
    -e DT_SAST_REPOSITORY_PLATFORM=AZURE_DEVOPS \
    -e DT_SAST_REPOSITORY_ID=$(Build.Repository.ID) \
    -e DT_SAST_REPOSITORY_HTML_URL=$(Build.Repository.Uri) \
    -e DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME="main" \
    -e DT_SAST_SCANNED_BRANCH=$(Build.SourceBranchName) \
    -e DT_SAST_SCAN_HEAD_REF="HEAD" \
    --mount type=bind,source="$(pwd)"/,target=/target \
    us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest \
    data_theorem_sast_analyzer scan /target
  displayName: 'Data Theorem On-Prem SAST'

Troubleshooting

SSL Errors

If the scanner if failing because of SSL errors, it may be because you are running the scanner behind a proxy that is making SSL verification fail.

If this is the case, we recommend to do the following:

You can build a custom Docker images that embeds your own valid SSL certificates

Make sure you have valid certificates that are able to call api.securetheorem.com from the machine that is running the Data Theorem On-Prem Scanner

The Dockerfile would look like this

Code Block

FROM us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest

# Copy SSL certificates to add
COPY --from=<YOUR_SOURCE> <PATH_TO_YOUR_SSL_CERTS> /usr/share/ca-certificates/custom

# Add certificates to /etc/ca-certificates.conf
RUN for crt in /usr/share/ca-certificates/custom/*.crt; do \
        echo "Adding $crt" && echo "custom/$(basename "$crt")" >> /etc/ca-certificates.conf; \
    done

# Update bundled CA certificates at /etc/ssl/certs/ca-certificates.crt
RUN update-ca-certificates
ENV DT_SAST_PATH_TO_SSL_CERTS_FILE=/etc/ssl/certs/ca-certificates.crt

If this is not working, please contact support@datatheorem.com for help

Version	Old Version 2	New Version Current
Changes made by	Victor Wang	Victor Wang
Saved on	Jan 30, 2024	Sept 30, 2024

Content Comparison

Versions Compared

Key

Requirements

Step 1: Generate a SAST Security Results API Key

Local Scanning Example

Local Scanning example

Sample output:

Github Actions examples

Scans on pushes

Scans on pull requests

Sample output:

GitHub Actions example

Scans on pushes

Scans on pull requests

Bitbucket pipeline example

Bitbucket pipeline example

Gitlab pipeline example

Azure DevOps Pipeline Example

Troubleshooting

SSL Errors