Versions Compared

Version Old Version 16 New Version 17
Changes made by

Victor Wang

Victor Wang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
stages:
  - security-scan

datatheorem-sast-scan-branch-job:
  only:
    - main  # Trigger on default branch push, replace 'main' with the name of your default branch
  tags:
    - gitlab-runner-docker # Needs to be an executor compatible with the`image` feature
  stage: security-scan
  image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
  script:
    - export DT_SAST_API_KEY=$DT_SAST_API_KEY
    - export DT_SAST_REPOSITORY_NAME=$CI_PROJECT_PATH
    - export DT_SAST_REPOSITORY_PLATFORM="GITLAB_ON_PREM"
    - export DT_SAST_REPOSITORY_ID=$CI_PROJECT_ID
    - export DT_SAST_REPOSITORY_HTML_URL=$CI_PROJECT_URL
    - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME=$CI_DEFAULT_BRANCH
    - export DT_SAST_SCAN_HEAD_REF=$CI_COMMIT_REF_NAME
    - data_theorem_sast_analyzer scan ./

datatheorem-sast-scan-merge-request-job:
  only:
    - merge_requests
  tags:
    - gitlab-runner-docker # Needs to be an executor compatible with the`image` feature
  stage: security-scan
  image: us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest
  script:
    - export DT_SAST_API_KEY=$DT_SAST_API_KEY
    - export DT_SAST_REPOSITORY_NAME=$CI_PROJECT_PATH
    - export DT_SAST_REPOSITORY_PLATFORM="GITLAB_ON_PREM"
    - export DT_SAST_REPOSITORY_ID=$CI_PROJECT_ID
    - export DT_SAST_REPOSITORY_HTML_URL=$CI_PROJECT_URL
    - export DT_SAST_REPOSITORY_DEFAULT_BRANCH_NAME=$CI_DEFAULT_BRANCH
    - export DT_SAST_SCAN_TARGET_REF=$CI_MERGE_REQUEST_TARGET_BRANCH_NAME
    - data_theorem_sast_analyzer scan ./
Code Block
Troubleshooting
SSL Errors
If the scanner if failing because of SSL errors, it may be because you are running the scanner behind a proxy that is making SSL verification fail.
If this is the case, we recommend to do the following:
  • Ensure the machine is able to connect with api.securetheorem.com
    For troubleshooting run curl -v https://api.securetheorem.com/apis/devops/v1/sast_checks_download_link
    The output should look like this if you have valid SSL certificates
     Code Block
    *   Trying 34.149.167.254:443...
* TCP_NODELAY set
* Connected to api.securetheorem.com (34.149.167.254) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=api.securetheorem.com
*  start date: Jun 19 00:50:43 2024 GMT
*  expire date: Sep 17 01:41:53 2024 GMT
*  subjectAltName: host "api.securetheorem.com" matched cert's "api.securetheorem.com"
*  issuer: C=US; O=Google Trust Services; CN=WR3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56435e7a8340)
> GET /apis/devops/v1/sast_checks_download_link HTTP/2
> Host: api.securetheorem.com
> user-agent: curl/7.68.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 401 
HTTP/2 401 
< content-type: application/json
content-type: application/json
< vary: Accept
vary: Accept
< cache-control: no-store
cache-control: no-store
< x-cloud-trace-context: cd829e04213d00117752acf8010832f9;o=1
x-cloud-trace-context: cd829e04213d00117752acf8010832f9;o=1
< date: Wed, 10 Jul 2024 10:14:41 GMT
date: Wed, 10 Jul 2024 10:14:41 GMT
< server: Google Frontend
server: Google Frontend
< content-length: 29
content-length: 29
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
< via: 1.1 google
via: 1.1 google
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< content-security-policy: default-src 'none'; object-src 'none'; frame-ancestors 'none'; report-uri https://o1421491.ingest.sentry.io/api/6767243/security/?sentry_key=e958eee4d16443b4a6674cda8c008ca7
content-security-policy: default-src 'none'; object-src 'none'; frame-ancestors 'none'; report-uri https://o1421491.ingest.sentry.io/api/6767243/security/?sentry_key=e958eee4d16443b4a6674cda8c008ca7
< expires: 0
expires: 0
< x-frame-options: DENY
x-frame-options: DENY
< referrer-policy: origin
referrer-policy: origin
< pragma: no-cache
pragma: no-cache
< x-content-type-options: nosniff
x-content-type-options: nosniff
< strict-transport-security: max-age=31536000; includeSubDomains; preload
strict-transport-security: max-age=31536000; includeSubDomains; preload

< 
* Connection #0 to host api.securetheorem.com left intact
{"title": "401 Unauthorized"}(.venv) victor@vw-datatheorem:~/Workspace/scandal-server/on_prem_scanner$ curl -v https://api.securetheorem.com/apis/devops/v1/sast_checks_download_link
*   Trying 34.149.167.254:443...
* TCP_NODELAY set
* Connected to api.securetheorem.com (34.149.167.254) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=api.securetheorem.com
*  start date: Jun 19 00:50:43 2024 GMT
*  expire date: Sep 17 01:41:53 2024 GMT
*  subjectAltName: host "api.securetheorem.com" matched cert's "api.securetheorem.com"
*  issuer: C=US; O=Google Trust Services; CN=WR3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e36eefc340)
> GET /apis/devops/v1/sast_checks_download_link HTTP/2
> Host: api.securetheorem.com
> user-agent: curl/7.68.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 401 
< content-type: application/json
< vary: Accept
< cache-control: no-store
< x-cloud-trace-context: d93feb47ce573d06071f2b54ce0b87f2;o=1
< date: Wed, 10 Jul 2024 10:16:24 GMT
< server: Google Frontend
< content-length: 29
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
< via: 1.1 google
< x-xss-protection: 1; mode=block
< content-security-policy: default-src 'none'; object-src 'none'; frame-ancestors 'none'; report-uri https://o1421491.ingest.sentry.io/api/6767243/security/?sentry_key=e958eee4d16443b4a6674cda8c008ca7
< expires: 0
< x-frame-options: DENY
< referrer-policy: origin
< pragma: no-cache
< x-content-type-options: nosniff
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< 
* Connection #0 to host api.securetheorem.com left intact
{"title": "401 Unauthorized"}
In this example, we can see that the certificates at /etc/ssl/certs/ca-certificates.crt (location may vary on your machine)
Note: the unauthorized response is expected, we are just checking the SSL verification here
  • Since the Data Theorem SAST scanner is running inside a Docker container, which may you have the certificates that can make valid calls for your proxy, we can mount the host machine’s SSL certificates directory in the Docker container and add a parameter to let the scanner know where to look for SSL certificates.
    This can be done like this:
     Code Block
    export DT_SAST_API_KEY=<YOUR API KEY>
export DT_SAST_REPOSITORY_NAME="<my_org>/<my_repo>"
docker run -it \
 -e DT_SAST_API_KEY=$DT_SAST_API_KEY -e DT_SAST_REPOSITORY_NAME=$DT_SAST_REPOSITORY_NAME  -e DT_SAST_NO_FORWARD_MODE=true -e DT_SAST_PATH_TO_SSL_CERTS_FILE=/etc/ssl/certs/ \
 --mount type=bind,source="$(pwd)"/,target=/target \
 --mount type=bind,source="/etc/ssl/certs/",target=/etc/ssl/certs/ \
 us-central1-docker.pkg.dev/prod-scandal-us/datatheorem-sast/datatheorem-sast:latest data_theorem_sast_analyzer scan /target