Source Scanner Customer Portal - Salesforce.com

Force.com Security Source Scanner Help

Introduction

The Force.com Security Source Scanner is a cloud based source code analysis tool built directly into our Force.com offering.  Salesforce has partnered with Checkmarx to provide free use of their Checkmarx Static Analysis Suite (CxSAST) as a high value addition that will help to enable our community to build trusted applications.

Requirements

In order for the scan to be successfully processed, the following must be true:

  • Each company, regardless of how many organizations they have, must
    • wait for the results of all pending jobs to complete before submitting another job.
    • comply with our license agreement with Checkmarx: if the scan is for an AppExchange Security Review, then no more than 3 scans per review. If the scan is not for the AppExchange, then no more than 360,000 lines of code scanned in any trailing 12-month period
  • The company organization being scanned must
    • contain less than 2 million lines of code (excluding static resources and packages which are not scanned)
    • have metadata API enabled
    • not use IP access controls that prevent access from Salesforce IP ranges
  • The username submitted must correspond to a user that
    • has "Author Apex" permission
    • has an email address that the submitter can access (If you are scanning a sandbox, you must fix the email after every sandbox refresh.)

Only unpackaged code is scanned. Source code within managed or unmanaged packages is not scanned to avoid inadvertently scanning code unrelated to your application.

In order to avoid being subject to the 360,000 lines of code per year limit, the organization being scanned should be linked to your publisher account in Partner Community. This will grant you access to scan on a per security review basis. If you attempt to scan code in an organization that is not linked to your publisher account, then the system will assume that you are not a partner and will apply the line of code limits.

Process

A new scan can be initiated by submitting your Salesforce.com username to the Force.com Security Source Scanner.  The scanner performs a series of checks and then emails a success or failure job creation notification to the email address associated to the username. The job is placed into our queue and once it completes, if any issues are found, a zipped html result file is emailed to the email address on file for the Author Apex username.

On average, a scan result for a small code size (e.g. less than 50,000 lines of code) is returned within an hour. All scans should complete within one day.

Supported Use Cases

The security of this service rests on the assumption that the recipient of an Author Apex email address is allowed to see scan results for an organization. Implicit in this assumption is that the email domain receiving the results has been approved by the owner of the organization to store sensitive data. A second, less obvious assumption is that this trust is assumed to be transitive: If you have an Author Apex account in an organization and so does someone else, then they can see your scan history just as you can see theirs. All Author Apex accounts are considered privileged accounts. This also applies to Author Apex accounts in sandboxes. If developers or subcontractors create free email accounts and assign their Author Apex accounts to use personal email addresses when developing for a business, then they will put their employers or customers at risk. The current service attempts to detect this use and auto-bans any such users (in some cases permanently banning the org from being scanned).

The following are our policies regarding user types, acceptable usage, and allowed email domains:

Partners

Partners scanning for the AppExchange Security Review can use 3 free scans per review. These users must submit scans with Author Apex accounts associated to the packaging organization for their Appexchange package. This packaging organization must be added to their Partner account on the AppExchange. They must use a business email domain and not a personal or free email domain.

Customers

Salesforce customers wishing to check the security of the unpackaged code in their production or sandbox organizations can scan 360,000 lines of code in any twelve-month period. These users must submit scans with Author Apex accounts that use their business email domain. When scanning sandboxes of production, the line of code usage will be applied to the original organization. Large organizations will quickly exceed their line of code limits, so this type of use is introductory in nature.

Contractors

Contractors doing bespoke development for a third party should provide an Author Apex username with a business email domain controlled by their customer. This will allow them to use their customer's free scan entitlements. If the contractor uses their own business email, then they will quickly exceed the line of code limits across all of their projects. Personal or free email addresses may not be used.

Personal users

Individual Developers studying Salesforce language development wishing to scan non production test code in their Development Organizations can use a personal/free email domain under certain conditions:

  • The following domains are allowed: gmail.com, yahoo.com.
  • Users must use the same email address (their personal address) for all Author Apex accounts associated to scan jobs in their org. Use of more than one personal email address per org may result in scan cancellation.
  • The same total line of code limitation applies as for Salesforce customers scanning their production or sandbox environment.
  • A sandbox org is not being scanned.
  • Each scan is less than 5000 lines of code.

Note that it is a violation of most business security policies to send senstive IP such as source code excerpts and security vulnerabilities in source code to a personal email address. The above safeguards are meant to mitigate the risk that a contract employee places their employer at risk.

General Restrictions

No user should attempt to integrate this scanner into their build system or submit automated scans. Checkmarx has paid offerings that feature professional support, multi-language support, differential scans, integration with source code management tools, integration with Jenkins, issue tracking systems, eclipse/IntelliJ support, ability to write your own queries, and other premium features. The free service is not intended to be a replacement for the paid service. Our primary use case is to support AppExchange Security Reviews.

Technical Overview

The solution consumes your code and builds a tree that represents every possible code flow in the application.  A set of rules is then applied to the parsed flow in order to detect possible quality and security issues.

For example, if the user selects the Security profile, then one rule is to search for Cross Site Scripting vulnerabilities. In that case, the scanner will check to see whether user controlled input, for example in a URL parameter, is reflected back to a VisualForce page without being properly encoded. If such a flow is detected, the tool reports that the codebase is vulnerable to cross site scripting and lists the chain of function calls starting with the tainted data (source) and ending with the vulnerable rendering on the page (sink).

In creating a scan request, the user selects from pre-defined suites of queries, called scan profiles. These correspond to 'Presets' in the CxSAST. The following scan profiles are available:

Security Profile

The scanner will help to detect the following security vulnerability types:

  • Cross Site Scripting (reflected, stored, and DOM based)
  • SOQL/SOSL Injection
  • Access Control Issues (Sharing, FLS)
  • Cross site request forgery attacks
  • Arbitrary Redirects
  • Overly permissive postMessage targets

Quality Profile

The Force.com Security Source Scanner will detect the following common Apex coding and design issues:

  • DML statements inside loops
  • SOQL/SOSL inside loops
  • Hardcoding Trigger.new[0]
  • Hardcoding Trigger.old[0]
  • Queries with no Where clause or no LIMIT clause
  • Not bulkifying apex methods
  • Async (@future) methods inside loops
  • Hardcoding IDs
  • Multiple triggers on same object
  • Static Resource referencing
  • Multiple Visualforce forms in the same page
  • Test methods without assert

Recommended Presets

If you own your own CxSAST, we recommend the following presets, which correspond to the above profiles. However, one of the main advantages of purchasing a scanner is being able to run custom queries and presets suited to your codebase and business rules, therefore the Salesforce presets should be viewed as a starting point for your own projects rather than a mandatory selection. For example, if your app makes heavy use of a third party Javascript framework, then you may want to customize your rules to detect the use of unsafe APIs in that framework. As the source scanner is meant to be used by everyone, we cannot provide a comprehensive list of such APIs.

Failed Queries

Each set of queries defined above is time limited, as is the entire scan. When a query times out, no results are returned for that query. As a result, it is possible to receive a different result set for the same codebase if it is scanned at different times (scan times are dependent on which other jobs are also in process). In this case, the report lists which queries did not complete in the summary table.

Code Scanner Report

Below is a sample output of a simple code flow:

 

More information on secure coding practices on the Force.com platform can be found in the secure coding guidelines.

Known Limitations

Those familiar with static analysis know that there is no tool that can find all bugs.  It's also well understood that these tools, because they lack insight into the context of the application, can produce false positives as well.  It is important to recognize that false negatives and false positives exist in these reports and any given report should not be considered a full and outright security assessment of your application and code.  Manual reviews will always be necessary to verify code correctness.

False Negatives

A false negative occurs when a tool is not able to identify an existing bug.  Decreasing the rate of false negatives is something we are actively working on.  Salesforce.com has several rule sets which are currently works in progress and will be enabled when the false positive rate has been reduced to an acceptable level.  To give some context, the following rule sets are currently works in progress.

  • isAccessible() checks are not currently performed
  • Vulnerabilities in S-Controls
  • Second order Injection attacks
  • Business logic rules, such as the requirement that secrets be stored in protected custom settings, cannot be detected as the scanner is not able to determine which strings are secrets. There are similar limitations with not being able to detect when sensitive data is written to the log.
  • The scanner does not perform character level taint tracking, nor does it have a sophisticated string-solving library. When tainted strings undergo manipulation via string substitution operations or regular expression capture, the scanner will assume that the strings have been sanitized.
  • The scanner is not able to detect vulnerabilities in static resources
  • The scanner is not able to track data flows in nested custom Visualforce components

False Positives

A false positive occurs when a bug is flagged as being legitimate, which a tool misinterprets as being an actual issue.  This can occur for multiple reasons, but the following are known causes of false positives:

  • The scanner is not always able to correctly understand data types leading to false positives. For example, fields such as booleans may be interpreted as cross site scripting vulnerabilities if they are rendered in a page unescaped.
  • The scanner will may not recognize client-side custom sanitization functions, especially if these are functions written in Javascript.
  • The scanner is often not able to understand the proper encoding context, falsely assuming that incorrectly sanitized fields are safe. For example, use of JSENCODE() functions or EscapeSingleQuotes outside of a quoted context.
  • There is only limited support for the Visualforce expression syntax and built-in functions. For example some variables passed to an IF() statement may be not be properly interpreted, leading to false positives.
The report output contains tips as to common locations where false positives can occur.

Inconsistent Scan results

A scan consists of a series of tests. For example, a search for stored cross site scripting (XSS) is one test. Each test is given a maximum time to run, and once this time period is reached, the test times out. In that case, no results are reported for the test. As scan performance is a function of many factors (current load on the machine, disk I/O, etc.) a scan test may time out in one run but complete in another, leading to inconsistent scan results across time.

In addition to that, as our equipment is updated and our version of the Checkmarx scanner is updated, results may change across time for the same source code. For this purpose, the version of the Checkmarx engine that performs the scan is listed in the 'Job Details' header under the field 'CxEngine'. For example, CxEngine: 8.1 HF1 corresponds to Checkmarx CxSAST version 8.1 Hotfix 1.

Finally, as we strive to reduce false positives and increase coverage, we hope you will receive more accurate results over time for the same codebase

Maximum scan size

As scanner resources are fixed and the computational cost of a scan grows non-linearly with code size, we cannot support unlimited scan lengths. Please limit your scan submissions to less than 2 million source lines of code. If you need to submit a large app for the AppExchange that consists of more than the maximum scan size, you can submit the job rejection email as evidence that you attempted to run the scan, fulfilling the requirement of a scan for the Security Review.

Monthly Patching

The third weekend of every month is reserved for monthly server patching. Users will not experience any interruption in submitting jobs, but jobs may take longer to process during this period as servers will be rebooted and some scans may need to be re-run. If you are experiencing delays in receiving scan acknowledgement emails or scan results during the third weekend of the month, please wait until Monday before filing a case.

Obtaining Support

The scanner is now supported. If you are a partner, file a support case in Partner Community. Otherwise, file a support case in the help and training portal. The case should be categorized as 'security and setup' with the 'security' topic.

Frequently Asked Questions

The scanner found no issues but I still failed the Security Review. Are you using a different, private scanner? Can you share access to it?

The Security Review is a manual process, performed by one of our security engineers. We do not have access to a different static analysis service. A clean Checkmarx scan does not guarantee that your code is free of security issues, nor should you rely on any single tool for this.

All tools (as well as human reviewers) have limited coverage, therefore your software development methodology should include a variety of different security checks, so that issues that are missed by one check are caught by another. For example, architectural reviews during the design phase, linters used as pre-commit hooks, periodic source code reviews to ensure compliance with our Secure Coding Guidelines, static analysis scans with tools such as Checkmarx, dynamic analysis testing using webdriver or a similar tool for websites, use of API fuzzers, and even third party penetration testing or code audits are all part of a modern software development process. The reason why we go into detail documenting known limitations is so you can plan for appropriate coverage when including additional checks in your development process.

I received a 'no code' email but am sure there is code in my organization to scan. What should I do?

First, make sure that there is unpackaged code to scan: Apex classes, triggers, Visualforce pages, Visualforce components, and AuraDefinitionBundles. The scanner does not download any packaged code.

Finally, if you still have no success, please file a support case and we will work with you to get your code scanned.

I received very small files named report_XXXX_.html.zip.html. Where is my report?

We use the Force.com Email API, which automatically converts large attachments to download links. In this case, your result file was unusually large and the download link was sent to you in place of the attachment. Some users report infinite redirects when they click on the download links if they are already logged into their Salesforce organization. If you are experiencing this issue, please click on the download links in a private browsing window.

If they still do not open, try downloading the attachment, reading the link in a text editor, and then navigating to this link in a private browsing window.

If you still cannot open the attachment, then file a support case and we will work with you to get your code scanned.

I did not receive any communication since submitting the username.

First, check your bulk mail folder. If you still cannot find the email, it is likely that you submitted an incorrect username. In this case, we will not be able to determine the email address for the account and so cannot communicate with you. Please wait one hour and then try re-submitting with the correct username. Next, verify that you can receive emails sent to the email address associated to the username, particularly if submitting a scan of a sandbox. (By default, sandbox organizations mangle the email addresses of user accounts during the cloning process.)

If the issue is still not resolved, you can file a support case

Is the scan disruptive?

No. You will see no impact in your org while the scan is being conducted.

I submitted a scan for an application on the NA21 or CS32 instances and received no result

Due to technical controls we cannot access user account data (such as email addresses) or pull code from these instances. As a result we can neither scan code nor send out failure emails. If you wish to scan code on these instances, please consider purchasing a Checkmarx license and scanning the code in-house. You can also file a case to have your code scanned -- the same line of code limits apply and you will need to provide your code to us in a zip file.

I found a false positive/false negative. What can I do?

If you believe you have a found a false positive or false negative not listed in this FAQ, please file a support case and we will investigate the issue. We cannot guarantee that all issues will be fixed, but we do work to improve the results over time.

My job was cancelled because I exceeded the line of code limits, but I need my scan results now - what can I do?

Our license agreement with Checkmarx limits the scan services that we can provide, if you need additional static analysis services, please contact Checkmarx directly at salesforce@checkmarx.com

How do I scan my organization more frequently or how do I scan larger organizations than are allowed by the free scanner service?

Please contact Checkmarx and inquire about purchasing a license so that you can integrate scans into your software development lifecycle.

I found the report confusing and do not understand the issues pointed out by the scanner. What should I do?

First, be sure to read the issue descriptions and any references provided.

Next, please read the Secure coding guidelines for additional examples and information related to your issue.

Finally, you are always welcome to book an office hour to discuss any security questions you may have. We would be happy to go over your Force.com code or scan results in a 30-minute live session.

Thank you for using the Force.com Security Source Scanner!