Intro to Vulnreport

Vulnreport uses a Ruby/Rack app that uses the Sinatra DSL framework. In general we run it on thin server, but it can work with many others. Vulnreport is also built and tested to work well on Heroku with a few add-ons (all with an available free tier).

If you're installing Vulnreport locally or on your own VM/server, the dependancies you'll need installed are:

Vulnreport is designed to be extensible and to hook into your existing workflow solutions and external systems. The goal is to make your pentesting process as simple and as painless as possible while generating clean, useful, and actionable finding reports for your customers and partners. Simultaneously, it keeps your finding data in a well-structured, organized form and allows you to report on it, helping you understand what you're finding where.

Installing Vulnreport

1A: Locally / On your own server

To install locally, simply clone the Git repo and run bundle install in the project directory. Once that's done, you'll need to either create a .env file based on .env.example or set the same variables in your local environment so they can be accessed from ENV.

Next, follow the instructions in Step 2 for running the install script.

1B: Heroku

You can deploy to Heroku automatically via the Deploy to Heroku button:

Deploy

After an automatic deploy, you can login to Vulnreport to finish configuration as normal.

To deploy to Heroku, first ensure that you have the Heroku CLI Toolbelt installed and a Heroku app created. Then:


git clone https://github.com/salesforce/vulnreport

heroku git:remote -a [Heroku app name]

heroku addons:create heroku-postgresql:hobby-dev
heroku addons:create heroku-redis:hobby-dev
heroku addons:create rollbar:free
heroku addons:create sendgrid:starter

The default ENV variables for your Heroku app at this point should be mostly good, but you can double check against .env.example. You'll definitely want to add/update VR_SESSION_SECRET. This can be done via the Heroku CLI Toolbelt or the web interface for your app under 'settings'. If this isn't your production install, you should change RACK_ENV to development.


heroku config:set VR_SESSION_SECRET=abc123456
heroku config:set RACK_ENV=production

git push heroku master

2: Install Script

Before attempting to login to Vulnreport and start using it, you'll need to run a one-time seed script to configure the database and initial settings. This script is found in the root of the repo as SEED.rb. To run locally, simply execute ./SEED.rb. If you're deploying on Heroku, you'll run it as heroku run ./SEED.rb -a [Vulnreport App]. If you used the automated 'Deploy to Heroku' feature, this step should have been handled for you automatically.

When you run the seed script, the process should go something like this:


Running ./SEED.rb on ⬢ my-vr-test... up, run.8035

Vulnreport 3.0.0.alpha seed script
WARNING: This script should be run ONCE immediately after deploying and then DELETED

Setting up Vulnreport now...

Setting up the PostgreSQL database...
	Done

Seeding the database...
	Done

User ID 1 created for you


ALL DONE! :)
Login to Vulnreport now and go through the rest of the settings!

3: Finish Configuring

Once Vulnreport is deployed and the seed script has been run, you're ready to do final configuration live in the app. First, you should delete the seed script. Then, login to your new Vulnreport instance with the default admin user - username admin password admin. This should be immediately rotated and/or SSO should be configured.

Your first stop once you've logged in should be to Settings > VR Settings (/admin/settings). Here you should set up your name and authentication settings. After that, you can reconfigure your user settings and set up Users, Organizations, Record Types, Vuln Types, etc. for your use case. Please see the Admin section of the documentation below for more information about configuring Vulnreport.


Users & Orgs

User access and permissions are broken into two concepts - Users, which are individual pentesters, engineers, or team members; and Orgs, which are groups of logically separated Users. An Org may be a pentest team, a project group, or a consulting company who has access to your Vulnreport instance to perform tests for you. Both Users and Orgs have specific permissions, requirements, and access controls that can be applied to them.

The primary means of access control to tests is controlled by Record Types and Orgs. When you create or configure a Record Type, you'll be able to configure which Orgs have access to Applications of that Record Type. Giving an Org access to the Record Type allows all Users of that Org to read/write to all Applications of that type.

Other important Org-level permissions are as follows:

  • Contractor Org - Indicates that this Org is used by external Contractors working with your team
  • Super Org - Bypasses individual Record Type permissions and allows members of this Org to see any Application or Test unless it is marked as private.
  • Use Reports - Allows Users of this Org to use reporting functionality
  • Require Test Approval - Requires all Users of this Org to go through an approval flow before being able to finalize (pass/fail) tests. The Orgs and Users who can approve this Org's reports can then be configured below.

There are some slightly more granular permissions available in the User admin view. Specifically:

  • User Verified - If a user is verified in the system. This toggle is important when SSO authentication is enabled along with the "auto create" Users option. In this case, users who have valid SSO authentication but no corresponding Vulnreport account will have a User automatically created in an "unverified" state until an Admin configures their User. Setting this to "No" will remove the User from their Org.
  • User Active - If a user is active. Users cannot be deleted for data integrity reasons, so if an engineer leaves your team their account should be set to Inactive.
  • VR Admin - User is an administrator and has full management ability. Admins also bypass almost all permissions checks. (Exclusing private tests)
  • Reports Only User - User is only allowed to view reporting functionality, not actual tests or vulnerabilities. Note that reports only Users must still be assigned to an Org with access to the Record Types they should report on.
  • Audit Monitor Alerts - User is allowed to view and respond to monitor alerts. See the Custom Interfaces section for more information about setting up monitor alerts.
  • Provisional Pass - User is allowed to approve requests for provisional passes if the record type allows for it. Depending on your own business logic, you may not care about this option/process.
  • Require Test Approval - User is required to go through an approval flow for their test results. This configuration is per-user; if the Org the User is in is required to go through an approval flow the User will be required to as well. If, in that case, the User is also set to require test approval, their approvers configuration will override that of the Org's. However, it is impossible for a User who is in an Org that requires approval to not requier approval themselves.
  • Pass to Contractor - User is allowed to assign an Application to a User belonging to a Contractor Org.
  • Monthly Allocation - User is configured to use the Monthly Allocation system for work tracking.
  • Allocation Coefficient - For the allocation system, total number of reviews this User would complete if allocated 100% for an entire year. This is used in calculating the allocation for a month based no the percentage selected by the User. For example, if a User's allocation coefficient was 100, and they were allocated 50% for a given month, the expectation would be 4 reviews completed in the given month.
  • Login History - Information about the 10 most recent login attempts by the user - IP address, user agent, and login type (SSO or Direct) are available. A full view of the login history is available as well by clicking the button.

Also available in the User admin view are capabilities to edit any field on the user, update which Org they belong to, and reset their username/password. Note that only an admin can update Users' username. If SSO authentication is enabled, the User admin view will also allow and admin to update the mapping between Vulnreport users and external authentication users.

A note on the External EID field - this data is not used by the core Vulnreport application, but is intended to allow you and your users to store an external entity ID mapping for use in custom integrations. If you are not using a custom integration that requires data about an external system mapping for User objects, you can ignore this field. As an example, the Salesforce Trust instance of Vulnreport uses this field to store User ID's for our Chatter system, so that Vulnreport can @-mention users as appropriate.


Applications & Tests

Applications and Tests are the primary objects in Vulnreport, used to represent security reviews, penetration tests, feature code reviews, etc.

Applications

The concept behind an Application is a representation of the entire lifecycle of the app, feature, product, etc. that you are testing. Tests belong to an Application, and represent an individual security test. In this way, you can track the security progress of an application/feature/product as you find vulns and they are remediated. Depending on your business logic, or if you do limited-engagement consulting, you may only ever have one Test on each Application. More likely, though, is a series of failed Tests leading to a final passed Test on a given Application.

To create an Application, click the 'Reviews' drop down in the left nav and select 'New Review'.

As it relates to access controls, every Application has a Record Type. Access controls based on Application Record Type are discussed above, in the Users and Orgs section of this documentation. Record Types have additional features, such as linking to external objects, controlling which types of vulns can be added to a Test, etc. These features are discussed in the admin/config and custom interfaces sections of the documentation below. In the image above, you can see that this Application is part of the General Review Record Type, denoted in the header of the Application view. This can be modified after creation by editing the test.

Applications can also be assigned 'Flags', which provide more granular segregation. Flags can be filtered by in reports and list views, making them useful for denoting categories of Applications within a Record Type. In the image above, you can see that the FooBar Security Review Application is flagged as 'Initial Review'. Flags can be configured as part of the admin controls in Vulnreport.

Clicking 'Edit' in the navbar from the Application view allows you to edit all details about the Application:

  • Type - This controls the Record Type of the Application
  • Owner - The User responsible for this Application
  • Geo - The geographic region that this Application is part of. If your business logic does not care about geographic area, it can be ignored.
  • Additional Emails - Not used by core Vulnreport, but may be useful based on integrations if you are automating the emailing of findings reports.
  • Global App - If marked 'Yes', this app will be viewable by anyone, regardless of Record Type access restrictions
  • Private App - If marked 'Yes', this app will be accessible only to the users manually specified below. Note that admin users will still be able to access the app, but will be presented with an interstitial warning that they are overriding access controls and that override will be logged in the app audit.
  • Allowed Users - If the app is marked as private, only these users will be able to access. If the app is not marked as private, users specified here will be able to override normal Record Type access rules to view this Application.
  • Application Flags - Flags marked on this Application, as discussed above.

If you're concerned about understanding the access rules for a specific application, you can click 'View Permissions Calculations' from the Application Edit view, or "Perms" from the Application view navbar to see a full listing of which orgs and users have access to the Application based on current settings

Significant actions on an Application are logged as Audit Records, and can be viewed from the "Audit" item on the Application view navbar. Actions logged for audit include creation, renaming, marking the Application as private or global, changing the record type, adding/removing a flag, and reassigning ownership. If the Application/Record Type is linked to an external integration, adding/changing/removing the external entity an Application is linked to is also logged.

Tests

Tests represent a single security test/penetration test/audit of an Application. To open a new test, navigate to the Application and click Start New Test. Tests will automatically be named "Test #", but this name can be edited from the Test view.

Essentially, a Test is simply a collection of Vulnerabilities. As you find an issue during a Test, add the Vulnerability from the Test view to associate it with the Test. More detail about Vulnerabilities and vuln data is in the documentation below. The icons next to each Vulnerability on the Test allow you to put the Vulnerability on your clipboard or delete it, respectively. Vulnerabilities can be copied and pasted between tests using the icons next to the Vulnerability listing on the Test bar or in the Test view navbar.

When you've completed work on a Test, you should finalize it one way or another using the status dropdown at the left side of the Test view navbar. A test can be closed as either a "Pass" or "Fail". If enabled for the Record Type that the Application containing the Test is on, a Test may be "Provisonally Passed", pending approval by a User with permissions to approve provisional passes. This feature may not be useful depending on your business logic.

Once a test is finalized, you (or your automation) can export/view the generated report document. This report is generated using all data contained in the confirmed Vulnerabilities on the Test as well as the stock remediation advice and issue descriptions entered for the Vulntypes used (see the admin/config documentation section). Vulnerabilities will be organized by Vulntype and sorted by Risk Level on exported reports. By default, reports are generated as self-contained HTML documents. If enabled in Vulnreport's configuration, a PDF export option can be used as well. PDF exports rely on generating the HTML report and converting it to a PDF document, so depending on the export template may be less ideal.

Much like Users, Test objects have an External EID field (accessible through the Edit Test view). This data is not used by core Vulnreport, but can be useful in developing custom integrations with external systems. If you do not use this field in a custom integration, you can safely ignore it.

XML Export/Import

Currently, Vulnreport supports an XML format to import Vulns to a specific Test. This is useful if you want Vulnreport to be on a different network than you do your pentests on and thus are using a different client to record findings while you actively pentest, but relies on being configured for your specific Vulnreport instance and Vulntypes configuration.

We're working on supporting a few other types of XML import (ZAP and Burp, for instance) as well as allowing arbitrary XML export/import between Vulnreport instances. Stay tuned as we hope to push these features soon.

The XML format Vulnreport currently supports is:


<?xml version="1.0" encoding="UTF-8"?>

<Test xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Vuln>
    <Type>[Vulntype ID]</Type>
    <File>[File Vuln Data]</File>
    <Code>
      [Code Vuln Data]
    </Code>
    <File>clsSyncLog.cls</File>
    <Code>
      hello world
    </Code>
    ...etc...
  </Vuln>

  <Vuln>
    <Type>6</Type>
    <File>clsSyncLog.cls</File>
    <File>CommonFunction.cls</File>
    <Code>
      12 Public Class CommonFunction{
    </Code>
  </Vuln>
</Test>

<?xml version="1.0" encoding="UTF-8"?>

<Test xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"">
  <Vuln>
    <Type>REQUIRED - EXACTLY 1 - INTEGER - ID of VulnType. 0 = Custom</Type>
    <CustomTypeName>OPTIONAL - EXACTLY 1 - STRING if TYPE == 0</CustomTypeName>
    <BurpData>OPTIONAL - UNLIMITED - STRING - Burp req/resp data encoded in our protocol</BurpData>
    <URL>OPTIONAL - UNLIMITED - STRING - URL for finding</URL>
    <FileName>OPTIONAL - UNLIMITED - STRING - Name/path of file for finding</FileName>
    <Output>OPTIONAL - UNLIMITED - STRING - Output details</Output>
    <Code>OPTIONAL - UNLIMITED - STRING - Code details</Code>
    <Notes>OPTIONAL - UNLIMITED - STRING - Notes for vuln</Notes>
    <Screenshot>
      OPTIONAL - UNLIMITED - Screenshots of vuln
      <Filename>REQUIRED - EXACTLY 1 - STRING - Filename with extension</Filename>
      <ImageData>
        REQUIRED - EXACTLY 1 - BASE64 - Screenshot data
      </ImageData>
    </Screenshot>
  </Vuln>

  ....unlimited vulns....

  <Vuln>
  </Vuln>
</Test>

Vulns & Vuln Data

Vulnerabilities are the objects that make up a Test. All data about findings during a security test are contained within Vulnerabilities attached to that Test. To add a Vulnerability to a Test, simply click on the Vulnerability type you want to add. If none of the standard types fit the finding you'd like to add, type a name for a 'custom vulnerability' at the bottom of the 'Add Vuln' section of the Test view. Admins can configure new standard Vulntypes and apply Vulntypes to specific Record Types through the Vulnreport configuration. Admins will also have the option to see which custom Vulntypes have been used on Tests and add them as standard types or merge them into existing standard types.

The Vulnerability view, from which you can add data to the Vulnerability, looks like this:

The Vulnerability view navbar lets you set/override three main properties:

  • Vulnerability status - Verified, Unverified, False Positive. Note that only 'verified' Vulnerabilities will be exported on reports.
  • Vulnerability risk level - Options are defined per Record Type, but default to Critical, High, Medium, Low, Informational. Vulntypes have a default setting, which can be overridden on a per-Vulnerability basis via this menu. Vulnerabilities will be organized by Vulntype and sorted by Risk Level (high to low) on exported reports.
  • Vulnerability CWE mapping - Which CWE ID this Vulnerability finding maps to, if any. Vulntypes have a default setting, which can be overridden on a per-Vulnerability basis via this menu. Setting this menu item to blank or zero will reset to the Vulntype default.

The top section of the Vulnerability view shows vuln data sections that have already been added and allows the reordering of these sections via drag and drop. The ordering of these sections will be replicated on exported reports. Existing vuln data sections can also be edited or deleted from this portion of the view using the respective icons.

The bottom section of the Vulnerability view allows the addition of new vuln data. There are 8 types of data section, all shown in the example image above. Which data sections are available on each Vulntype is configurable in the Vulnreport configuration settings for each Vulntype.

  • URL - The URL this vulnerability was found at/on.
  • Filename - The filename this vulnerability was found at in source code.
  • Screenshot - Screenshot of the vulnerability/results of exploitation.
  • Code / Request - Code in question or vulnerable code. Alternatively, the request that produced a vulnerability or was exploitable.
  • Output / Response - Output of vulnerable code. Alternatively, the response from a vulnerability or exploitation.
  • Notes - Freeform text notes about the Vulnerability.
  • Payload - Binary payload file that caused a vulnerability. This vuln data section is for reference only and is not usually exported.

The Burp Data section is unique - if you install the vulnreport.jar Burp plugin as part of your Burp classpath, you will be able to right-click any request/response in Burp and copy the data to a string which can be decoded by Vulnreport when pasted in this section type. The decoded request/response and URL will be added as vuln data sections.

The Vulnreport Burp plugin is a legacy plugin, and should be loaded as part of your Burp classpath when launching. An example Burp launch script is:


java -Xmx1024m -classpath /Tools/Burp/vulnreport.jar:/Applications/burpsuite_pro.jar:. burp.StartBurp

It is possible to 'star' a Vulnerability from the breadcrumb navbar. Starring a Vulnerability is a great way to save critical or representative vulns for review later. Another good usage of starred Vulnerabilities is for collecting interesting/representative vulns to use when onboarding new testers. All starred vulns can be viewed from the 'Reports' section.


Administration and Configuration

Vulnreport & Authentication Config

Vulnreport configuration includes both system options, style options, and authentication methods. All settings are available from the Settings > VR Settings left nav:

  • VR Instance Name - The name of your company/team. This is for styling only.
  • Vulnreport Root - The root URL of your Vulnreport installation. Trailing slash not required.
  • VR Footer (HTML) - HTML footer Vulnreport will display on all pages
  • Max Screenshot Size - The maximum size of uploads for the screenshot vuln data section type in KB.
  • Max Payload Size - The maximum size of uploads for the payload vuln data section type in KB.
  • PDF Export Option - If on, PDF export reports will be enabled on all Tests as an option. This option requires WKHTMLTOPDF be installed in your environment. On Heroku, that requires the WKHTMLTOPDF-Heroku gem be installed via manual addition of the Gem to your Gemfile and deploying the modified Gemfile to Heroku.
  • SSO Settings - If enabled by toggling SSO Auth to yes, these settings will control integration with your SAML SSO provider. You should use the Vulnreport path /saml/finalize as the SSO URL for your provider.
  • SSO - Auto-Create Users - If yes, valid users logging in from SSO who do not yet have a Vulnreport User mapped to their SSO account will have a Vulnreport User created for them in "Unverified" state. An admin will have to verify and grant access permissions to the User.
  • Login Auth - If yes, Vulnreport-direct authentication (user/password) will be enabled. Minimum password length can be set here as well. It is possible for both SSO and Login auth to be enabled at the same time.
  • IP Restrictions - If yes, Vulnreport will enforce IP restrictions on all requests. Individual IP addresses, IP address ranges, or CIDR's may be used here, comma-separated. Formats may be mixed. Note that Vulnreport will track the IP's a User logs in from no matter this configuration's setting. This history is viewable in the User Admin UI.

Vulnreport Mail Config

Vulnreport needs to be able to send out certain types of mail messages - e.g. password resets. Vulnreport core code does not handle the mailing of test reports, though this can be easily implemented using a VR Linked Object through the same mail commands. In order to send out mail, Vulnreport uses either the standard system sendmail command or custom SMTP/mail settings.

Note that if you use the automatic Heroku Deploy or follow the installation instructions for Heroku you will have provisioned a free SendGrid component and Vulnreport will have automatically been configured to use SendGrid mail servers. If you are running on Heroku it is impossible to use the sendmail

  • VR NoReply Address - The email address Vulnreport should use as "from" when sending emails.
  • Mail Method - Whether Vulnreport should use sendmail or custom settings
  • Mail Server Addr - Custom SMTP mail server address
  • Mail Server Port - SMTP mail server port to use
  • Domain - HELO domain provided by client to server
  • Username - SMTP server username to authenticate with
  • Password - SMTP server password to authenticate with

Vulnreport Cron Jobs Config

Vulnreport allows you to implement custom Cron-style jobs that have access to the database and Vulnreport environment. More information on this is below in the VR Crons documentation. Any job registered with Vulnreport (done on init) can be monitored and enabled/disabled from this admin UI. Jobs can also be run off-cycle from this interface. Any job run manually will be run in a background thread.

  • Job Name - UI-friendly job name defined in the VRCron subclass for this job
  • Schedule - Job schedule - either "every XX" or "cron [crontab schedule]"
  • Enabled? - If green the job is enabled and will run normally. If red the job will not run when scheduled. Click to toggle.
  • Run Now - Manually run the job immediately in a background thread
  • Last Run - Date/Time this job last completed or errored out
  • Last Return - Return value (converted to string) of last run or error of last failure

Record Types & Flags

Record Types can be created to segregate Applications into logical groupings. Record Types can also affect how Applications are treated in ways like exported report templating, the names of the Vulnerability priority levels, and whether Applications of the type can be given to a contractor User. Additionally, Record Types control which standard Vulntypes are available on Tests for Applications of the type.

  • Active - Whether or not the Record Type can be used. For data integrity reasons, Record Types can't be deleted. Deactivating them will prevent new apps from being created with them.
  • Export Format - Which template for exported reports this Record Type should use.
  • Linked Object - Whether or not this Record Type is linked to an external system via a VR Linked Object (see Custom Interfaces), and if so, which.
  • Contractor Passable - If yes, Applications of this type can be assigned to Users who are in a Contractor Org
  • Provisional Pass - If yes, Tests on Applications of this type can be provisionally passed
  • Default Private - If yes, Applications of this type will be automatically set as private when created. The initial access settings will be the creating User only.
  • Vuln Priorities - The 5 strings representing the priorities for Vulnerabilities on Tests for Applications of this type
  • Record Type Assignments - Orgs that can access Applications/Tests of this type
  • Vulntypes - Standard Vulntypes that should be available on Tests for Applications of this type

Flags are used to mark Applications at a more granular level than Record Types, and can be filtered by in reports and list views. Flags are defined solely with a name and optional description. Optionally, a Font Awesome Icon can be specified to be used with the flag. Flags can then be added to Applications from the Application Edit view (or programatically be a custom integration).

Vulntypes

Vulntypes are standard types of Vulnerabilities that you expect to find often on Tests. Vulnreport is seeded with 24 standard Vulntypes which represent commonly found web application vulnerabilities, including the OWASP Top 10. Feel free to modify, delete, or add to these Vulntypes to make Vulnreport better suit your workflow use case.

  • Enabled - Whether or not the Vulntype can be used. For data integrity reasons, Vulntypes can't be deleted. Deactivating them will prevent new Vulnerabilities from being created with them.
  • Label - The "public-facing" String that will be used to label this Vulntype. Used in reports and on the "Add Vuln" section of the Test view. This can differ from the Vulntype name for administrative purposes, e.g. if you want to have two "Stored XSS" Vulntypes with different descriptions for different Record Types, but want them both to appear as "Stored XSS" publicly.
  • CWE Mapping ID - CWE ID that Vulnerabilities of this Vulntype relates to by default. Can be overridden on a per-Vulnerability basis.
  • Default Priority - Priority that Vulnerabilities of this Vulntype will be set to by default. Can be overridden on a per-Vulnerability basis.
  • Report HTML - HTML that will be used in exported reports to describe this Vulnerability. Best used for explaining what the Vulnerability is, how it is caused, and giving remediation advice.
  • App Record Types - Record Types that this Vulntype will appear on.
  • Enabled Section Types - Vuln data section types that can be added to Vulnerabilities of this Vulntype.

During testing Users may add "custom vulns" to reports when none of the standard Vulntypes represent the vulnerability that has been found. From the Vulnreport configuration view you can see all "Custom Vulntypes" that have been used in Vulnreport and which tests they were used on. If a certain type of custom vuln has been used frequently, you can add it as a standard Vulntype and automatically convert all previous instances of it to that standard Vulntype. It is also possible to merge all instances of a custom Vulntype with an existing standard Vulntype.

Dashboards

Dashboards are the home/landing page in Vulnreport. The primary view that any User sees upon logging in is the Dashboard assigned to their Org or that they have personally chosen. The default Vulnreport Dashboard shows 3 panels - My Active Reviews, My New Reviews, and My Pending Approvals. Which Applications these panels show is fairly self explanatory.

When configuring a new custom Dashboard, you have several options. You can add an arbitrary number of panels based on predefined types (combined, in many cases, with selecting a Record Type for each panel). You can also choose, optionally, for the Dashboard to display a 4-item stats bar at the top. Stats boxes show only a record count, but otherwise are configured the same way panel is. When viewing a Dashboard, any user can select which Geo (or all) to filter the Dashboard by. Dashboards using custom code can also be written via a custom interface exposed by Vulnreport - see the doc section on custom interfaces below.

The types of panels/stats boxes that you can use are:

  • My Active Reviews (All) - All Applications on which the Dashboard viewer is the reviewer of an active Test
  • My Active Reviews (Type) - All Applications of a specific Record Type on which the Dashboard viewer is the reviewer of an active Test
  • My New Reviews (All) - All Applications owned by the Dashboard viewer with no Tests
  • My New Reviews (Type) - All Applications owned by the Dashboard viewer of a specific Record Type with no Tests
  • My Passed Reviews - All Applications with a passed Test by the Dashboard viewer
  • My Failed Reviews - All Applications with a failed Test by the Dashboard viewer
  • My Completed (All) Reviews - All applications with a completed Test by the Dashboard viewer
  • In-Progress Reviews - Applications of a specific Record Type that are new or have an active Test
  • Passed Reviews - Applications of a specific Record Type with a passed Test
  • Failed Reviews - Applications of a specific Record Type with a failed Test
  • Closed Reviews - Applications of a specific Record Type with any completed Test
  • All Reviews - All Applications of a specific Record Type
  • New Reviews (No Tests) - Applications of a specific Record Type with no Tests
  • Pending Approvals (All) - Applications with a Test pending approval for which the Dashboard viewer is authorized to approve
  • Pending Approvals (Type) - Applications of a specific Record Type with a Test pending approval for which the Dashboard viewer is authorized to approve

Export Formats

Report templates can be defined with a name, description, and ERB templating code. A description of the variables that should be used to generate an exported report. Vulnreport comes with a generic export report based on the template that Salesforce uses for AppExchange Security Review reports. This template can be modified, but not deleted (guaranteeing that Vulnreport always has at least one template to use). The ERB code that is used to generate the report has full-scope access to the environment, so please be careful with who you give admin rights to and double check template code carefully before deploying and using it.


Custom Interfaces

Vulnreport offers several interfaces which allow custom code you write to hook into Vulnreport for the purposes of customizing it, hooking it in to external systems, or otherwise fitting Vulnreport into your existing workflow and business logic. The interfaces are described below and links are provided to the interface code. You may find it useful to build the Vulnreport code documentation using yard doc in the app root directory. You can also browse the latest version of the code documentation here.

VR Linked Objects

A VR Linked Object (referred to in the code as a VRLO) is specified by Record Type(s) and provides hooks into all major actions that occur on Applications of that Record Type and their Tests. VRLO's also allow you to add custom items to the alerts and navbars of both the Application and Test views. As an example: Salesforce uses VRLOs to ensure that when we pass or fail an AppExchange Security Review, appropriate actions are automatically taken via our APIs to list the application publicly, notify the partner, and set other information in our internal systems.

A sample VR Linked Object is provided in /linkedObjects/Examples/salesforceCase.rb. This sample implements a link between Vulnreport Applications and Salesforce Case Objects. This VRLO will display information about a Salesforce Case in Vulnreport when it is linked to a Vulnreport Application. Upon assignment, creation and resolution of security Tests in Vulnreport, the Salesforce Case Object will be updated with status and owner information (demonstrating usage of the extEID field on the User object). As a sample, this VRLO code should not be used directly, but can be used as a basis for implementing your own integrations.

A VR Linked Object is defined by class MyNewVRLO < VRLinkedObject and the VRLO interface (yard doc) defines the following methods:

  • vrlo_key - Return a unique String key for this VRLO.
  • vrlo_name - Return a String naming this VRLO for use in UI.
  • getLinkedObjectInfoPanel(app, uid, params, env) - Fire when loading the Application view for a linked Application. Lets you display information about the externally linked system as well as custom menu items and alerts.
    • Alerts are returned as an Array with each element a Hash representing a single Alert. The Hash takes the form {:color => String, :html => String}. :color can be 'danger' (red), 'warning' (yellow), 'success' (green), or 'info' (blue). :html is a String of HTML to display in the alert body.
    • Menu items are returned as an Array with each element a Hash representing a single custom menu item. The Hash takes the form {:icon => String, :text => String, :href => String, :authMethods => Array, :isDropdown => Boolean, :dropdownOpts => Array}. :icon is the ID of a Font Awesome Icon. :text is the menu item text, which should be as short as possible. :href is the URL to link to (can be relative). :isDropdown is true if the custom menu item is a dropdown menu. If this is true, dropdownOpts must be specified.
      • :authMethods is an Array of Arrays, where each Array has 2 elements. Element 0 represents a method to call from the VulnreportAuth module. Element 1 is an array (which can be empty) of arguments to pass to the called method. If and only if this method call returns true will the custom menu item be displayed to the User.
      • :dropdownOpts is an Array where each element is a Hash of the form {:href => String, :text => String}. :text is the menu item text, which should be as short as possible. :href is the URL to link to (can be relative).
  • getTestAlerts(app, test, uid) - Fired when loading the Test view for a Test on a linked Application. Let's you display alerts and custom menu items. See above for the syntax for returning custom alerts and menu items.
  • doCreateAppActions(app, uid) - Fired when an Application is created on a linked Record Type.
  • doAppReassignedActions(app, uid, newOwnerUid) - Fired when a linked Application changes owners.
  • doNewTestActions(app, test, uid) - Fired when a new Test is created on a linked Application.
  • doInProgressActions(app, test, uid) - Fired when a Test on a linked Application is put in progress. Note that this action does not fire on Test creation even though a newly created Test is considered "in progress".
  • doPassActions(app, test, uid) - Fired when a Test on a linked Application is passed.
  • doProvPassActions(app, test, uid) - Fired when a Test on a linked Application is provisionally passed. This action fires on approval of a provisional pass, not request.
  • doFailActions(app, test, uid) - Fired when a Test on a linked Application is failed.
  • doDeleteAppActions(app, uid) - Fired when a linked Application is deleted in Vulnreport.
  • getLinkedObjectText(app) - Should return a String representing the externally linked object for a given Application.
  • getLinkedObjectURL(app) - Should return a URL directing to the externally linked object for a given Application.
  • validateEntityID(eid, app) - Validate that a given external entity ID is valid for this type of link.
  • doSearch(q) - Return custom search results for the external linked system when a Vulnreport search is run.

Any method not implemented by your VRLO will be ignored - the default behavior is a noop/return success. If necessary, you can define custom routes in the main Vulnreport application for your VRLO by reopening the Vulnreport main class in the VRLO file or in a file required by the VRLO file.

Custom Code Dashboards

Vulnreport allows you to write Dashboards based on custom-code if the prebuilt options for Dashboard configuration do not suit your needs. Custom code Dashboards implement the VRDashConfig interface. During Vulnreport initialization, all custom Dashboards will be detected and a database object for the Dashboard created based on the unique key defined by your custom class. This allows you to define custom settings in the code which can be controlled through the Vulnreport configuration UI.

A custom Dashboard is defined by class MyNewDash < VRDashConfig and the VRDashConfig interface (yard doc) defines the following methods:

  • vrdash_key - Return a unique String key for this Dashboard.
  • vrdash_name - Return a String naming this Dashboard for use in UI.
  • vrdash_settings - Return a Hash with custom settings that should be implemented for this Dashboard. The Hash is made up of Hashes of the form {:name => String, :default => Object}. :name is the String representing the setting (displayed in the admin configuration area) and :default is the default value of the setting. Your custom code will receive the current values of these settings when a Dashboard generate is requested.
  • generate(dc_settings, uid, geo=GEO::USA, cache=true) - Called when the custom Dashboard is requested. Passed a hash of the custom settings and the ID of the User generating the dashboard. This method should returns information for Dashboard panels and stat boxes. See code comments/documentation for a full specification of return values.

VR Crons

Cronjobs provide for many useful functions, such as keeping tabs on reviews, sending out regular emails to Users, warnings about how long a Test is taking, etc. For additional convenience, Vulnreport lets you hook a cron into the Vulnreport app, allowing you to easily access all Vulnreport objects from the cron code. The Vulnreport scheduler processes all custom crons on initialization and schedules them based on the schedule definition in your custom code.

A custom VR Cron job is defined by class MyNewCron < VRCron and the VRCron interface (yard doc) defines the following methods:

  • vrcron_name - Return a String naming this cron for use in UI.
  • vrcron_type - Type of scheduling - either :every or :cron.
  • vrcron_schedule - Return a string schedule - if type is :every, then of the form 30m, if the type is :cron then using normal crontab scheduling, e.g. 0 7 1 * *
  • cron - The method that will be invoked by Vulnreport at the schedule specified.

Monitor Alert Audit Records

In conjunction (usually) with VR Crons, it is possible to define a special type of Audit Record to keep an eye out for actions in external systems (or in Vulnreport) that you may want to be alerted to. This is done defining MONITOR_EVENT_TYPES and creating AuditRecord objects using them.

Once you have defined Monitor Event Types you will see the "Monitor Alerts" interface appear in Vulnreport's left nav for all Users who you have given the "Audit Monitor Alerts" permission. This interface will alert you to all Audit Records of the defined Monitor Event Types and allow you to mark them as "OK" or flag them for further review.

To define Monitor Event Types, modify models/enums.rb and add new event type constants to the EVENT_TYPE module, under the comment header for "Monitors - 90x". Then add these EVENT_TYPEs to the MONITOR_EVENT_TYPES array definition.

Next, modify the four functions defined at the top of routes/monitors.rb as directed in their definitions.

You should now be ready to created Audit Record objects using the Event Types and track them in the Monitors interface.

Cache and Direct DB Queries

Vulnreport implements a Redis caching layer which can be managed through the UI on the VR Settings -> Cache Settings page. This caching layer is not used by core Vulnreport functionality, but is intended to be used to cache slow-changing data from external systems. For instance, at Salesforce we cache most of the data we pull from Salesforce orgs about our pentests. This data is input before the test request comes to us, and once it's there it doesn't change much while we perform the test. So we can speed up Vulnreport's performance without impacting data quality by caching it after getting the data once (on initial Application creation/load through the VRLO and a custom Dashboard). Best practice is always to include a "refresh" option in your UI if you are caching data.

If you find the need to directly manipulate Vulnreport's objects, you can do so through IRB. Please note that this should be used as a last resort, as directly manipulating objects in the database can miss things that Vulnreport does, like creating object links, Audit Records, updating timestamps, etc. To interact with the database directly, assuming you are in irb, run require './models/init'. At this point you will have access to all Vulnreport objects. For example, you could run Application.count(:owner => 3) to see how many Applications are owned by the User with ID 3.


License & Authors

If you have ideas or run into problems, or just have general feedback, we'd love to hear from you. Feel free to open up an Issue or a Pull Request on the Git Repo. You can also email us at vulnreport_public [at] salesforce [dot] com.

Vulnreport was developed by the Salesforce Product Security team and is ©2016 salesforce.com, inc. and licensed under the BSD 3-Clause license. For full license text, please see LICENSE.txt in the repo.

Author: Tim Bach <tim.bach [at] salesforce [dot] com>