Cyber Security and Your SaaS Ecosystem—Part 3

Cyber Security and Your SaaS Ecosystem—Part 3

Evaluating SaaS Applications that Handle Sensitive Data

Use of Software as a Service (SaaS) applications continues to grow, replacing traditional enterprise software in more and more areas of the business. When considering a SaaS solution for critical business uses, it is paramount organizations thoroughly vet vendors and their software to identify risks. In part one of this three-part blog series, we introduced the use of a context framework for initial evaluation of SaaS vendors.  In part two, we explained how to more thoroughly evaluate the business prospects of a SaaS firm.

In this final installment, we share our expert advice on how to evaluate vendors whose SaaS applications are being considered to process, store, or transmit regulated, sensitive, or high-value privacy data, and pose a higher level of risk to the business. Some key domains where reliance on a SaaS vendor presents risk:

  • Strategic and Operations Risk – impact to business results or dependence upon the service to complete key business processes
  • Technical Risk – reliance to provide methods to interact with other IT systems, internal, or external
  • Compliance Risk – reliance to maintain compliance with regulations or laws
  • Privacy Risk – reliance on SaaS provider’s security controls and business processes to protect user and data subject privacy information
  • Cyber Security Risk – reliance to implement and maintain effective security controls

Resources and time for thorough evaluations are finite for most organizations. Even when highly sensitive data is involved that can result in compliance or reputation risk evaluation, organizations must decide how much due diligence is enough or what dimensions are most important to verify. Here are areas to consider.

Strategic and Operations Risk

If you are making a strategic investment in a SaaS vendor and believe the investment creates a competitive advantage for your business, evaluate the strategic risks like you would any investment, merger, or acquisition. In addition to a thorough cyber security risk assessment, your evaluation should assess:

  • Whether other customers demand change to the scope and features of the product that differ from your firm’s use cases.
  • Availability of the service because of technical reasons.
  • Availability of the service because of business failure or restructuring.

In addition, you should consider having a Business Impact Analysis completed for the service and its planned integrations to help you quantify the risk associated with availability. And, since many SaaS firms are privately held, you may need to discuss business and exit plans with the principals or investors. This Capshare blog provides a good primer on metrics important to VC investors and some topics for discussion.

Technical Risk

Most firms will engage their IT teams early in the SaaS selection process. Important areas of consideration for technical teams may include:

  • Functional creep – SaaS providers tend to invest in functionality that is used and valued by the bulk of their customers. If your functional desires are not in the majority, your use cases may not get met, and new bells and whistles will have little value to your firm.
  • Integration and data volumes – Some SaaS providers set pricing or technical limits based on application programming interface calls or total data ingestion or transfer. These limits may not work for your company.
  • Lock-in and lock-out – The more work you do integrating SaaS applications or writing custom scripts or reports, the more locked in you become and vulnerable to future pricing or competitive transition. Additionally, be sure to evaluate terms that allow a SaaS provider to lock out users or integrations from the system. It is always a bad IT day when a key SaaS application is not available because your firm crossed a threshold no one was watching. 

Compliance, Privacy, and Cyber Security Risk

For simplicity, let us consider compliance and privacy as inputs to Cyber Security SaaS evaluation. In which case, your Cyber Security team is well aware of the firm’s compliance and privacy requirements, e.g. General Data Protection Regulation (GDPR) for Privacy, Payment Card Industry Data Security Standard (PCI), Sarbanes–Oxley Act of 2002 (SOX), or Health Insurance Portability and Accountability Act (HIPAA). We assume your company has a standard method for evaluating third-party risk for IT or application providers that tiers providers in some fashion like the following figure. 

Evaluation Type and Activities

Selection Factors

Commodity - Survey Questionnaire Only

  • Targeted questionnaire at onboarding and at some cadence
  • Human or system scoring results in low risk relationship
  • Some back and forth clarifying response by email
  • Low compliance, privacy, or commercial risk due to the nature of the services provided and data handled by the third party

Table Top

  • Conduct a table top review using teleconference or other collaboration software of the survey and remediation items
  • Agree to remediation items and track to conclusion
  • Medium or low privacy, compliance, or security risk
  • Some items on survey require remediation

Assessment (not evidence-based

  • Onsite assessment team to review questionnaire and related questions with managers and technical principals
  • Agree to remediation items and discuss priority and impact if not improved
  • Medium or high privacy, compliance, or security risk
  • Some items on survey require remediation

Evidence-Based Assessment

  • Onsite assessment and extended evidence collection for verification
  • Technical discovery and control auditing
  • Agree to remediation items and discuss priority and impact if not improved
  • High or very high privacy, compliance, or security risk
  • Known remediation or previous failures to remediate
Figure 1 Assessment Tiers

Given this structure and the general risk profile of the SaaS marketplace, we recommend, as a minimum, that for SaaS applications handling sensitive compliance, privacy, or business data you conduct an on-site assessment. If the risk level is very high, you should use an evidence-based assessment approach.

Key Resources for an Evidence-Based Assessment

Step 1: Submit a Questionnaire

Though practitioners talk about shared assessments and mechanisms for consortia aggregated assessments, firms tend to craft their own questionnaires and evaluation processes. Vendors are quite practiced at completing security questionnaires and have many stock answers to stock questions. Well-known resources for questionnaires, process management, and preliminary assessment are:

Step 2: Request SaaS Firm Certifications, Standards, and Regulations

Large multi-national SaaS platforms, such as Microsoft and, maintain and publish a long list of security and privacy certifications—credentials that small and even mid-sized SaaS firms often cannot afford. The security materials made available under non-disclosure agreement from smaller players tend to include:

  • AICPA SOC 1 and 2 – keep in mind that management may select the scope of the audit. Also be aware of the common myths about service organization controls.
  • Most recent application penetration testing results or summary.
  • Security policy and security architecture.
  • Assessment results to a security framework such as ISO/IEC 27001 or NIST Cybersecurity Framework (there is a difference between being ISO certified and having a 3rd party assessment using ISO/IEC 27001/2 as a basis).

When materials such as SOC reports and 3rd party questionnaires are combined with the business context, you have what is needed for an onsite assessment.

Step 3: Preparing for an Onsite Assessment

Your request for an onsite visit to review the security questionnaire may generate responses like “our stuff is in the cloud, there is really nothing much to see at our operations.” You may need to insist that meeting their key players and seeing the operations is important to you and your team.

Your best participants in an onsite review are your cyber risk experts who routinely evaluate 3rd party risk. They should include your technical specialist(s) who is most familiar with the risk controls you may want to review, cloud application security, and secure development lifecycle.

In preparation for an onsite visit, the following resources are valuable to scope questions and guide technical control discussions.

  • Cloud Security Matrix (CSM) which maps key controls to a plethora of applicable standards like SOC, NIST-800-53, ISO27001/2, HIPAA / HITECH, and PCI DSS.
  • Applicable standards related to sensitive data. For example, if the system is going to process the PII of European Union Data Subjects, GDPR focus is important.
  • CAIQ provides a series of audit style questions for CSM control domains.
  • Open source intelligence on the firm and its principals.

Often, companies do not have the time or skill-level to conduct interviews and thoroughly vet vendors. In these cases, most partner with a 3rd party cyber security expert, like Revolution Security, to assist or fully manage security assessments and vendor evaluations. Given our deep expertise, our security consultants and technical team members often provide a deeper, more informative assessment than internal IT teams. If you are evaluating SaaS vendors for your business, contact us today to learn how we can assist you.

Contact us today to schedule a meeting with our SaaS security specialists.

Request a Meeting


Cyber Security and Your SaaS Ecosystem—Part 1
Cyber Security and Your SaaS Ecosystem—Part 2