Take Stock of Your Environment

Evaluate what you have already or what you will have soon due to planned purchases or upgrades. It’s important to start with the products you already have, so you can get the most out of your pilot.

You'll want to think about these areas:

1. Evaluate existing or planned tools for integration What tools do you already have deployed in your environment? Do you already have upgrades or purchases in the works? The Product Integration tab in the S-PET can help you take stock.
2. Evaluate existing or planned input data What data and data feeds do you have or have planned? Where is it stored or accessed? What format(s) is it in? What tools can use it?
3. Analyze existing or planned processes What are your well-established, well-understood processes? Are there any updates in the works? What tools/data do you need for each process? What compliance and regulatory policies and practices are required? Does your documentation include roles, responsibilities, decision criteria and dependencies on other tools, organizations or personnel?
4. Evaluate security needs and implications What are your security needs? What security policies do you need to comply with? What execution and access privileges will your tools and processes require? How should these be managed and enforced? Do you need to limit access to specific data? How will you authenticate and authorize tools and processes? How will you protect and manage credentials, access control lists, etc.?

Helpful Information:

• The Product Integration tab in the S-PET tool can help you take inventory of your current and planned environment.
• The Introduction to IACD Playbooks and How to Build an IACD Playbook whitepapers help you identify what you need to describe in your processes and playbooks,
• Additional information and tutorials about building the playbooks can be found in the IACD Playbooks and Workflows section.

Define Processes and Workflows

Next, you'll want to select a process to pilot and develop the playbooks and workflows you'll need for it.

1. Select initial process
   You should choose a process that's well-understood by your personnel, and has well-defined decision-making criteria. You should also think about the tools and data you'll need to support that process–make sure you'll be able to access and run it in your pilot environment. If you need to use sensitive or protected information, make sure that your tools and environment (including cloud-based storage and processing) are approved for your type of data.
2. Define playbooks and workflows including human oversight
   As you develop your playbooks workflows, make sure to identify and include the workflows and steps where people may need to check on what's happening--you're getting ready for a pilot, so you will probably need to have some extra scrutiny. And be sure to think about trouble shooting–if something unexpected happens, how will you figure out where the problem might be?
3. Iteratively refine processes, playbooks and workflows
   Plan for a few iterations of your process, playbooks and workflows. As you break it down and map it into your tools, you'll find areas that aren't as well-defined or consistent as you thought. You’ll also want to think through how you’ll implement your required security needs.

Helpful Information:

• The High-Benefit/Low-Regret Automated Actions as Common Practice whitepaper can help you decide which processes you might consider automating.
• The Introduction to IACD Playbooks and How to Build an IACD Playbook whitepapers help you identify what you need to describe in your processes and playbooks, and the Playbook Thin Specification provides details.
• Additional information and tutorials about building the playbooks can be found in the IACD Playbooks and Workflows section.
• For suggestions from experiences from other SOAR adopters, take a look at Implementer Insights, Operationalization Lessons Learned, and the IACD & FS-ISAC Financial Pilot Results.
• To see examples of orchestration you can watch the Orchestration Example: Automated IT/OT Recovery and Advanced Orchestration Techniques: Reversibility videos.

Select the Right SOAR Tool

If you haven't selected any SOAR tools already, now's the time to do it. It's not a good idea to expend all the resources you'll need for a pilot on the wrong tools–you've got a good handle on your environment and have a good set of processes and workflows you can use for evaluation. With a little homework, you can do the evaluation you need to make a good decision now.

1. Evaluate tools based on essential and desired characteristics
   You'll want to choose tools that are right for your needs and your environment. You can use the S-PET tool to help you decide what's most important to you, then talk to potential vendors to see which tools might meet your needs. Even if you’re using your own internally developed and supported environment and tools, you can still evaluate current and proposed features to ensure you have the best fit for your needs.
2. Assess security implications
   Be sure to think through the security implications. Your tools and processes will need access to potentially sensitive data, services, and other tools. They will need to comply with and implement your required security policies, and will need to use authentication and authorization methods approved for your organization.
3. “Test drive” tools in your environment with your data
   Once you've identified one or more tools that might meet your needs, it's a good idea to "try before you buy"–an in-house evaluation can help you avoid surprises later.

Helpful Information:

• The S-PET tool can help you identify which tool characteristics are most important to you.
• The Asking the Right Questions whitepaper and the Orchestration Thin Specification can help you identify which tools have the features you need.
• The ICD Conceptual Reference Model and IACD Baseline Architecture whitepapers help you identify how an orchestrator will fit into your environment,
• The Orchestration Tool Test Drive Guidance can help you plan and execute an evaluation for your environment.

Select Pilot Products and Data

Now you can start putting all the pieces together for your pilot. Exactly what do you need for your pilot? What are the key aspects of your situation that you need to evaluate?

1. Select orchestrator
   Based on the evaluation you've done, choose the right orchestrator for you.
2. Select products and tools to be integrated
   What tools and data do you need to integrate with your orchestrator in order to execute the processes and workflows you've defined? Think carefully about matching your scope to your available resources. Maybe you don't need to do all of the data, or use all of the tools–you can limit your risks now and incrementally expand later. Be sure to think about which tools and data can run in your pilot environment–do you have the licenses you need? Required access control and protection in place?
3. Select or generate data
   What data will you use, and where will it be stored? Are you using live data? Do you need to separate it from your normal operational data? Do you have enough storage space allocated? Be sure to consider interim processing and results in addition to input and output data.
4. Define pilot environment
   Once you’ve figured out the tools and data you’ll be using for your pilot, you should define your pilot environment. Do you need a separate, isolated environment for new tools (potentially not approved for your network yet) or sensitive data? Are you using your live data and tools? Do you need to interface with external tools and data? How will you implement those interfaces for the pilot? Will you need to duplicate those interfaces for your pilot environment? How will you collect metrics and measures?

Helpful Information:

• The S-PET tool, including the Product Integration tab, can help you keep track of the characteristics are most important to you, and the tools and data in your current and planned environment that will provide them.
• The ICD Conceptual Reference Model and IACD Baseline Architecture whitepapers help you categorize and align your tools with your broader SOAR architecture,
• The Orchestration Tool Test Drive Guidance can help you plan and execute product evaluations for your environment.
• If you plan to use or produce Indicators of Compromise (IOCs), the Information Sharing under IACD page as well as the Actionable Information Sharing overview, and the AIS Fact Sheet provide more details specific to IOCs.
• If you plan to use or produce other threat information, the Autoimmunity whitepaper and Autoimmunity video can assist.

Plan Pilot

And finally, you can plan your pilot. Set expectations, identify the resources and training you'll need, and get any approvals in place. At a minimum, consider the following:

1. Define measures of success
   What does success look like? What can you measure and how will you measure it?
2. Define roles and responsibilities
   Who will plan, execute, and do analysis for (both during and after) your pilot? Be as specific as possible so you can estimate the resources you'll need, and what training, access, and limitations they might have. How will you balance pilot needs and operational needs? Be sure to include support personnel, operations personnel, and management and oversight, as well as pre-pilot set-up, execution and post-pilot analysis.
3. Prepare facilities, tools, and hardware
   You'll not only need servers, software, and rack space. You'll also need places to sit, analyze data, and evaluate and adjust pilot activities. What installation and integration activities do you need? Who will perform them? What needs to be done before, during and after your pilot?
4. Prepare personnel (operations and engineering support)
   Identify all the personnel you'll need, including engineering support, and make sure they will be available when you need them.
5. Train Pilot Personnel
   Define the training (including tool support training) that you'll need. Make sure you have the training packages ready when you need them. Define schedule and methods for training all the personnel who will be participating in the pilot and ensure training is completed before pilot begins.
6. Implement security and isolation
   Depending on your piloting needs, you may need special considerations for your pilot. You may have outside personnel, particularly vendor support, that will need access to your facilities and possibly your systems. You may need to strictly isolate your pilot environment from your operational environment.
7. Define Outputs and Reporting
   Identify what results you intend to document, how you will document them, and to whom these will be communicated. How will you present results? What aspects will you report? What comparisons will you make? You may need to capture some baseline data before you start your pilot.

Helpful Information:

• For more information about measuring success, see the Security Automation and Orchestration Metrics and Measures.
• For suggestions from experiences from other SOAR adopters, take a look at Implementer Insights, Operationalization Lessons Learned, and the IACD & FS-ISAC Financial Pilot Results.

Ready to Pilot

Now you're ready to pilot SA&O. You should have the following:

1. Buy-in within your organization
2. Processes and workflows defined
3. Tools and data installed and configured
4. Operations and support personnel allocated and trained
5. High level understanding of your final product

At a minimum, your plan for piloting should include:

• Operational modes and constraints
  • relationship to current operations (integrated, standalone, parallel operations)
  • duration and operating periods
• Personnel
  • required skills, expertise and training identified
  • schedule and methods to train all personnel defined
  • appropriate personnel identified and allocated to pilot
  • management reporting and decision-making requirements identified and documented
• Processes and Procedures for pilot
  • orchestration playbooks identified and baselined
  • pilot workflows developed
  • risk management and service restoration procedures defined (e.g., backup/recovery, fallback/rollback, etc.)
  • support procedures and resources defined and verified (e.g., technical support contact lists, anticipated O&M remote access, call in lists, etc.)
  • SA&O status/notifications integrated (and clearly distinguishable) with organization announcement processes
• Operational process and procedure modifications for pilot
  • metrics determined and measurement points identified
  • security requirements developed and approved
  • modifications to current systems and processed identified and implemented
  • "failsafe" mechanisms identified and verified to be working
• Systems and Tools
  • Products, tools and equipment updated and baselined, with license numbers and expiration dates documented
  • Database and backup synchronization defined and verified
  • Processing, memory and storage capacity (including surge capacity or 'operating margin') estimated, allocated and monitored for all clients and servers
  • Monitoring of SA&O pilot and affected systems documented and verified (displays, status indicators, critical thresholds/limits identified, etc.)
  • System resiliency and restoration capabilities (e.g., failover) documented and verified
• Interfaces
  • IT System Interfaces (sensors, tools, monitoring, etc.)
  • Input data (e.g., threat/reputation feeds)
  • Sharing/output data