-
Notifications
You must be signed in to change notification settings - Fork 328
1516 create documentation for invoke scubacached #1813
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
1516 create documentation for invoke scubacached #1813
Conversation
…tps://github.com/cisagov/ScubaGear into 1516-create-documentation-for-invoke-scubacached
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Went through all the examples in scubacached.md and verified everything worked as expected.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall. Could we add one more example for how Invoke-SCuBACached could be used when testing Rego on injected settings that can't be tested in a live tenant?
We could use something like the authentication policy migration settings for AAD 3.4 as an example which @MichaelHicks-MSFT previously worked on in PR #1778. Testing for this PR involved using Invoke-SCuBACached to modify the JSON output.
Co-authored-by: mitchelbaker-cisa <149098823+mitchelbaker-cisa@users.noreply.github.com>
@mitchelbaker-cisa I added a section for this. please review. @MichaelHicks-MSFT make sure my example makes sense too |
| - [Power Platform](docs/troubleshooting/power.md) | ||
| - [Microsoft Graph](docs/troubleshooting/graph.md) | ||
| - [Network & Proxy](docs/troubleshooting/proxy.md) | ||
| | Topic | Resource | Notes | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the concept of a table but for the nested product-specific issues the format is confusing. Can we switch back to the list format?
- [Multiple Tenants](docs/troubleshooting/tenants.md)
- [Product-Specific Issues](docs/troubleshooting/)
- [Defender](docs/troubleshooting/defender.md)
- [Exchange Online](docs/troubleshooting/exchange.md)
- [Power Platform](docs/troubleshooting/power.md)
- [Microsoft Graph](docs/troubleshooting/graph.md)
- [Network & Proxy](docs/troubleshooting/proxy.md)
- [Cached Execution](docs/execution/scubacached.md)
| Invoke-SCuBACached -ProductNames aad -OutPath "C:\ScubaResults\AAD_Policy_Test" | ||
| ``` | ||
|
|
||
| 2. Manually modify the `ProviderSettingsExport.json` file to inject test scenarios (example): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While ProviderSettingsExport.json can be used, it is the legacy file option. Switch this to ScubaResults_<guid>.json instead.
|
|
||
| When testing Rego policies against specific configuration scenarios that can't be replicated in a live tenant, you can modify the exported JSON data and use cached mode to test the policy logic. | ||
|
|
||
| For example, when testing AAD authentication policy migration settings |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could define a couple different scenarios here.
For example, when testing AAD authentication policy migration settings:
// Update the "PolicyMigrationState" property to test with other values such as "preMigration", "migrationComplete", or "migrationInProgress"
{
"authentication_method_policy": {
"AuthenticationMethodConfigurations@odata.context": "https://graph.microsoft.com/beta/$metadata#policies/authenticationMethodsPolicy/authenticationMethodConfigurations",
"PolicyVersion": "1.5",
"PolicyMigrationState": null,
"DisplayName": "Authentication Methods Policy"
}
}
Or when testing AAD conditional access policy settings:
// Update the "State" property to test with other values such as "Report only" or "disabled"
{
"conditional_access_policies": [
{
"DisplayName": "Test Migration Policy",
"State": "enabled",
"Conditions": {
"Users": {
"IncludeUsers": ["All"]
}
},
"GrantControls": {
"BuiltInControls": ["mfa"]
}
}
]
}
|
|
||
| `Invoke-SCuBACached` is a specialized version of the main ScubaGear function that allows you to run security assessments in two distinct modes: | ||
|
|
||
| 1. **Full mode** - Works identically to `Invoke-SCuBA` (exports provider data, runs Rego analysis, and generates reports) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not exactly. One thing Invoke-Scubacached does different than Invoke-SCuBA even when run regularly is how the output artifacts are grouped together.
When Invoke-SCuBA is executed a directory folder of the form M365BaselinConformance_{datetimestamp} is generated in the file path specified by the OutPath parameter. The ScubaGear outputs (json, html) files are put under this folder.
When Invoke-ScubaCached is executed the directory folder (M365BaselineConformance) is NOT created and the SCuBA outputs (JSON, HTML files etc.) are put directly into the file path specified by OutPath.
The fact a M365BaselineConformance folder is not created by Invoke-ScubaCached was it to make it easier to do rego develpment without having to rerun the providers.
Run full mode once to pull in a provider JSON.
Then on each subsequent run just run in cached mode. The user doesn't need to mess with OutPath.
| ### Development and Testing | ||
|
|
||
| - Use cached mode during development to speed up iteration cycles | ||
| - Keep a "golden" export for consistent testing across different configurations | ||
| - Use descriptive output folder names to organize different test scenarios |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specifically for Rego development. This was what the function was orginally made for.
A developer would be able to work on the rego then run ScubaCached to quickly see their rego changes on the HTML report. Without having to wait for the entire provider to run.
Our own main use case for this this function right now is the functional test pipeline. The code for the pipeline makes HEAVY use of this function to manipulate cached JSON to test SCuBA baseline policies that can't be tested safely. Such as manipulating MFA conditional access policies.
Add documentation for Invoke-ScubaCached
🗣 Description
This PR introduces documentation for the Invoke-ScubaCached command.
The new docs describe what the command does, its parameters, usage scenarios, and how it differs from Invoke-Scuba.
Documentation includes:
💭 Motivation and context
While
Invoke-ScubaCachedhas been available as part of ScubaGear for some time, its functionality was not documented.This created confusion for users on:
Invoke-Scuba(live scan) andInvoke-ScubaCached(cached evaluation).Adding documentation addresses these gaps and supports user adoption, particularly for environments with strict API throttling or offline validation requirements. This documentation also provided guidance for Developers and those troubleshooting ScubaGear results.
This resolves [Issue #1516]
Closes #1516
🧪 Testing
Validation steps performed:
Steps for reviewers to test the docs:
✅ Pre-approval checklist
✅ Pre-merge checklist
PR passed smoke test check.
Feature branch has been rebased against changes from parent branch, as needed
Use
Rebase branchbutton below or use this reference to rebase from the command line.Resolved all merge conflicts on branch
Notified merge coordinator that PR is ready for merge via comment mention
Demonstrate changes to the team for questions and comments.
(Note: Only required for issues of size
Mediumor larger)✅ Post-merge checklist