Introduction

Microsoft Application Inspector is a software source code characterization tool that helps identify coding features of first or third party software components based on well-known library/API calls and is helpful in security and non-security use cases. It uses hundreds of rules and regex patterns to surface interesting characteristics of source code to aid in determining what the software is or what it does and received industry attention as a new and valuable contribution to OSS on ZDNet, SecurityWeek, CSOOnline, Linux.com/news, HelpNetSecurity, Twitter and more and was first featured on Microsoft.com.

Application Inspector is different from traditional static analysis tools in that it doesn't attempt to identify "good" or "bad" patterns; it simply reports what it finds against a set of over 400 rule patterns for feature detection including features that impact security such as the use of cryptography and more. This can be extremely helpful in reducing the time needed to determine what Open Source or other components do by examining the source directly rather than trusting to limited documentation or recommendations.

The tool supports scanning various programming languages including C, C++, C#, Java, JavaScript, HTML, Python, Objective-C, Go, Ruby, PowerShell and more and can scan projects with mixed language files. It also includes HTML, JSON and text output formats with the default being an HTML report similar to the one shown here.

It includes a filterable confidence indicator to help minimize false positives matches as well as customizable default rules and conditional match logic.

Be sure to see our project wiki page for more help https://Github.com/Microsoft/ApplicationInspector/wiki for illustrations and additional information and help.

Goals

Microsoft Application Inspector helps you in securing your applications from start to deployment.

Design Choices - Enables you to choose which components meet your needs with a smaller footprint of unnecessary or unknowns features for keeping your application attack surface smaller as well as help to verify expected ones i.e. industry standard crypto only.

Identifying Feature Deltas - Detects changes between component versions which can be critical for detecting injection of backdoors.

Automating Security Compliance Checks - Use to identify components with features that require additional security scrutiny, approval or SDL compliance as part of your build pipeline or create a repository of metadata regarding all of your enterprise application.

Contribute

We have a strong default starting base of Rules for feature detection. But there are many feature identification patterns yet to be defined and we invite you to submit ideas on what you want to see or take a crack at defining a few. This is a chance to literally impact the open source ecosystem helping provide a tool that everyone can use. See the Rules section of the wiki for more.

Official Releases

Application Inspector is in GENERAL AUDIENCE release status. Your feedback is important to us. If you're interested in contributing, please review the CONTRIBUTING.md.

Application Inspector is availble as a command line tool or NuGet package and is supported on Windows, Linux, or MacOS.

Platform specific binaries of the ApplicationInspector CLI are available on our GitHub releases page.

The C# library is available on NuGet as Microsoft.CST.ApplicationInspector.Commands.

The .NET Global Tool is available on NuGet as Microsoft.CST.ApplicationInspector.CLI.

If you use the .NET Core version, you will need to have .NET Core 3.1 or later installed. See the JustRunIt.md or Build.md files for more.

Customizing Rules

For customizing the rules used, see the project [wiki] (https://github.com/microsoft/ApplicationInspector/wiki) for additional background on Rules, Tags and more used to identify features which use a systematic hierarchical nomenclature e.g. Cryptography.Protocol.TLS to more easily represent features.

Basic CLI Usage

> dotnet ApplicationInspector.CLI.dll or on *Windows* simply ApplicationInspector.exe <command> <options>

Microsoft Application Inspector

(c) Microsoft Corporation. All rights reserved

ERROR(S):
  No verb selected.

  analyze        Inspect source directory/file/compressed file (.tgz|zip) against defined characteristics

  tagdiff        Compares unique tag values between two source paths

  tagtest        Test (T/F) for presence of custom rule set in source

  exporttags     Export unique rule tags to view what code features may be detected

  verifyrules    Verify custom rules syntax is valid

  packrules      Combine multiple rule files into one file for ease in distribution

  help           Display more information on a specific command.

  version        Display version information.

Examples:

Command Help

  Usage: dotnet ApplicationInspector.CLI.dll [arguments] [options]

  dotnet ApplicationInspector.CLI.dll <no args> -description of available commands
  dotnet ApplicationInspector.CLI.dll <command> <no args> -arg options description for a given command

Analyze Command

  Usage: dotnet ApplicationInspector.CLI.dll analyze [arguments] [options]

  Arguments:
  -s, --source-path              Source file or directory to inspect
  
  -m, --match-depth              First match or best match based on confidence level (first|best)

  -o, --output-file-path         Output file path

  -f, --output-file-format       Output format [html|json|text]

  -e, --text-format              Match text format specifiers

  -r, --custom-rules-path        Custom rules file or directory path

  -t, --tag-output-only          Output only identified tags

  -i, --ignore-default-rules     Exclude default rules bundled with application

  -d, --allow-dup-tags           Output contains unique and non-unique tag matches

  -b, --suppress-browser-open    Suppress automatically opening HTML output using default browser

  -c, --confidence-filters       Output only matches with specified confidence <value>,<value> [high|medium|low]

  -k, --file-path-exclusions     Exclude source files (none|default: sample,example,test,docs,.vs,.git)

  -x, --console-verbosity        Console verbosity [high|medium|low|none]

  -l, --log-file-path            Log file path

  -v, --log-file-level           Log file level [Debug|Info|Warn|Error|Trace|Fatal|Off]

  --help                         Display this help screen.

Scan a project directory, with output sent to "output.html" (default behavior includes launching default browser to this file)

  dotnet ApplicationInspector.CLI.dll analyze -s /home/user/myproject 

Scan using custom rules

  dotnet ApplicationInspector.CLI.dll analyze -s /home/user/myproject -r /home/user/myrules 

Write to JSON format

  dotnet ApplicationInspector.CLI.dll analyze -s /home/user/myproject -f json

Write just unique tags found to file

  dotnet ApplicationInspector.CLI.dll analyze -s /home/user/myproject -f json -t

Tag Diff Command

Use to analyze and report on differences in tags (features) between two project or project versions e.g. v1, v2 to see what changed

  Usage: dotnet ApplicationInspector.CLI.dll tagdiff [arguments] [options]

  Arguments:
  --src1                        Source 1 to compare

  --src2                        Source 2 to compare

  -t, --test-type               Type of test to run [equality|inequality]

  -r, --custom-rules-path       Custom rules file or directory path

  -i, --ignore-default-rules    Exclude default rules bundled with application

  -o, --output-file-path        Output file path

  -k, --file-path-exclusions     Exclude source files (none|default: sample,example,test,docs,.vs,.git)

  -x, --console-verbosity       Console verbosity [high|medium|low|none]

  -l, --log-file-path           Log file path

  -v, --log-file-level          Log file level [Debug|Info|Warn|Error|Trace|Fatal|Off]

  --help                        Display this help screen.

Simplist way to see the delta of tag features between two projects

  dotnet ApplicationInspector.CLI.dll tagdiff --src1 /home/user/project1 --src2 /home/user/project2

Basic use

  dotnet ApplicationInspector.CLI.dll tagdiff --src1 /home/user/project1 --src2 /home/user/project2 -t equality

Basic use

  dotnet ApplicationInspector.CLI.dll tagdiff --src1 /home/user/project1 --src2 /home/user/project2 -t inequality

Tag Test Command

Used to verify (pass/fail) that a specified set of rule tags is present or not present in a project e.g. user only wants to know true/false if cryptography is present as expected or if personal data is not present as expected and get a simple yes/no result rather than a full analysis report.

Note: The user is expected to use the custom-rules-path option rather than the default ruleset because it is unlikely that any source package would contain all of the default rules and testing for all default rules present in source will likely yield a false or fail result in most cases.

  Usage: dotnet ApplicationInspector.CLI.dll tagtest [arguments] [options

  Arguments:
  -s, --source-path          Source file or directory to inspect

  -t, --test-type            Test to perform [rulespresent|rulesnotpresent]

  -r, --custom-rules-path    Custom rules file or directory path

  -o, --output-file-path     Output file path

  -k, --file-path-exclusions     Exclude source files (none|default: sample,example,test,docs,.vs,.git)

  -x, --console-verbosity    Console verbosity [high|medium|low|none]

  -l, --log-file-path        Log file path

  -v, --log-file-level       Log file level [Debug|Info|Warn|Error|Trace|Fatal|Off]

  --help                     Display this help screen.

Simplest use to see if a set of rules are all present in a project

  dotnet ApplicationInspector.CLI.dll tagtest -s /home/user/project1 -r /home/user/myrules.json

Basic use

  dotnet ApplicationInspector.CLI.dll tagtest -s /home/user/project1 -r /home/user/myrules.json -t rulespresent

Basic use

  dotnet ApplicationInspector.CLI.dll tagtest -s /home/user/project1 -r /home/user/myrules.json -t rulesnotpresent

Export Tags Command

Simple export of the ruleset tags representing what features are supported for detection

  Usage: dotnet ApplicationInspector.CLI.dll exporttags [arguments] [options]

  Arguments:
  -r, --custom-rules-path       Custom rules path

  -i, --ignore-default-rules    Ignore default rules bundled with application. 

  -o, --output-file-path        Path to output file

  -x, --console-verbosity       Console verbosity [high|medium|low]

  --help                        Display this help screen.

Export default rule tags to console

  dotnet ApplicationInspector.CLI.dll exporttags

Using output file

  dotnet ApplicationInspector.CLI.dll exporttags -o /home/user/myproject/exportags.txt

With custom rules and output file

  dotnet ApplicationInspector.CLI.dll exporttags -r /home/user/myproject/customrules -o /hom/user/myproject/exportags.txt

Verify Rules Command

Verify a custom ruleset is compatible and error free for use with Application Inspector. Note the default ruleset is already verified as part of the Build process and does not normally require a separate verification.

  Usage: dotnet ApplicationInspector.CLI.dll verifyrules [arguments]

  Arguments:

  -d, --verify-default-rules Verify default rules

  -r, --custom-rules-path    Custom rules file or directory path

  -o, --output-file-path     Output file path

  -x, --console-verbosity    Console verbosity [high|medium|low|none]

  -l, --log-file-path        Log file path

  -v, --log-file-level       Log file level [Debug|Info|Warn|Error|Trace|Fatal|Off]

  --help                     Display this help screen.

Simpliest case

  dotnet ApplicationInspector.CLI.dll verifyrules -r /home/user/mycustomrules

Pack Rules Command

Condense multiple rule files into one for ease in distribution with Application Inspector

  Usage: dotnet ApplicationInspector.CLI.dll packrules [arguments]

  Arguments:

    -d, --pack-default-rules    Repack default rules.  Automatic on Application Inspector build.

    -r, --custom-rules-path     Custom rules file or directory path

    -o, --output-file-path      Output file path

    -i, --not-indented          Remove indentation from json output

    -x, --console-verbosity     Console verbosity [high|medium|low|none]

    -l, --log-file-path         Log file path

    -v, --log-file-level        Log file level [Debug|Info|Warn|Error|Trace|Fatal|Off]

    --help                      Display this help screen.

Simplist case to repack default rules into default or alternate location

  dotnet ApplicationInspector.CLI.dll packrules -d -o /home/user/myproject/defaultrules.json

Using custom rules only

  dotnet ApplicationInspector.CLI.dll packrules -r /home/user/myproject/customrules -o /home/user/mypackedcustomrules.json

Build Instructions

See build.md