QOMPASS AI

AI TRUSTWORTHINESS TEST PLATFORM — PYTHON

Radar

Radar

Version 1.00Source on GitHub

OVERVIEW

Radar is the "Risk Analysis Diagnostic Assessment Resolver" aka "R.A.D.A.R" (GitHub repo description; README subtitle: "Your Nist-Endorsed AI Check-up"): NIST's software test platform for assessing the trustworthy characteristics of AI — valid and reliable, safe, secure and resilient, accountable and transparent, explainable, privacy-enhanced, and fair — supporting the Measure function of the NIST AI Risk Management Framework.

HOW IT WORKS

  1. 1You generate a deployment from the repo's Cookiecutter template (`cookiecutter-templates/cookiecutter-radar-deployment/`) with Cruft, run its `init-deployment.sh`, and start the containerized micro-services with Docker Compose.
  2. 2An experiment is composed as entry points, and entry points as tasks: you write an executable script plus a config file declaring its parameters (the MLflow entry-point construct), where each task is a call to a Python function in the task-plugin registry, ordered by Prefect.
  3. 3You package the entry point into a tarball/zip and submit the job — archive plus parameter choices — to the Testbed REST API, the heart of the architecture, which also registers experiments, logs results and artifacts, and hosts the model registry.
  4. 4A Testbed worker — a managed process in a Docker container provisioned with the job's environment — runs the entry point; task plugins (attacks, defenses, data, estimators, metrics) can be installed and updated with zero downtime.
  5. 5Radar automatically snapshots resources so experiments are reproducible, and tracks the full history of experiments and their inputs (traceable by design).
  6. 6You monitor job status and review results/artifacts through the REST API, the web frontend, or the MLflow tracking dashboard; models can be tested first-, second-, or third-party, including red-teaming in a controlled environment.
How it works, step by step — each step as the project's own documentation states it.

CONCEPTS

RADAR'S PIPELINE — ITS OWN MAPPING ON THE AI RMF MEASURE FUNCTION1 · SCOPEsystem under test1st / 2nd / 3rd-party2 · GENERATEattacks + tasks asentry points (plugins)3 · EXECUTETestbed worker runs itDocker · job environment4 · MEASUREresults + artifactsresources snapshotted5 · REPORTAPI · web UI · MLflowhistory fully traceableTESTBED REST API — SUBMISSION + LEDGERa job is an entry-point archive plus parameter choices, submitted to the API — the heart of the architecturetasks are Python functions in the plugin registry (attacks, defenses, data, estimators, metrics), ordered by Prefectthe API also registers experiments, logs results and artifacts, and hosts the model registry · plugins install with zero downtimeMEASURED AGAINST — THE SEVEN TRUSTWORTHINESS CHARACTERISTICSvalid + reliablesafesecure + resilientaccountable + transparentexplainableprivacy-enhancedfaircharacteristics as the AI RMF names themthe pipeline shape is the project's own — the RMF supplies the Measure function and the characteristics, not this workflow
Radar's assessment pipeline — scope, generate, execute, measure, report — drawn as the project's own mapping onto the Measure function of the NIST AI Risk Management Framework. The workflow is Radar's (from its own docs); the seven characteristics it measures against are the RMF's, as listed in Radar's summary.

CONFIGURATION

OptionDefaultWhat it does
RADAR_SESSION_PROTECTIONstrongSession protection level; must be one of `none`, `basic`, `strong` (invalid values raise)
RADAR_REMEMBER_COOKIE_DURATION`14 * 86400` (14 days, seconds)Remember-me cookie duration; cookie name is `radar_remember_token`
RADAR_REMEMBER_COOKIE_SECUREunset (cookie Secure flag off unless the variable is present)Presence of the variable enables the Secure remember-cookie flag
RADAR_LOGIN_DISABLEDunset (login enabled unless the variable is present)Presence of the variable disables login
RADAR_CORS_ORIGINhttp://localhost:5173CORS origin for the REST API (the frontend dev origin)
RADAR_PLUGINS_BUCKETpluginsObject-storage bucket for task plugins
RADAR_SWAGGER_PATH/Path for the Swagger API docs
RADAR_BASE_URLunset (`None`)Base URL of the deployment
`RADAR_RESTAPI_DATABASE_URI` (prod)sqlite:///<cwd>/radar.dbProduction SQLAlchemy database URI
`RADAR_RESTAPI_DEV_DATABASE_URI` (dev)sqlite:///<cwd>/radar-dev.dbDevelopment SQLAlchemy database URI
RADAR_DEPLOY_SECRET_KEYnot reproduced (a placeholder default exists in source)Flask secret key for the dev/prod/test configs
RADAR_BRANCHexample `main`Branch the deployment template is checked out from in the Running Radar quickstart (`export RADAR_BRANCH=main`)
`datasets_directory` (Cookiecutter prompt)not stated in sourceTemplate variable for where datasets are stored; the guide recommends setting it when generating the deployment

CONTROLS

ActionControl
Generate a deployment`cruft create https://github.com/usnistgov/radar --checkout $RADAR_BRANCH --directory cookiecutter-templates/cookiecutter-radar-deployment` (URL as written in the guide)
Initialize and start a deployment`cd radar-deployment` (default folder name), run `init-deployment.sh`, then start the services (Docker Compose files generated by the template)
Submit a jobUpload the entry-point archive and parameter choices to the Testbed REST API
Monitor a job / review resultsQuery the Testbed API, use the web frontend, or open the MLflow dashboard
Work in the web frontendViews for Experiments, Entry Points, Jobs, Queues, Plugins, Models, Artifacts, Tags, and Groups
Extend the testbedWrite task plugins / entry points in Python via the SDK and plugin registry (Prefect-ordered tasks)

NOT COVERED

- This is a large upstream NIST codebase (the docs still carry the NIST copyright header and name the platform Dioptra's successor terminology in places); Qompass-specific changes versus upstream were not identified from the files read, and the README itself is boilerplate beyond the title/badges. - The Running Radar guide's later sections (template prompt-by-prompt descriptions, initialization details) were read only in part; Cookiecutter prompt names/defaults other than `deployment_name` (default folder `radar-deployment`) and `datasets_directory` are not tabulated. - Worker/queue configuration (RQ workers under `src/radar/rq/`, MLflow/MinIO/Postgres service settings in the deployment template's env files) was not enumerated for this pack. - The frontend view names in CONTROLS come from the repo file listing, not from a read of the router/UI source.

REFERENCES

  1. NIST AI Risk Management Framework 1.0 (AI RMF 1.0)The framework whose Measure function Radar supports and whose seven trustworthiness characteristics the pipeline measures AI systems against.
  2. NIST AI 100-2e2025 — Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and MitigationsNIST's current adversarial machine learning taxonomy: the attack vocabulary behind the attack task plugins in Radar's plugin registry.