We gave an AI agent permission to attack our test application.

Live application security assessment·15 min read·As assessed September 2026

This is not an independent penetration test. We commissioned it, chose the scope, and reviewed the result ourselves. It was performed by Astra with extra-high effort. At our early stage, this is the strongest practical adversarial assessment we can run and repeat today: real hostile requests against an isolated non-production deployment, with the method, exclusions, and outcome published for scrutiny.

AI-assisted penetration assessment

This was performed by Astra with extra-high effort under Daybreak Trusted Cyber Access. We authorized the work, supplied isolated test tenants and credentials, and limited it to a non-production deployment.

The agent reviewed our test coverage and relevant source, then sent live hostile requests through browser sessions, the REST API, and MCP/OAuth. A human reviewed this public version. We removed target, fixture, and credential detail, but did not change the outcome or turn an excluded area into a pass.

This is a point-in-time internal assessment from September 2026. It is not a certificate or an independent third-party opinion.

The result

No confirmed exploitable vulnerabilities in the tested scope

149
API route modules mapped
199
unauthenticated probes
5
authorization boundaries
0
confirmed vulnerabilities

The agent did not reproduce an authentication bypass, a cross-tenant read or write, role escalation, a cross-site request forgery bypass, stored script execution, credential-scheme confusion, unsafe file acceptance, or MCP authorization bypass. Temporary data was removed and temporary credentials were revoked when the run finished.

How to read this result

Zero findings means none of the tested attacks worked. It does not mean the application has no vulnerabilities. The result belongs to one deployment, one date, the roles and plans we exercised, and the limits listed below.

The scope

From public routes to five authorization boundaries

We wanted the test to follow the paths a real attacker would care about, not just repeat a source review. The agent started with no credentials, then worked through owner and member browser sessions, Contributor and Auditor REST credentials, an edit-granted Restricted collaborator, and an OAuth-authenticated MCP connection across two fictional organisations.

Production-equivalent, fully separated

This was a production-equivalent deployment, not a reduced test harness. The environment uses the same application codebase, build path, runtime, middleware, and service topology as production. Changes can reach it before production promotion, so the exact deployed release may be newer. Its Worker, databases, queues, session store, evidence storage, service bindings, secrets, and test data are separate. Production itself was not probed.

SurfaceIncludedWhat we were looking for
Unauthenticated web surfaceTestedData exposure, internal routes, authentication-adjacent paths, unsafe error order, permissive CORS, sensitive files, and missing browser protections.
Browser sessionsTestedOwner and member sessions, cookie tampering and precedence, credential confusion, CSRF, stored script injection, object access, and tenant isolation.
REST APITestedContributor and Auditor bearer credentials, permission enforcement, cross-tenant access, malformed input, body limits, CORS, and immediate token revocation.
Restricted collaboratorTestedAn edit-granted observation, nested resources, object ownership, ordinary update validation, and stronger actions that stayed outside the grant.
MCP and OAuthTestedDiscovery, consent integrity, scope narrowing, JSON-RPC framing, tool authorization, token rotation, replay detection, and revocation.
ProductionExcludedProduction itself was excluded and not probed. The test environment mirrors its deployment architecture while keeping every resource, credential, and data set separate.
The method

Coverage review first, live probes second

The run started by mapping 149 deployed API route modules and comparing them with our existing smoke suite. That suite already exercises the core sign-in, authorization, tenant isolation, validation, API-token, OAuth, MCP, upload, billing, and background-job paths. The useful question was where a passing happy-path check still left an attacker-shaped gap.

The agent then used those gaps to drive 199 unauthenticated method and path probes followed by authenticated requests. It compared valid controls with hostile variants, checked the stored state after a denied write, and repeated tenant tests in both directions. When a response did not match the first expectation, it checked the documented contract and the postcondition before calling it a flaw.

  • Prove the boundary, not the status code. A denied write was followed by an owner-side read to confirm that no mutation happened. A foreign read was compared with the same request for an owned object.
  • Use two tenants. Object isolation was tested in both directions between separate test organisations, including nested and wrong-parent resources.
  • Keep credentials separate. Browser sessions, API tokens, and OAuth access tokens were deliberately presented to the wrong authentication seams to look for scheme confusion.
  • Render dangerous content for real. Stored script payloads were opened in an authenticated Chromium browser. Looking only at response text would not prove whether the browser executed them.
  • Clean up and reconcile. Temporary records were deleted, temporary tokens and grants were revoked, and the final state was checked for leftover test markers.
Browser and session testing

Browser sessions, isolation, and hostile input

AreaResultHow it was tested
Unauthenticated exposurePASSA 199-request inventory across 149 deployed API route modules covered protected account, organisation, observation, security, billing, integration, and malformed-request paths. Protected routes stayed protected.
Authentication-adjacent exposurePASSPublic authentication-adjacent paths and malformed requests returned bounded, generic responses. No protected data or secret-bearing diagnostic output appeared.
Sensitive-file exposurePASSCommon sensitive and diagnostic paths did not return protected content, source material, or secret-bearing responses.
Browser sessionsPASSOwner and member sessions were combined with foreign authorization headers, tampered cookies, fallback cookie names, and duplicate cookie values. No header override, downgrade, or cross-scheme acceptance appeared.
Tenant isolationPASSTwo-tenant checks covered observations, comments, relations, evidence, inline images, search, filters, audit visibility, wrong-parent paths, and mixed-tenant operations in both directions.
Cross-site request forgeryPASSTwelve checks used hostile and null origins, hostile referers, JSON and multipart bodies, required request markers, valid controls, and post-write state checks.
Stored script injectionPASSA real authenticated browser rendered script tags, event handlers, executable links, embedded document and vector content, title markup, and an external image request. Nothing executed or fetched.
Security-object ownershipPASSCross-user operations against passkey and MFA method identifiers were denied without changing the target object.
Outbound target validationPASSPrivate, loopback, over-broad, insecure, falsely attested, and wrong-type scan or identity-provider targets were rejected and were not saved.
Tenant boundary

The object tests went past the top-level record

A top-level observation check is not enough. Multi-tenant mistakes often hide one level down, where an endpoint confirms the parent but trusts a child ID, or where a bulk operation filters some objects and forgets others. The run therefore used foreign IDs as both the parent and the child across comments, relations, evidence, images, search, filters, and deletion.

Result

Foreign and wrong-parent requests returned a generic absence or an empty tenant-scoped result. A mixed bulk request changed only the caller's own record. The foreign record survived. Search, filters, and audit output did not disclose the other tenant's marker.

A design decision we changed

Shared means explicitly shared

The test noted that a Restricted collaborator who could open one observation could also see relation metadata for linked observations they had not been granted. At the time, we treated that as by design: the relational context was considered part of the whole observation, while opening the linked observation itself remained denied.

On review, that boundary was too broad. Restricted collaborators now see a relation only when they have explicit access to both the observation they are viewing and the linked observation. Relations to ungranted observations, including their titles and risk metadata, are omitted from their view.

See the resulting observation

We recorded and mitigated this design issue in SecurityTrackr itself. Open the three-page security observation (PDF) to see the product-generated export we use internally.

REST API

Bearer authentication and permissions

A short-lived Contributor token was created for the test. The first matrix used missing credentials, the wrong authentication scheme, an empty bearer value, an encrypted browser-session value presented as a bearer, an unknown well-formed token, the right token identifier with the wrong secret, and the valid credential. It also combined a valid bearer with a browser cookie and sent the API token to session-only and MCP routes.

  • Malformed bearer values looked the same. Invalid credentials returned one bounded unauthorized response shape. The response did not reveal whether a token identifier existed.
  • The permission set stayed narrow. The token could read and update ordinary register content within its role. Governance decisions, comment writing, evidence writing, and the SIEM feed remained unavailable.
  • Browser and API credentials did not merge. A session cookie did not add authority to the bearer request, and the bearer did not enter browser-session or MCP boundaries.
  • Cross-origin access stayed closed. Hostile-origin requests and preflights did not receive permission for a browser to read authenticated API responses.
REST API

Objects, validation, and Credential revocation

The same two-tenant fixtures were exercised through the public API. Lists and searches were checked for foreign records; direct reads and updates used foreign IDs; relations used owned and foreign targets; comments and evidence were tested against the Contributor and Auditor permission boundaries. Invalid statuses, risk values, cursors, IDs, searches, JSON shapes, and an oversized body were sent to the relevant endpoints.

ControlResultObserved behavior
Object isolationPASSOwn records worked. Foreign reads, updates, relation targets, and wrong-parent deletes revealed nothing and caused no mutation.
Permission enforcementPASSRead permissions did not become write permissions. Auditor requests could not patch or ingest, and Contributor requests could not delete observations, accept risk, write comments, or upload evidence.
Validation and size boundsPASSMalformed values and shapes produced bounded client errors. Oversized input hit the documented request-size control rather than entering application logic.
Credential revocationPASSThe other tenant could not revoke the token. The owning tenant could, and the very next API request was unauthorized. No reusable secret appeared in token history.
MCP and OAuth

The agent connection was tested as a protocol, not just a login

The run used an existing authorized MCP connection for tool behavior and temporary read-only OAuth grants for consent, rotation, replay, and revocation tests. It did not modify the existing long-lived connection credential.

AreaResultHow it was tested
Discovery and challengePASSPublic metadata, the unauthenticated bearer challenge, protocol initialization, negotiation, and ping were checked end to end.
Tool cataloguePASSEvery advertised tool name was unique and every input schema rejected undeclared properties.
JSON-RPC boundaryPASSUnknown methods and tools, bad argument shapes, true notifications, batches, null, malformed JSON, oversized bodies, hostile origins, and a rate-limited burst were exercised.
Tool authorizationPASSOwned reads and updates worked within scope. Foreign objects stayed absent. Missing collaboration scopes were rejected before object lookup, so foreign IDs did not become an oracle.
Consent integrityPASSAn extra unrequested write scope was injected into the consent submission. It did not appear in the issued token, and the read-only grant remained unable to update data.
Rotation and replayPASSWider-scope refresh, wrong-client refresh, valid rotation, reuse of a superseded refresh token, authorization-code replay, and cross-tenant revocation were all tested. Replay killed the relevant credential family.
The numbers

How much was exercised

Evidence setResultWhat it covered
Unauthenticated route inventory199 probes149 deployed API route modules, protected routes, public authentication helpers, internal service paths, webhooks, malformed input, and method handling.
Tenant and nested authorization2 tenantsBidirectional object, nested-resource, search, filter, audit, and bulk-operation checks.
CSRF matrixhostile originsJSON and multipart writes across valid, hostile, missing, and null origin signals.
Live-browser script injectionbrowser testedExecution, dangerous DOM elements and attributes, links, image fetches, and harmless-text controls.
Bearer authentication2 REST rolesMalformed credentials, valid access, cookies, wrong seams, CORS, caching, and rate-limit metadata across Contributor and Auditor boundaries.
API, MCP, and OAuthexpanded matrixAuthorization, validation, protocol framing, consent, scope, rotation, replay, revocation, and cleanup.

These numbers show depth, not completeness. Hundreds of probes can all miss the one path nobody thought to ask about. That is why we keep the exclusions beside the result and plan to repeat the run as the surface changes.

The honest part

What this assessment did not cover

The limits below are material. We are publishing them because a report that quietly turns every exclusion into a pass is worse than no report.

  • No tester independence. We authorized, scoped, and reviewed the work. Astra with extra-high effort is capable of adversarial testing, but it is not an independent consultancy with organizational separation from us.
  • No production testing. No request was sent to production. Deployment configuration confirms that the tested environment uses the production application architecture with separate resources. Its exact release can be newer while a change waits for promotion.
  • No denial of service. There was no sustained load, distributed rate-limit exhaustion, or attempt to degrade availability. Size and burst controls were tested only far enough to observe the boundary.
  • Limited roles and identity providers. The run exercised owner and member browser sessions, Contributor and Auditor REST credentials, an edit-granted Restricted collaborator, and narrow OAuth grants. It did not complete every human role, SSO provider, or password, Turnstile, and backup-code ceremony.
  • No destructive or cost-amplifying work. Organisation deletion, billing changes, real malware, arbitrary external targets, and broad AI-spend tests were excluded.
  • No company or supply-chain audit. Cloud account configuration, CI/CD, employee processes, physical controls, dependencies, and the wider network perimeter were outside this application-focused run.
Why publish it

Useful evidence beats an empty trust badge

We would rather show the test we could run today than imply we have an independent report we do not have. The useful parts are reproducible: what was in scope, what attacks were attempted, how success and failure were decided, what was cleaned up, and where the result stops applying. Customers can judge the evidence instead of taking a badge on faith.

We still want an independent third-party assessment when the company stage and customer requirements justify it. This report does not stand in for that independence. It does give us a concrete baseline to automate, repeat, and hand to that future tester.

Next

What we are carrying forward

  • Turn the strongest probes into regression tests. Cookie precedence, CSRF edge cases, nested tenant isolation, bearer error parity, OAuth consent tampering, replay-family invalidation, and malformed MCP bodies should run automatically.
  • Repeat after security-boundary changes. Authentication, tenant authorization, API tokens, OAuth, MCP, evidence handling, and rich-text rendering are the triggers for another live pass.
  • Add an independent view later. Before making a contractual or regulatory claim that requires independence, commission a third-party penetration test and publish its scope and limitations with the same candour.
How this was made

Curated from our internal report for public release. We removed implementation detail that would help an attacker, generalised file references to the controls they point at, and dropped internal tracking IDs. We did not change a single verdict or its reasoning. When we re-run the internal assessment, we refresh this page by hand.

This is the product the report graded.

A purpose-built security register with per-org encryption, mandatory MFA, and EU data residency. Free tier, no card.