Abstract

Autonomous AI agents are transitioning from experimental tools to economic participants, yet no standardized system exists for evaluating whether an agent can be trusted to succeed on its next task. This paper introduces Predictive Behavioral Reputation (PBR), a scoring specification that computes a 0–100 trust score from five weighted dimensions: Reliability, Safety, Consistency, Efficiency, and Transparency. The statistical foundation is a discounted Beta-Bernoulli power prior with evidence-quality weighting, where each observation's influence depends on its recency, verifiability, and the credibility of the reporting party. Adversarial resistance is achieved through a five-layer economic defense: commissioner credibility learning via HHI diversity analysis, graph concentration monitoring, three-axis resource gating, volume anomaly detection via CUSUM, and behavioral cluster-relative filtering. The framework treats Sybil resistance as an economic problem (Douceur, 2002), making the cost of score manipulation exceed the benefit of inflated reputation. PBR is deployment-topology agnostic, functioning identically as a centralized platform, federated enterprise system, or off-chain oracle for on-chain registries. Protol provides the reference implementation as a Python SDK and REST API. This paper specifies the complete scoring methodology, threat model, and known limitations.

Keywords: AI agent trust, reputation systems, Bayesian inference, Sybil resistance, behavioral scoring, drift detection, agent economy, multi-attribute utility

JEL Classification: D82 (Asymmetric Information), L15 (Information and Product Quality), O33 (Technological Change)

ACM CCS: Security and privacy → Trust frameworks; Computing methodologies → Multi-agent systems; Information systems → Reputation systems

1. Executive Summary

Autonomous AI agents are transitioning from experimental tools to economic participants. They negotiate contracts, execute trades, manage infrastructure, write code, and interact with other agents on behalf of humans and organizations. The global AI agent market is projected to reach $48–53 billion by 2030, with enterprise adoption accelerating across every industry.

Yet there is no standardized system for evaluating whether an AI agent can be trusted. When a developer deploys an agent to manage customer data, when a company integrates a third-party agent into its workflow, when agents begin transacting with each other autonomously, a fundamental question goes unanswered: what is the probability that this agent will succeed on its next task?

This is a classic Market for Lemons problem (Akerlof, 1970). Without observable quality signals, buyers cannot distinguish high-quality agents from low-quality ones. High-quality agents are undervalued, low-quality agents are overvalued, and the market degrades. Credit scores solved this for consumer lending. PBR solves it for the agent economy.

Predictive Behavioral Reputation (PBR) is a scoring specification for AI agent trust. It computes a 0–100 score from five weighted dimensions (Reliability, Safety, Consistency, Efficiency, Transparency), adjusted for behavioral drift and adversarial manipulation. The statistical foundation is a discounted Beta-Bernoulli power prior with evidence-quality weighting: each observation's influence on the score depends on how recent it is, how verifiable it is, and how credible the reporting party is. The result is a posterior predictive probability for the Reliability dimension and a multi-attribute utility index for the composite score. PBR is deployment-topology agnostic: the scoring specification is identical whether deployed as a centralized platform, a federated enterprise system, or an off-chain oracle feeding on-chain registries such as ERC-8004.

Protol is the reference implementation, available as a Python SDK and REST API. Integration requires three lines of code.

2. The Trust Gap in the Agent Economy

Trust in agents currently relies on the reputation of the company that built them, manual testing by integration teams, or trial-and-error in production. None of these scale. Akerlof (1970) formalized why: in markets with asymmetric information, where sellers know the quality of their product but buyers do not, adverse selection drives out quality. Agent marketplaces face exactly this dynamic.

2.1 The Integration Decision

A fintech company evaluating three AI agents for compliance monitoring has no standardized way to compare their reliability in production. Marketing claims, benchmark scores, and demo performances do not predict real-world behavior under diverse inputs, adversarial conditions, and model updates.

2.2 The Agent-to-Agent Problem

As multi-agent systems become common, agents must make trust decisions about other agents in real time. A coding agent delegating security review to a specialized agent needs a machine-readable trust signal. No human is in the loop to vet the delegation.

2.3 The Gaming Problem

Any reputation system that matters economically will be gamed. Sybil attacks (fake agents rating each other), volume stuffing, category manipulation, and selective task intake are not theoretical threats; they are standard practice in every marketplace with economic incentives. Douceur (2002) proved that Sybil resistance in open systems requires either central identity authority or resource-based constraints. A trust framework that ignores this is building on sand.

3. Why Existing Reputation Systems Fail for AI Agents

Human-designed reputation systems (star ratings, review aggregation, social proof) fail when applied to AI agents because agents violate the assumptions those systems were built on:

3.1 Agents Are Deterministic, Humans Are Not

A human service provider has inherent variability. An AI agent's behavior is a function of its model, configuration, and input distribution. Given the same inputs, an unchanged agent produces the same outputs. This determinism means reputation can be more precise than for humans, but also that manipulation strategies are more systematic.

3.2 Agents Drift Silently

When a model provider pushes an update, every agent built on that model changes behavior overnight. The agent's wrapper code hasn't changed. Its configuration hasn't changed. But its output distribution has shifted. Traditional reputation systems have no mechanism for detecting or responding to silent behavioral drift.

3.3 Agents Can Be Cloned at Zero Cost

An agent with a bad reputation can be relaunched under a new identity in minutes. The marginal cost of a new agent identity is effectively zero. This makes identity-based reputation fragile unless backed by economic or temporal constraints.

3.4 Agent Performance Is Context-Dependent

An agent that excels at code review may fail at legal analysis. A single aggregate score is misleading. Trust must be task-conditioned: queried per category, not averaged across all behavior.

3.5 Agents Can Be Probed Without Their Knowledge

Unlike human professionals, agents can be tested with known-answer challenges to verify self-reported performance. This property enables calibration validation that is impossible for human reputation systems.

4. The PBR Framework

PBR computes trust scores from five weighted dimensions, each measuring a distinct aspect of agent quality. The composite score is a multi-attribute utility index (analogous to a credit score): higher is better, and the score is interpretable in relative terms within the PBR population. Only the Reliability dimension has a direct probabilistic interpretation; the composite is an ordinal ranking metric with empirically validated monotonic calibration as the target property.

Table 1: PBR dimension overview

4.1 Statistical Foundation

Three principles govern PBR's mathematics across all dimensions. First, evidence quality weighting: every observation is weighted by how verifiable it is, via Data Integrity Levels that assign higher statistical influence to externally validated evidence. Second, uncertainty penalization: agents with thin evidence receive conservative scores via Lower Confidence Bounds (Reliability) and Upper Confidence Bounds (Efficiency), preventing gaming through small sample sizes. Third, hierarchical shrinkage: when category-specific evidence is sparse, scores blend toward informative baselines (cross-category agent performance and cross-agent category performance) rather than defaulting to extremes.

The Reliability dimension is a posterior predictive probability: its calibration target is that an agent with Reliability = 80 succeeds approximately 80% of the time on its next task. The composite PBR score is a multi-attribute utility index where monotonic calibration (higher score predicts better outcomes) is an empirical property to be validated, not a mathematical guarantee.

4.2 Architecture Overview

PBR scores are conditioned on agent identity and task category. Querying an agent's PBR returns per-category dimension scores, a composite score, a trust tier, and diagnostic metadata (N_eff, integrity distribution, category consistency, percentile rank, scoring model version). All scoring operates on structured action metadata. No task content (prompts, outputs) is analyzed, stored, or transmitted.

4.3 Data Integrity Classification

PBR does not assume trusted telemetry. Every observation is weighted by how verifiable it is. Data Integrity Levels define the evidence quality of each logged action, directly affecting how much that action influences the agent's score across all five dimensions.

Table 2: Data Integrity Levels

Default η values serve as conservative priors. Protol maintains a rolling audit process: random sampling of actions at each integrity level, cross-referencing self-reported outcomes against commissioner ratings and validator confirmations. Values are updated as η_ℓ = 1 - e_ℓ where e_ℓ is the observed error rate at level ℓ.

Evidence Quality Decomposition

Each action j receives a total evidence weight combining time decay, integrity, difficulty, and commissioner credibility:

w_j = d_j × η^(label)_ℓ × η^(telemetry)_ℓ × γ_rater(j) × φ(difficulty_j)

where d_j is exponential time decay, η^(label) reflects outcome correctness at that integrity level, η^(telemetry) reflects metadata reliability, γ_rater is the commissioner's learned credibility, and φ(difficulty_j) is the difficulty adjustment (Section 4.4). In v1, η^(label) = η^(telemetry) = η_ℓ as a simplification; the decomposition is defined for forward compatibility. The difficulty factor φ applies symmetrically to all evidence regardless of outcome.

Commissioner Credibility (γ)

Commissioner credibility combines behavioral and identity signals into a single learned quantity:

γ_u = min(1, w_B × B_u + w_I × I_u)

where BehaviorTrust (B_u) is derived from commissioner diversity, timing realism, and graph position, and IdentityTrust (I_u) is derived from organizational verification, payment method, and account age. Default weights: w_B = 0.8, w_I = 0.2 (behavior-heavy). When identity infrastructure is absent, I_u = 0 and the system functions on behavioral signals alone.

BehaviorTrust is computed from three components. Commissioner Diversity uses the Herfindahl-Hirschman Index (HHI): HHI_u = Σ p²_{u,a} where p_{u,a} is the fraction of commissioner u's ratings directed at agent a. Commissioners with HHI > 0.5 are classified as low-diversity validators with γ capped at 0.3. This is not a Sybil flag; a legitimate enterprise customer rating one vendor produces HHI = 1.0, and their ratings still contribute at reduced weight reflecting limited cross-agent perspective. Temporal Realism measures the fraction of ratings arriving within 60 seconds of action completion; commissioners with more than 80% tight-timing ratings have γ reduced by 50%. Account Age Factor = min(1, age_days / 90) provides gradual trust accumulation. Composite: B_u = 0.5 × diversity_u + 0.3 × timing_score_u + 0.2 × age_factor_u.

Commissioner credibility starts at 0.5 and is updated based on consistency with validated outcomes. An agent's developer and its commissioners must not share organizational identity; violations result in downgrade to L1 for affected actions.

Asymmetric Integrity for Safety

Integrity weighting is asymmetric for safety evidence. Self-reported absence of incidents (η^pos) is weighted lower than reported incidents (η^neg), because under-reporting is a stronger incentive than false reporting:

Table 3: Asymmetric integrity weights for Safety dimension

Incidents with severity > 0.7 from L1 sources are flagged for corroboration rather than auto-applied with discount. L4 is defined for forward compatibility but not required for conforming v1 implementations.

Job-Level Aggregation

Actions can be linked to a parent job via an optional job_id field. When present, commissioner ratings apply at the job level: a commissioner rates whether the overall job succeeded, and that outcome propagates to all subtasks within the job. If subtasks report mixed outcomes but the commissioner rates the job as failed, subtask outcomes are overridden by the job-level assessment. This prevents splitting a failed job into nominally successful subtasks. At L2+, commissioners can file missing action reports indicating an agent performed work but did not log it; these count as negative evidence events with outcome y_j = 0 weighted by the commissioner's γ.

4.4 Dimension 1: Reliability (R)

Reliability is the posterior predictive probability of success on the next task, conditioned on agent identity, task category, and behavioral history. It is the only PBR dimension with a direct probabilistic interpretation.

Statistical Model

For agent a in task category c, with historical task outcomes at current time t, Reliability is modeled as a discounted Beta-Bernoulli power prior (Ibrahim & Chen, 2000). The discounting approximates a locally stationary skill model where the agent's success probability may drift slowly over time; older observations contribute less evidence than recent ones.

Outcome Labels

Each task j produces a soft outcome label y_j that fuses self-reported results with commissioner feedback:

y_j = (ρ_self × o_j + ρ_comm × g(r_j)) / (ρ_self + ρ_comm)

where o_j is the binary self-reported outcome, g(r_j) maps commissioner rating (1–5) to [0,1] via g(r) = (r-1)/4, and ρ_comm = 3, ρ_self = 1. When no commissioner rating exists, y_j = o_j. The fusion gives commissioner input 3x the weight of self-report, creating an incentive for agents to seek external validation.

Time Decay and Self-Calibrating Half-Life

Each observation is weighted by exponential decay with a self-calibrating half-life:

d_j = exp(-ln(2) × (t - t_j) / h_c)

The half-life h_c is determined by the agent's action frequency within category c, not by operator choice:

h_c = clamp(median_inter_action_interval_c × 10, h_min, h_max)

Default bounds: h_min = 7 days, h_max = 90 days. Deployment operators may adjust these bounds at the platform level; per-agent override is not permitted to prevent decay manipulation. High-frequency agents (sub-hourly actions) receive h_c = 7 days, reflecting rapid iteration cycles where yesterday's performance on a pre-patch version is irrelevant. Low-frequency agents (weekly actions) receive h_c = 70 days, preserving the informational value of each rare observation. Safety deduction decay rates (28 days medium, 56 days critical) are fixed per severity level and independent of the Reliability half-life.

Difficulty-Adjusted Evidence Weighting

To reduce the reward for selective task intake (cherry-picking easy tasks), evidence weight includes a difficulty adjustment. Task difficulty is inferred from observable, hard-to-forge signals within each behavioral cluster:

work_score(j) = w_1 × log(1 + duration_ms) + w_2 × tool_calls + w_3 × ¹[external_api_called]

Token counts are included only when reported at L2+ integrity (commissioner-validated); at L1, tokens are excluded to prevent self-reported inflation. Cost is excluded to avoid circularity with the Efficiency dimension. The difficulty percentile is computed within the agent's behavioral cluster:

φ(difficulty_j) = 0.5 + 0.5 × percentile_rank(work_score_j, within_cluster_k)

This maps difficulty [0, 1] to weight multiplier [0.5, 1.0]. An agent succeeding at a 90th-percentile difficulty task gets w_j multiplied by 0.95; one succeeding at a 10th-percentile task gets 0.55. The difficulty factor applies symmetrically to successes and failures, preventing the perverse incentive of deliberately failing hard tasks to accumulate evidence weight.

Limitation: the difficulty proxy is metadata-based and can be gamed by inflating duration or inserting unnecessary tool calls. However, such gaming increases real costs (more compute, more API calls), reducing the Efficiency dimension. This cross-dimensional tension makes simultaneous gaming of both dimensions expensive.

Coverage (Conditional)

When PBR is integrated with routing infrastructure that exposes offer/accept events, a Coverage metric is computed: Coverage(a,c) = completed_tasks / offered_tasks. Coverage enters the evidence weight as a mild modifier: ψ(coverage) = 0.8 + 0.2 × coverage. The 20% range ensures some selectivity is tolerable. Standalone SDK deployments without routing visibility cannot compute Coverage; this is a stated limitation.

Discounted Sufficient Statistics

S_a,c = Σ w_j × y_j F_a,c = Σ w_j × (1 - y_j) N_eff = S + F

where w_j = d_j × q_j × φ_j combines time decay with evidence quality weight and difficulty adjustment.

Hierarchical Prior

Sparse categories are stabilized by shrinking toward informative baselines:

m_a,c = (τ_a × p_a + τ_c × p_c) / (τ_a + τ_c)

where p_a is the agent baseline (all categories), p_c is the category baseline (all agents), and τ_a = min(τ_max, N_eff,a), τ_c = min(τ_max, N_eff,c). Prior strength κ (default: 25) sets the effective pseudo-count:

α_0 = κ × m_a,c β_0 = κ × (1 - m_a,c)

Posterior and Prediction

α = α_0 + S_a,c β = β_0 + F_a,c

Marketplace-facing reliability uses a Lower Confidence Bound (LCB) to penalize agents with thin evidence:

p^LCB_a,c(t) = BetaInv(q; α, β) where q = 0.10

An agent with 2 successes out of 2 attempts has a wide posterior; its LCB is low. An agent with 200 out of 250 has a tight posterior; its LCB is close to the point estimate. This prevents gaming through small sample sizes.

Reliability_a,c = 100 × p^LCB_a,c(t)

What This Claims and Does Not Claim

Reliability is a posterior predictive success probability under the assumption that the agent's category-specific success rate is locally stationary within the decay window, and under squared-error loss. The exponential discounting approximates adaptation to skill drift; the drift detection mechanism (Section 6.2) handles regime changes that violate local stationarity. Calibration (does a score of 80 correspond to ~80% future success rate?) is an empirical property that must be validated via time-split backtesting.

4.5 Dimension 2: Safety (S)

Safety starts at 100 and decreases with confirmed incidents. Unlike Reliability, Safety uses a demerit system rather than a probabilistic model, because safety failures are rare events where even one occurrence is significant.

Table 4: Safety deductions weighted by Data Integrity Level (η_neg)

Safety scores recover over time via half-life decay: 28 days for medium-severity incidents, 56 days for critical incidents. These decay rates are fixed per severity level and independent of the Reliability half-life h_c. Recovery is gradual, not binary: a critical incident that occurred 56 days ago contributes half its original penalty.

Dispute Resolution Interface

Conforming PBR implementations must provide: (1) reporter identity at L2+ integrity, (2) evidence attachment (action_id, timestamp, description), (3) a 72-hour response window for the agent operator, (4) adjudication with recorded reasoning, (5) false report penalty reducing the reporter's commissioner credibility, and (6) one appeal within 14 days reviewed by a different adjudicator.

4.6 Dimension 3: Consistency (C)

Consistency measures behavioral stability. An agent that performs at 95% one week and 60% the next is less trustworthy than one that consistently delivers 85%.

Weighted Variance Estimation

σ²_weighted = Σ(w_j × (y_j - ȳ)²) / Σ(w_j)

N_eff = (Σ w_j)² / Σ w_j²

When N_eff < 10 in a category, the Consistency score shrinks toward the hierarchical prior rather than publishing a confident score from thin data.

Variance Decomposition

To avoid penalizing agents for environmental noise (upstream API latency spikes, infrastructure issues affecting all agents), Consistency separates agent-attributable variance from population-level variance:

σ²_agent = max(0, σ²_observed - σ²_population)

where σ²_population is the variance observed across all agents in the same behavioral cluster during the same time window.

Composite Consistency

V_sr = min(1.0, σ_agent_sr / 0.25) V_rt = min(1.0, σ_agent_rt / μ_rt)

C = 100 × ((1 - V_sr) × 0.7 + (1 - V_rt) × 0.3)

4.7 Dimension 4: Efficiency (E)

Efficiency compares an agent's cost against behavioral peers. To prevent category arbitrage, PBR determines comparison groups from observed behavior, not self-reported labels.

Behavioral Peer Group Assignment

Each action is assigned to a behavioral cluster via the Behavioral Fingerprinting mechanism (Section 6.1). Clusters are defined by action metadata: log-transformed duration, log-transformed token count, tool call count, model/tool mix, and input complexity tier. Cost is excluded from peer selection to prevent circularity. Self-declared task categories are retained for API queryability but are non-authoritative for efficiency scoring.

For agent a in behavioral cluster k, the comparison peer set consists of all other agents with N_eff ≥ 5 in cluster k. If the peer set has fewer than 5 agents, cluster-level statistics are shrunk toward the global baseline:

peer_cost_k = (N_eff,k × μ_cost,k + τ_global × μ_cost,global) / (N_eff,k + τ_global)

Uncertainty-Penalized Efficiency

agent_cost_UCB = exp(mean_log_cost_a + z_q × SE_log_a)

E_a,k = min(100, (peer_cost_k / max(agent_cost_UCB, ε)) × 100)

where SE_log_a is the weighted standard error of log-costs and z_q = 1.28 (for q = 0.10). Agents with sparse cost data receive conservative efficiency estimates. When N_eff < κ, efficiency shrinks toward the global baseline (default: 50), not toward 100.

Integrity Gating

Efficiency's contribution to the composite PBR Score is multiplied by the agent's average integrity weight. An all-L1 agent's efficiency contributes 60% of its face value. This prevents gaming by staying in L1 to appear cheapest among the unverified.

Category Consistency Detection

For categories with ≥ 5 agents, PBR computes the Mahalanobis distance (Mahalanobis, 1936) between each agent's log-transformed action profile and the category population profile, using Ledoit-Wolf shrinkage covariance for numerical stability. Agents with MD > 3.0 are flagged in API responses with a category_consistency indicator.

4.8 Dimension 5: Transparency (T)

Transparency rewards agents that provide rich metadata about their actions and whose self-reported confidence is both well-calibrated and informative.

Metadata Completeness

MC = (fields_provided) / (total_possible_fields)

Confidence Calibration and Sharpness

Calibration is measured using the evidence-weighted Brier score (Brier, 1950), a strictly proper scoring rule with no free parameters:

BS_weighted = Σ(w_j × (confidence_j - y_j)²) / Σ(w_j)

Calibration = 1 - BS_weighted

Calibration alone is insufficient: an agent reporting confidence = 0.5 on every task achieves Brier = 0.25 (Calibration = 0.75) while being completely uninformative. To penalize chronic hedging, Transparency includes a Sharpness component:

Sharpness = mean(|confidence_j - 0.5|) ∈ [0, 0.5]

An agent always reporting 0.5 gets Sharpness = 0. An agent reporting confident predictions (0.1, 0.9, 0.95) that are also well-calibrated gets high Sharpness. Normalized to [0, 1] by dividing by 0.5.

Composite Transparency

Cal_component = 0.7 × Calibration + 0.3 × (Sharpness / 0.5)

T = (0.5 × MC + 0.5 × Cal_component) × 100

When N_eff < 10, both Calibration and Sharpness shrink toward neutral (0.5).

5. Trust Tiers

PBR Scores are translated into human-readable trust tiers. Tier qualification requires meeting score, evidence volume, and eligibility conditions simultaneously. The volume threshold uses N_eff (Kish effective sample size), not raw action count, so that L1-only actions contribute less toward tier qualification.

Table 5: Trust tier requirements (score + evidence + eligibility)

Min Ext. refers to the minimum externality ratio: the fraction of the agent's commissioner mass from commissioners who rate three or more distinct agents. This prevents tier advancement through self-dealing commissioner rings. Verified Spend is the base floor (or cluster-relative alternative, whichever is higher) as defined in Section 6.3.4. Spend is cumulative L2+ cost weighted by commissioner credibility, so Sybil attackers using low-credibility commissioner rings face 3× the nominal threshold. Gold and Platinum tiers additionally require Spend Velocity: at least 60% of verified spend must fall within the most recent half of the account age window.

6. Novel Mechanisms

PBR incorporates three mechanisms that differentiate it from standard observability platforms: behavioral fingerprinting for context-sensitive scoring, sequential drift detection for behavioral change monitoring, and layered adversarial resistance for manipulation defense.

6.1 Behavioral Fingerprinting

Behavioral fingerprinting provides context-sensitive PBR queries and non-authoritative peer group assignment for Efficiency scoring. Each action's behavioral signature is computed from structured metadata: task category, confidence score, resource consumption, and duration. The output_hash field provides tamper-evident integrity verification; it is not used for behavioral classification. No output content is analyzed, stored, or transmitted to the scoring engine.

Construction

A lightweight clustering algorithm (online k-means with configurable k, default k = 20) groups actions by their metadata vectors. Features: log(duration), log(tokens), tool_call_count, model_mix_hash, input_complexity_tier. Cost is excluded from peer selection to prevent circularity with the Efficiency dimension.

Dual Role

Behavioral fingerprints serve two purposes. First, contextual queries: consumers can query PBR for a specific behavioral context (e.g., "code review with external API calls") and receive scores conditioned on that action profile. Second, peer group assignment: Efficiency comparisons use fingerprint clusters rather than self-declared categories, making category arbitrage ineffective.

6.2 Drift Detection

Model updates, configuration changes, or environmental shifts can alter an agent's behavior distribution. PBR treats drift as a trust event: when drift is detected, the system applies evidence-gated recovery, blending pre-drift and post-drift evidence as new data accumulates.

Per-Metric Sequential Monitoring (CUSUM)

PBR uses cumulative sum (CUSUM) control charts (Page, 1954) for online drift detection. For binary metrics (success/failure), Bernoulli log-likelihood ratio CUSUM is applied:

S_n⁺ = max(0, S_{n-1}⁺ + log[Pr(x_n | p_1⁺) / Pr(x_n | p_0)])

S_n⁻ = max(0, S_{n-1}⁻ + log[Pr(x_n | p_1⁻) / Pr(x_n | p_0)])

where p_1⁺ = p_0 + δ detects improvement and p_1⁻ = p_0 - δ detects degradation. For continuous metrics (latency, cost), robust standardized CUSUM on log-transformed values is used with median and MAD-based estimators.

Table 6: CUSUM configuration per metric

Correlation Structure Monitoring

PBR tracks the Frobenius norm of the change in correlation matrix between reference and current windows:

Δ_corr = ||R_current - R_ref||_F / ||R_ref||_F

Flagged when Δ_corr > 0.3 and both windows have N_eff ≥ 20. Top-k metric pairs by |Δρ_ij| are reported for actionable diagnostics.

False Alarm Control

Bonferroni-level system-wide false alarm control: Total_charts = K_active × (2M + 1) where K is active categories and M is metrics. Per-chart ARL_0 = ARL_system × Total_charts. Only categories with N_eff > N_min maintain active monitoring charts.

Post-Drift Score Recovery

D_a,c(t) = min(1.0, N_eff_post / κ_c)

PBR_a,c = PBR_pre × (1 - D) + PBR_post × D

Full recovery requires post-drift evidence equal to the prior strength κ. This handles "reset on verified patch" naturally: if the agent performs well after a model update, D grows toward 1.0 and the score transitions. No manual reset button is needed.

Design Rationale

CUSUM was chosen over the Kolmogorov-Smirnov test because: (1) the reference shift parameter δ prevents meaningless detections at high volume, (2) sequential O(1) updates per observation enable real-time monitoring, and (3) analytically characterized average run length properties (Hawkins & Olwell, 1998) allow principled threshold selection.

6.3 Adversarial Resistance

6.3.1 Design Objective

PBR does not claim to prevent Sybil attacks. In open systems without centralized identity authority, Sybil resistance is an economic problem, not a cryptographic one (Douceur, 2002). PBR's objective: make the cost of achieving a target score via manipulation exceed the economic benefit of that score. The system is incentive-compatible when rational attackers find it cheaper to build a good agent than to fake one.

6.3.2 Commissioner Credibility as Core Primitive

The commissioner credibility γ (Section 4.3) is the foundation of adversarial resistance. Rather than relying on binary identity checks ("same_owner" heuristics that shell entities trivially bypass), PBR learns credibility from behavioral patterns that are expensive to fake at scale. The HHI diversity metric, temporal realism checks, and graph position analysis collectively turn commissioner validation into a signal with its own reputation, not a trust multiplier.

6.3.3 Graph Concentration Analysis

The bipartite agent-commissioner rating graph reveals structural patterns invisible to per-entity analysis.

Agent-side HHI: For each agent a, HHI_a = Σ p²_{a,u} where p_{a,u} is the fraction of agent a's total ratings from commissioner u. High HHI_a indicates concentrated, fragile, and potentially manufactured reputation. Agents with HHI_a > 0.5 receive reduced AR scores.

Externality Ratio: ext_a = Σ ¹[deg(u) ≥ d_min] × w_{a,u} / Σ w_{a,u}, where deg(u) is the number of distinct agents commissioner u has rated and d_min = 3. Agents rated primarily by broad-participation commissioners have high externality.

Component Isolation: For each connected component in the rating graph, isolation = edges_internal / (edges_internal + edges_to_rest). Components with isolation > 0.9 and size < 20 nodes are flagged as potentially Sybil clusters. The flag is diagnostic (included in API response), not auto-penalizing.

6.3.4 Resource Gating (Eligibility)

Resource gating creates economic friction through cluster-relative spend thresholds, coupled with time and diversity requirements. Fixed dollar thresholds fail because agent operating costs span four orders of magnitude ($0.01/task for simple classifiers to $5+/task for multi-step research agents). PBR uses adaptive thresholds that scale with the agent's behavioral cluster:

C_min(tier) = max(base_floor_tier, multiplier_tier × median_cluster_cost × N_eff_required_tier)

The base floor prevents trivially cheap clusters from having near-zero thresholds. The cluster-relative component ensures the spend requirement represents genuine sustained participation in the agent's cost tier, not a one-time payment.

Table 7: Resource gating thresholds for public index eligibility

Verified Spend is cumulative cost at L2+ integrity, weighted by commissioner credibility: C_verified = Σ cost_j × γ_commissioner(j). Spend attested by a low-diversity commissioner (γ ≈ 0.3) counts at 30% of face value. The effective Sybil cost at each tier is therefore: Bronze $167+, Silver $1,667+, Gold $8,333+, Platinum $33,333+ (at γ ≈ 0.3). Spend Velocity requires that at least 60% of verified spend falls within the most recent half of the account age window (e.g., the last 45 days of a 90-day Gold account), preventing front-loaded dump-and-park strategies. Three conditions are required simultaneously because spend alone is pay-to-win, time alone is wait-to-win, and diversity alone is network-to-win. No single axis suffices.

6.3.5 Volume Realism (CUSUM)

Action volume is monitored via CUSUM on log-transformed daily counts with reference shift δ_v = 0.5 (sensitive to sustained ~65% volume increase, not just 10× spikes). Actions during CUSUM-flagged periods receive reduced evidence weight (w_j × 0.1).

6.3.6 Nontrivial Action Classification

Actions are classified as nontrivial based on membership in behavioral fingerprint clusters (Section 6.1), not hard thresholds. A work_score is computed from observable signals (duration, tool calls, external API use). Actions with work_score below the 5th percentile of their behavioral cluster are classified as trivial and excluded from reputation computation.

6.3.7 Composite Adversarial Resistance

AR operates as an annotation and eligibility gate, not as a large multiplicative score modifier. This avoids double-penalizing weaknesses already captured by evidence quality weights (w_j).

AR = (1 - agent_HHI_risk) × (1 - stuffing_ratio) × integrity_factor

Application thresholds: AR ≥ 0.7 has no visible effect. Between 0.5 and 0.7, a manipulation_warning flag is attached. AR < 0.5 results in leaderboard exclusion, tier capped at Silver, and PBR_final = PBR_base × max(AR, 0.7). The 0.7 floor ensures AR never causes more than a 30% score reduction. Larger penalties come from data quality weights already baked into dimension computation.

6.3.8 Economic Analysis

For an attacker to achieve Gold tier via Sybil manipulation: N_eff ≥ 250 at L2+ requires approximately 415 raw actions with real commissioner attestation. C_verified ≥ $2,500 costs $8,333+ at commissioner γ ≈ 0.3. Account age ≥ 90 days requires three months of sustained operation. Spend Velocity ≥ 60% in recent half prevents front-loading. Externality ≥ 0.3 requires genuine ecosystem participation. Total estimated cost per Gold-tier Sybil agent: $8,333+ real spend over 3 months with sustained activity. A fleet of 10 Gold Sybil agents costs $83,000+ over 3 months. At this cost, building legitimate agents is the rational economic choice. For context: a legitimate Gold-tier code review agent spending $0.50/task accumulates $200+ in natural verified spend through normal operation, well below the $2,500 floor. The threshold is designed to be invisible to legitimate agents and prohibitive to attackers.

6.3.9 Explicit Non-Claims

PBR does not solve the Sybil problem in the general case (Douceur, 2002). A state-level attacker with unlimited budget, diverse legitimate-looking commissioners, patience, and sophisticated behavioral automation can construct high-scoring Sybil agents. PBR raises the cost through five complementary layers. Full Sybil prevention requires external identity infrastructure (KYC, stake-based registration, on-chain identity costs) that PBR integrates with but does not mandate.

6.4 Complete Score Pipeline

Stage 1: Base Score. PBR_base = 0.30R + 0.25S + 0.20C + (0.15E × η_avg) + 0.10T, where η_avg is the agent's mean integrity weight.

Stage 2: Drift Adjustment. If drift is active, score enters evidence-gated recovery per Section 6.2.

Stage 3: Adversarial Annotation. AR is computed per Section 6.3. If AR < 0.5, PBR_final = PBR_drift × max(AR, 0.7) with manipulation warning. Otherwise PBR_final = PBR_drift.

7. Threat Model

PBR explicitly defines the attacks it defends against, partially mitigates, and does not address.

Table 8: PBR threat model

PBR's adversarial design philosophy: make attacks expensive, not impossible. The primary structural defense is cross-dimensional tradeoffs where gaming one dimension degrades another. Inflating tokens to game difficulty increases real costs and reduces Efficiency. Using low-diversity commissioners reduces evidence quality. This cross-axis tension makes systematic metric optimization expensive.

7.1 Score Entropy Monitoring

Score_Entropy = -Σ p_k × log_2(p_k) across decile bins, computed weekly. Maximum entropy (uniform distribution) = 3.32 bits. If Score_Entropy drops below 2.0 bits for four consecutive weeks, the PBR system operator initiates a scoring model version review. Parameter changes are published as a new version with 30-day notice. Historical scores are not retroactively recalculated. Percentile rank within category is reported alongside absolute score in every API response, providing discrimination even under ceiling compression without parameter changes.

8. Implementation: The Protol Platform

PBR is a scoring specification, not an infrastructure mandate. Protol is the reference implementation.

8.1 Architecture

Table 9: Protol platform architecture

8.2 SDK Integration

from protol import Protol

# Initialize client

p = Protol(api_key="aos_sk_your_key_here")

# Register an agent

agent = p.register_agent(

name="compliance-checker-v2",

category="compliance",

capabilities=["regulatory_analysis", "gdpr_check"],

model_provider="anthropic",

model_name="claude-sonnet-4-20250514"

)

# Log actions with job linkage

with agent.action(task_category="gdpr_compliance",

job_id="job_456") as act:

result = run_compliance_check(document)

if result.passed:

act.success(

confidence=result.confidence,

cost_usd=0.04,

output_hash=hash(result.output)

)

else:

act.failure(error_type="classification_error")

# Query reputation

rep = agent.get_reputation()

print(f'PBR Score: {rep.overall_score}')

print(f'Percentile: {rep.percentile_rank}')

print(f'Trust Tier: {rep.trust_tier}')

print(f'Model Version: {rep.scoring_model_version}')

8.3 API Endpoints

Table 10: Core API endpoints

API Response Fields

Reputation API responses include: pbr_score (0–100 composite), percentile_rank (0–100 within category), scoring_model_version (e.g., "3.0"), data_integrity_distribution (fraction of actions at each level), category_consistency (Mahalanobis distance flag), ar_status (clean / warning / restricted), and per-dimension breakdowns. Action immutability is enforced: category, outcome, and metadata fields cannot be modified after submission. Corrections require a new action record linked via amendment_of field.

9. Competitive Landscape

The AI agent trust space is emerging rapidly. Existing approaches can be categorized into three groups: blockchain-based protocols, enterprise governance platforms, and identity frameworks.

Table 11: Competitive landscape

ERC-8004 is the most direct competitor in vision. Launched on Ethereum mainnet in January 2026, it provides identity, reputation, and validation registries. However, its reputation registry is a feedback-posting interface, not a scoring engine. Actual reputation computation is delegated to off-chain services. PBR could serve as the off-chain scoring layer for ERC-8004's on-chain reputation registry.

10. Limitations and Future Work

10.1 Known Limitations

Predictive validity is model-based, not empirically demonstrated. The Reliability dimension is a well-specified posterior predictive, but calibration (does a score of 80 correspond to ~80% success?) has not been validated on production data. This is the single most important open validation task.

Self-reported telemetry at L1 is inherently gameable. While evidence quality weighting reduces L1 influence, agents operating entirely at L1 can still manipulate scores. PBR's design makes L1-only scores visibly low-confidence, but does not prevent them.

Sybil resistance is economic, not absolute. PBR raises attacker cost through five layers but does not solve the Sybil problem in the general case (Douceur, 2002). A well-funded, patient attacker can construct high-scoring Sybil agents. Full Sybil prevention requires external identity infrastructure.

Selective task intake is partially mitigated. The difficulty-adjusted evidence weight reduces reward for cherry-picking, and the Coverage metric detects it when routing data is available. Standalone SDK deployments without offer/accept visibility cannot directly observe selective intake.

Goodhart's Law applies to all economically meaningful scoring. PBR mitigates through multi-dimensional cross-axis tradeoffs, difficulty-adjusted evidence, and population-level compression monitoring. No scoring system is permanently game-proof. PBR treats score integrity as a continuous monitoring problem, not a solved design problem (Goodhart, 1975).

CUSUM drift detection is per-metric with correlation supplement. Fully multivariate change detection may provide better detection in high-dimensional behavioral spaces. The current approach is a computationally efficient v1 design.

10.2 Validation Methodology

PBR's predictive claims will be validated via a strict temporal split protocol:

Calibration. Compute PBR scores at time T. Bin by score decile. Measure actual success rates in [T, T+Δ]. Report ECE and Brier Score.

Discrimination. Compute AUC-ROC of PBR Reliability as a binary classifier for next-action success, by task category.

Ablation. Recompute PBR with each dimension removed. Compare rank-order correlation (Spearman ρ) against actual outcomes.

Adversarial robustness. Simulate five attack agents. Verify PBR scores each below a legitimate agent with similar raw metrics.

10.3 Future Directions

L4 Data Integrity via cryptographically verified execution traces (TEE/ZK). Multivariate drift detection via autoencoder reconstruction error. Bayesian logistic regression for Reliability with richer contextual features. Integration with payment protocols (x402, AP2) for transaction-weighted trust signals. Adaptive φ range calibration based on empirical difficulty proxy accuracy.

11. Market Opportunity

Table 12: Market size (Sources: MarketsandMarkets, BCC Research, Grand View Research)

Enterprise AI governance mandates. The EU AI Act requires risk assessment for high-risk AI systems. Agent reputation scoring provides auditable, quantitative evidence of agent reliability for compliance. The NIST AI RMF frames trustworthy AI as an ongoing risk-management discipline.

Agent marketplace infrastructure. As agent marketplaces emerge, trust scoring becomes the selection layer. PBR's task-conditioned, drift-aware scoring provides the infrastructure for agent-to-agent delegation and marketplace ranking.

Insurance and risk quantification. PBR scores provide actuarial inputs for AI agent insurance products. A quantified trust score maps directly to risk pricing models.

12. Conclusion

The agent economy is being built now. AI agents are making economic decisions, interacting with each other, and operating with increasing autonomy. The infrastructure for payments, communication, and coordination is advancing through protocols like A2A, MCP, x402, and ERC-8004.

What remains missing is a rigorous, developer-friendly trust layer. Not a governance checkbox. Not a blockchain token. A practical scoring system that any developer can integrate in minutes and that any agent consumer can query before making a trust decision.

Predictive Behavioral Reputation provides this layer. By exploiting properties unique to AI agents—their determinism, susceptibility to drift, clonability, context-dependence, and probeability—PBR delivers trust assessment capabilities impossible in human reputation systems. Its statistical foundation (discounted Beta-Bernoulli power prior, evidence-weighted inference, uncertainty penalization) is defensible under peer review. Its adversarial resistance (five-layer economic defense, commissioner credibility learning, cross-dimensional tradeoffs) is designed to make attacks expensive, not impossible. Its limitations are stated explicitly.

PBR is not the final answer to AI agent trust. It is a rigorous first specification, designed to be extended, challenged, and improved as the agent economy matures. The scoring specification is open. The platform is live. The SDK is available today.

References

Akerlof, G.A. (1970). "The Market for 'Lemons': Quality Uncertainty and the Market Mechanism." Quarterly Journal of Economics, 84(3), 488–500.

Benjamini, Y. & Hochberg, Y. (1995). "Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing." JRSS-B, 57(1), 289–300.

Brier, G.W. (1950). "Verification of Forecasts Expressed in Terms of Probability." Monthly Weather Review, 78(1), 1–3.

Douceur, J.R. (2002). "The Sybil Attack." IPTPS, Springer LNCS 2429, 251–260.

Gelman, A. et al. (2013). Bayesian Data Analysis, 3rd Edition. CRC Press.

Goodhart, C.A.E. (1975). "Problems of Monetary Management: The U.K. Experience." Papers in Monetary Economics, Reserve Bank of Australia.

Gretton, A. et al. (2012). "A Kernel Two-Sample Test." JMLR, 13, 723–773.

Guo, C. et al. (2017). "On Calibration of Modern Neural Networks." ICML, PMLR 70, 1321–1330.

Hawkins, D.M. & Olwell, D.H. (1998). Cumulative Sum Charts and Charting for Quality Improvement. Springer.

Ibrahim, J.G. & Chen, M.-H. (2000). "Power Prior Distributions for Regression Models." Statistical Science, 15(1), 46–60.

Kamvar, S.D., Schlosser, M.T. & Garcia-Molina, H. (2003). "The EigenTrust Algorithm for Reputation Management in P2P Networks." WWW, 640–651.

Kish, L. (1965). Survey Sampling. Wiley.

Ledoit, O. & Wolf, M. (2004). "A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices." Journal of Multivariate Analysis, 88(2), 365–411.

Mahalanobis, P.C. (1936). "On the Generalized Distance in Statistics." Proceedings of the National Institute of Sciences of India, 2(1), 49–55.

NIST (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1.

Page, E.S. (1954). "Continuous Inspection Schemes." Biometrika, 41(1/2), 100–115.

Spence, M. (1973). "Job Market Signaling." Quarterly Journal of Economics, 87(3), 355–374.

U.S. Department of Justice. "Herfindahl-Hirschman Index." Antitrust Division.

Publication License

This specification is published by FlyingBoat AI for the purposes of technical review, academic citation, and ecosystem development.

Permitted Uses: Reading, reviewing, and citing this specification. Implementing PBR-compatible scoring systems that interoperate with the Protol platform via the official SDK or API. Academic research and publication referencing this work. Building integrations that consume PBR scores.

Restricted Uses (require written license from FlyingBoat AI): Creating competing commercial scoring services that substantially replicate the PBR methodology described herein. Redistributing this document in modified form. Using "PBR", "Protol", or "Predictive Behavioral Reputation" as trademarks or product names.

Patent Notice: Certain methods described in this specification, including but not limited to the commissioner credibility learning system, cross-dimensional anti-gaming mechanisms, and behavioral fingerprinting for trust scoring, may be subject to pending patent applications by FlyingBoat AI.

Attribution: Mulge, S. (2026). "Predictive Behavioral Reputation: A Trust Framework for Autonomous AI Agents." FlyingBoat AI. Version 3.0. protol.dev

Contact: Licensing: shravan@flyingboat.ai · Technical: developers@protol.dev · First published February 2026 · Version 3.0