Part 1 — What is Trust & Safety, actually? · The Criminologist's Guide to T&S

The job in one paragraph

Trust & Safety — T&S for short — is the function inside an online platform that decides what conduct is acceptable, detects when that conduct is violated, and enforces consequences. It sits at the intersection of policy, law, technology, and human judgment. If a platform is a city, T&S writes the rules of conduct, staffs the 911 line, trains the dispatchers, and argues with city council about what should be illegal in the first place — often at the same time. A senior T&S analyst at a major platform may review more cases in a week than a federal prosecutor handles in a year.

The Digital Trust & Safety Partnership defines T&S as "the part of a digital service's operations that focuses on understanding and addressing the harmful content or conduct associated with that service." The definition is intentionally bland; the actual work is anything but. T&S professionals daily make consequential calls about speech, violence, exploitation, fraud, and manipulation at scales no criminal justice system has ever faced. Per the Trust & Safety Professional Association, upwards of 100,000 people work in T&S worldwide.

Criminology asks why people behave; Trust & Safety asks what the platform should do about it before lunch. The frameworks are the same. The clock is different.

— The translation, in one line

Criminology ↔ Trust & Safety: a concept table

Almost every concept in a criminologist's toolkit has a T&S analog. The shape of the work is recognizable; the speed and the unit of analysis are not. The table below is the fastest orientation we can give you. The full literature map lives in Part 2.

Criminology concept	Trust & Safety equivalent	Note
Deviance	Violating content / policy violation	Both require a normative baseline; both are contested.
Social control	Content enforcement / account action	Removal, suspension, downranking, demonetization.
Victimization	Abuse reports / user harm	Reports are the primary visible signal.
Routine activity theory	Account integrity / detection logic	Classifiers operationalize guardianship at scale.
Labeling theory	False positives / wrongful enforcement	Over-enforcement labels legitimate users.
Deterrence	Strike systems / escalating consequences	Suspension hierarchies mirror sentencing theory.
Recidivism	Ban evasion	Suspended accounts that return — a central T&S problem.
Criminal career	Bad-actor trajectory	Serial abusers exhibit career patterns across accounts.
Organized crime	Coordinated inauthentic behavior (CIB)	Networks operating as an organized adversarial system.
Hot spots	Bad-actor infrastructure / network clusters	Graph concentration mirrors geographic concentration.
Situational crime prevention	Safety by design / friction	Reduce opportunity by modifying the environment.
Criminal intelligence	Threat intelligence / OSINT	Systematic collection on adversarial actors.
Intimate partner violence	Technology-facilitated abuse (TFA)	Abusers exploit platform features as instruments of control.
Grooming	Child safety / CSAM production pathways	The grooming process is an object of detection and disruption.
Dark figure of crime	Under-reporting / unreported harm	The gap between harm and reports is the platform's dark figure.
Transnational crime	Cross-platform, cross-jurisdiction harm	Networks span platforms and legal regimes.

The major sub-disciplines

Policy

Policy teams write and maintain the platform's Community Standards, Terms of Service, and internal operational guidelines. They decide which behaviors are prohibited, how policies are worded, how edge cases resolve, and how the rules interact with regional law. The work is fundamentally normative and quasi-legislative: you are setting rules that govern billions of people. The criminological parallel is legislation and standard-setting; the tension between over-inclusion and under-inclusion is constant.

Operations / Content Moderation

Operations is the largest workforce segment in T&S by headcount, though much of it is outsourced to contracting firms (Accenture, Teleperformance, Sama, Telus International) in lower-wage markets. Operators review flagged content — images, videos, text, live streams — against policy and make removal or action decisions. Specialist teams handle escalations: terrorism, CSAM, NCII, targeted harassment. This is also where the secondary-trauma literature is concentrated; the criminological parallel is frontline criminal justice administration, with similar debates about discretion, equity, and worker wellbeing.

Integrity / Data Science

Integrity teams (sometimes "Signal," "Counter-Abuse," or "Trust Engineering") use machine learning, statistical modeling, and behavioral analytics to detect harmful activity before users report it. This includes classifier development, anomaly detection for coordinated inauthentic behavior, graph analysis to find fake-account networks, and risk scoring for new accounts and transactions. The criminological parallels are crime analysis, environmental criminology, and — with all the appropriate critiques attached — predictive policing.

Threat Intelligence / Investigations

Threat-intel teams track adversarial actors who deliberately probe platform defenses: organized fraud networks, state-sponsored influence operations, child exploitation networks, terrorist organizations. The work blends open-source intelligence (OSINT), digital forensics, network analysis, and coordination with law enforcement. A criminologist with methods training in network analysis or dark-web research will feel immediately at home here.

Product / Engineering (Trust by Design)

Product and engineering roles embed safety into platform features before they ship. A Trust-by-Design product manager reviews new features for abuse potential and designs safeguards into the product architecture — friction mechanisms, identity verification flows, reporting interfaces. This is situational crime prevention applied at the product level. Engineers build the classifiers, hash-matching systems, and enforcement infrastructure.

T&S Operations / Program Management

A horizontal function that coordinates across the other disciplines: vendor management, workflow tooling, metrics and reporting, regulatory response, cross-functional project management. This is where much of the formal DSA/OSA compliance infrastructure now lives. For a criminologist with policy or program-evaluation experience, this is often the most accessible entry point.

A working glossary

You will hear these terms in your first week. The full vocabulary is larger; this is the floor.

Perceptual hashing / PhotoDNA: Converting a piece of content into a numerical fingerprint that matches even if the image is cropped or re-colored. PhotoDNA, developed by Microsoft Research with Dartmouth, is the cornerstone of industry CSAM detection.
CSAM: Child Sexual Abuse Material. US platforms are required by law (18 U.S.C. § 2258A) to file a CyberTip with NCMEC when they discover CSAM on their service.
NCII: Non-Consensual Intimate Imagery. StopNCII.org operates a hash-sharing database modeled on the CSAM infrastructure.
CSEA: Child Sexual Exploitation and Abuse — broader than CSAM, covering grooming, live-streamed abuse, sextortion, and production.
CIB: Coordinated Inauthentic Behavior. Networks of fake or coordinated accounts that manipulate discourse. Originally Meta's term, now standard usage.
Ban evasion: A suspended account that creates a new account to continue violating. The platform analog of recidivism, and one of the field's hardest detection problems.
Sock puppet: A fake account operated by a real person to fake support or endorsement. Distinct from bots, often used alongside them.
Brigading: Coordinated mobilization of a group to attack a single target — flooding a post with hostile content, reports, or downvotes.
Doxxing: Malicious publication of someone's private information to expose them to offline harm. Both an enforcement category and a tool in harassment campaigns.
Hash-matching: Comparing content against a database of known violating hashes. Used for CSAM, NCII, and terrorist content (the latter via GIFCT).
Friction: Design interventions that slow or raise the cost of harmful behavior — interstitials, confirmations, rate limits, identity steps. Borrowed directly from situational crime prevention.
Transparency report: A periodic public disclosure of enforcement actions and government requests. Required under the DSA for very large platforms.
Adversarial adaptation: Bad actors modifying their behavior in response to detection — a digital arms race that parallels criminal adaptation to policing.
SAR / KYC: Suspicious Activity Report; Know Your Customer. Financial-compliance vocabulary that increasingly shows up in payments-platform T&S.
CyberTipline: NCMEC's mandatory CSAM reporting system in the United States.

Where the field is right now

T&S is under simultaneous and opposite pressure. Rolling layoffs at Meta, Google, Microsoft, and others have cut headcount substantially since 2022 — postings with "trust and safety" in the title were down 70% from January 2022 by early 2023 (NBC News, citing Indeed data). At the same time, the EU's Digital Services Act and the UK's Online Safety Act are pushing platforms to staff back up in compliance-facing roles. The labor market is volatile, but the regulatory floor is rising. For criminologists, that means more T&S work tied to formal governance, audit, and risk-assessment work than at any point in the field's history.