Meta internal tools for community safety

Exploring the evolution of a tooling system that seeks to keep the Meta community safe

Background

Millions and millions of images & videos are posted everyday across Meta’s different platforms like Facebook and Instagram. How a single problematic piece of content is flagged, reviewed and possibly taken down involves a complex orchestration of detection, human review, and machine-learning technologies.

This project was an early phase design exploration focused on how to evolve the current system of tools used for coordination, investigation and review. A design sprint was used to bring together designers from different tooling-related teams, producing a set of provocative designs 5 years in the future, that drove leadership conversations forward.

A reimagined vision for a system of organised tools that streamline and connect integrity efforts across Meta. 

Design manager role

Assembling a cross-organisational team of designers

As Design manager, one primary consideration was how to bring together individual designers from different tooling products, encouraging them to think expansively beyond their individual tools and consider the tooling landscape at large. Pushing for inclusion of designers from a cross-section of tools, and guiding them into the energy-intensive design sprint process paved the way for the design vision to be created.

Storytelling around an entire tooling system

In order to address the systemic challenges of expanding tech & design debt, the direction underpinning the design vision needed to be on a tooling system level, rather than individual tools. A fine line needed to be walked around demonstrating a relatable UI/UX tooling experience through a screen prototype, and the central ideas addressing system-level concerns.

One tooling system, many different users

In previous sprint methods, a single user and their problems or jobs-to-be-done guide the trajectory of the design sprint. In this exercise, the sprint method was adapted to encompass system-wide exploration. For example, it was necessary to guide the designer group to establish a foundational architecture of tools of which they could hang their subsequent ideas from.

Outcomes

A provocative vision for the future

A Design Vision Sprint outputted an end-to-end prototype that was interwoven through a future-facing system of tools, demonstrating how a less fragmented tooling environment could enable Meta to act more quickly and effectively to reduce harm to its users. This presentation & prototype was shown to stakeholders at the organisational leadership level to gain support to begin making these tooling changes.

Problems and opportunities were framed through the lenses of efficiency & effectiveness. Namely: 

• Reduced operational cost - especially around human review processes

• Reduced Average Handling Time

• Acceleration of proactive detection & enforcement actions

Tools that worked well together

Central to the design vision was the idea that this tooling system needed to be consolidated into a reduced number of tools that interoperated more effectively together. In order to achieve this consolidated system of tools, a foundational architecture for the entire system needed to be prioritised. 

Automation-first approaches 

Envisioning the scale of the policy & moderation challenge to only increase over time, the design vision focused on the promotion of automation-processes to reduce unnecessary manual work happening in tools. Within the design sprint, this looked like automation accelerating the sourcing of content examples to train classifiers, and the faster routes to human review.