OpenAI Proposes Perplexity as Key Metric for AI Safety and Progress
OpenAI's chief scientist argues for using perplexity as a universal benchmark to track AI advancements and coordinate safety standards across the industry.

SAN FRANCISCO — OpenAI's Chief Scientist Jakub Pachocki has proposed perplexity—a measure of how well a model predicts the next token in a sequence—as a foundational metric for tracking AI progress and establishing safety standards. In a detailed blog post, Pachocki revealed OpenAI's internal perplexity measurements for models like GPT-3.5 and GPT-4, showing how lower perplexity correlates with emergent capabilities such as chain-of-thought reasoning and advanced problem-solving.
Pachocki argues that while downstream benchmarks quickly become obsolete, perplexity offers a stable way to compare models across labs and anticipate unforeseen capabilities. For example, GPT-4's log-perplexity of ~0.91 enabled breakthroughs in reasoning, while newer models below 0.8 have demonstrated unexpected proficiency in mathematics and cybersecurity. OpenAI now advocates for industry-wide limits tied to perplexity thresholds to ensure safety research keeps pace with advancements.
The proposal comes as AI begins to accelerate its own development, particularly in pretraining research. Pachocki notes that recent models with log-perplexity around 0.76 have produced groundbreaking mathematical results, suggesting a similar leap could occur in deep learning. However, he warns that alignment and monitoring may not advance quickly enough to match this acceleration, heightening the need for coordinated safeguards.
While perplexity has limitations—such as lab-to-lab comparability and its indirect relationship to practical tasks—OpenAI believes it could serve as a shared benchmark for 'general intelligence.' The company calls for transparency agreements among frontier developers to establish safety milestones before crossing critical perplexity thresholds. Reported with files from Hacker News.
NewsCritical Check Point Vulnerability CVE-2026-123039 Exploited in PHP2Shell Campaign
Security researchers detail post-exploitation evidence and recovery steps for the exploited Check Point CVE-2026-123039 vulnerability with CVSS 7.8 score.
NewsDeveloper Reveals Method for Scraping App Store Reviews Without Proxies
A Node.js developer shares a technique for extracting international App Store review data in JSON format, enabling sentiment analysis without complex scraping infrastructure.
NewsDatabricks SQL AI Forecast V2 Adds Holiday Support and Covariates
Databricks SQL's ai_forecast function now supports holiday effects and covariates, streamlining time-series forecasting directly within SQL queries.
NewsHow a WordPress LMS Front End Was Rebuilt with React and Cloudflare Pages
A team modernized a WordPress LMS front end using React and Cloudflare Pages while maintaining core functionality like logins and payments.