Sunday, November 22, 2026Coding news · developer employment · the CodeCore archive · RSS
CodeCoreCoding news, employment and the Vancouver tech community

OpenAI Proposes Perplexity as Key Metric for AI Safety and Progress

OpenAI's chief scientist argues for using perplexity as a universal benchmark to track AI advancements and coordinate safety standards across the industry.

· Hacker News

SAN FRANCISCO — OpenAI's Chief Scientist Jakub Pachocki has proposed perplexity—a measure of how well a model predicts the next token in a sequence—as a foundational metric for tracking AI progress and establishing safety standards. In a detailed blog post, Pachocki revealed OpenAI's internal perplexity measurements for models like GPT-3.5 and GPT-4, showing how lower perplexity correlates with emergent capabilities such as chain-of-thought reasoning and advanced problem-solving.

Pachocki argues that while downstream benchmarks quickly become obsolete, perplexity offers a stable way to compare models across labs and anticipate unforeseen capabilities. For example, GPT-4's log-perplexity of ~0.91 enabled breakthroughs in reasoning, while newer models below 0.8 have demonstrated unexpected proficiency in mathematics and cybersecurity. OpenAI now advocates for industry-wide limits tied to perplexity thresholds to ensure safety research keeps pace with advancements.

The proposal comes as AI begins to accelerate its own development, particularly in pretraining research. Pachocki notes that recent models with log-perplexity around 0.76 have produced groundbreaking mathematical results, suggesting a similar leap could occur in deep learning. However, he warns that alignment and monitoring may not advance quickly enough to match this acceleration, heightening the need for coordinated safeguards.

While perplexity has limitations—such as lab-to-lab comparability and its indirect relationship to practical tasks—OpenAI believes it could serve as a shared benchmark for 'general intelligence.' The company calls for transparency agreements among frontier developers to establish safety milestones before crossing critical perplexity thresholds. Reported with files from Hacker News.

News

Critical Check Point Vulnerability CVE-2026-123039 Exploited in PHP2Shell Campaign

· DEV Community

Security researchers detail post-exploitation evidence and recovery steps for the exploited Check Point CVE-2026-123039 vulnerability with CVSS 7.8 score.

News

Developer Reveals Method for Scraping App Store Reviews Without Proxies

· DEV Community

A Node.js developer shares a technique for extracting international App Store review data in JSON format, enabling sentiment analysis without complex scraping infrastructure.

News

Databricks SQL AI Forecast V2 Adds Holiday Support and Covariates

· DEV Community

Databricks SQL's ai_forecast function now supports holiday effects and covariates, streamlining time-series forecasting directly within SQL queries.

News

How a WordPress LMS Front End Was Rebuilt with React and Cloudflare Pages

· DEV Community

A team modernized a WordPress LMS front end using React and Cloudflare Pages while maintaining core functionality like logins and payments.