Public is not computable: Project ProtoStar stress-tests AI-ready open science with challenges

U.S. federal R&D spend is cited around $190B per year, with NIH/NSF tightening public-access expectations. Putting files online is not the same as making them discoverable, combinable, and reusable by models. Hugging Face’s 29 October 2026 ProtoStar announcement frames a testable question: can publicly funded artifacts released in AI-ready form be found, reused, and joined across institutions to accelerate discovery and public benefit?

Thesis: without machine-actionable formats and open challenges around real tasks, open-access policy mostly stacks static files; ProtoStar turns “reusable” into a fail-able experiment via institutional MoUs plus sequenced challenges.

Founding partners include CMU, Purdue, Georgia Tech, ASU, and GW (via Open Forum for AI), Astera Institute, Hugging Face as infrastructure, and ORCA. Participants formalize MoUs to release suitable datasets, models, and code on open infrastructure.

CMU’s Climate Risk Intelligence challenge: from history through 2018, forecast 2019–2026 average temperature, precipitation, hottest day, and cooling degree days for 210 U.S. cities versus simple statistical baselines, plus a short adaptation policy idea. Submissions are HF datasets; a public leaderboard tracks forecast error, with final 2026 scores due in early 2027 once full-year observations land.

Demand-side numbers: >17k public Hub datasets in core science domains (~29k with medicine/math); new science datasets in the first nine months of 2026 more than 3× 2025’s same span (still substantial after excluding the largest uploader). NIH NCBI’s 18 public repos topped 11M downloads; NASA IMPACT’s 27 ~770k; ~90% of dataset downloads come from fully documented sets. HF is among NIH-listed generalist repositories.

That shows demand for discoverable science assets — not yet that ProtoStar MoUs change institutional behavior. The challenge’s ex-post observational scoring avoids unverifiable pure futures.

Steelman: universities and labs already run domain repositories; another Hub initiative may fragment effort or warp incentives toward leaderboard-friendly dumps. Short policy blurbs in the climate challenge are hard to grade rigorously and risk performative entries. Federal public-access rules vary by agency; voluntary MoUs have limited teeth.

HF is both infrastructure partner and Hub operator — success metrics that only count Hub downloads can miss equivalent reuse in domain archives.

For funders, the piece moves compliance from “PDF plus attachment” toward machine-actionable artifacts with task-level verification. For AI research tooling, cross-institution join friction — not another foundation-model parameter count — is the bottleneck. For urban climate and the public sector, a baseline-referenced forecast challenge offers a path from model error into adaptation debate — if policy review is not decorative.

Watch six months: valid CMU submissions and error versus baselines; whether a second institution sequences its challenge on time; and whether MoU orgs actually upload documented AI-ready assets. If challenges yield reproducible cross-resource projects beyond brochure pages, the thesis holds; if only signing ceremonies and mirror uploads appear, rewrite it.