New work on ML research agents offers a concrete answer to a long-standing puzzle: why does benchmark-driven progress keep being real, even after years of iterating against the same held-out data? The answer comes down to compressibility. A model whose strategy fits into a few tokens can't be relying on memorization. Memorized data wouldn't fit. When a compressed strategy enables one model to reproduce another's performance, it's evidence that the original gains came from something structural, not from benchmark-specific tuning. Across eight datasets, 32-token prompts were sufficient for a fresh ML research agent with no access to validation data to reproduce a source agent's performance on the large majority of problems. One language-modeling strategy survived compression to 16 tokens with no loss in held-out performance. The same pipeline identified overfitting when it was deliberately induced: in 38 of 102 runs, validation-specific gains vanished after compression.
Amazon Science
خدمات الأبحاث
Seattle، Washington ٣٩٤٬٨٣٩ متابع
The latest news and research from Amazon’s science community. #AmazonScience
نبذة عنا
Amazon Science gives you insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Our scientists continue to publish, teach, and engage with the academic community, in addition to utilizing our working backwards method to enrich the way we live and work. Follow us on LinkedIn and visit our website to get a deep dive on innovation at Amazon, and explore the many ways you can engage with our scientific community. #AmazonScience
- الموقع الإلكتروني
-
https://www.amazon.science
رابط خارجي لـ Amazon Science
- المجال المهني
- خدمات الأبحاث
- حجم الشركة
- أكثر من ١٠٠٠١ موظف
- المقر الرئيسي
- Seattle, Washington
- تم التأسيس
- 2020
- التخصصات
- Artificial Intelligence، Machine Learning، Computer Vision، Cloud، Economics، Sustainability، AI، ML، Conversational AI، Natural Language Processing، NLP، Robotics، Security، Privacy، Information، Knowledge Management، Operations، Scientific Research، Search، Amazon، و Alexa
التحديثات
-
Amazon and Defense Advanced Research Projects Agency (DARPA) brought together more than 150 researchers from academia, government, and industry for a three-day workshop in Seattle on how AI is transforming mathematical discovery and reasoning. The program featured opening remarks from Amazon Web Services (AWS) CEO Matt Garman, a keynote from Fields Medalist Terence Tao, perspectives from the National Security Agency, lightning talks from research teams, and panels on formal verification and AI-assisted mathematical reasoning. Amazon's investment in this space goes back more than a decade – from formal verification running in production at AWS to the Lean theorem prover led by Leonardo de Moura. The company has also committed $1M in AWS credits to support the research teams doing this work. Workshops like this help connect Amazon scientists with the broader research community and push the boundaries of what's possible.
-
-
-
-
-
+٥
-
-
Congratulations to Amazon Redshift researchers for being awarded Best Paper Runner-Up: Industrial Track at VLDB 2026 for "FastCompose: Eliminating Compilation Cold Starts in Query Execution with Composition." The work addresses a fundamental tradeoff in compilation-based query execution: when compiled code isn't cached, queries wait on compilation before any data processing begins. FastCompose resolves this by generating lightweight LLVM IR code that assembles pre-compiled operators into execution plans, cutting compilation time from seconds to milliseconds. On TPC-DS benchmarks, the approach achieves a 7x speedup, with composition and compilation yielding identical results on a single codebase. Read the paper: https://amzn.to/4qRWGIn
-
-
Verus verifies complex projects with thousands of lines of code and proof in the time prior tools took to verify individual functions. That speed enables an interactive development loop and lets AI agents iterate on proofs faster. Amazon has used this open-source, automated program verifier for Rust to prove correctness of key primitives in the Nitro Isolation Engine and critical internal infrastructure.
-
Eight out of ten LLM judges agree. But how independently did they arrive at that answer? When judges share a prompt template, training lineage, or model family, a majority vote can make evidence look far more convincing than it really is. The vote count inflates confidence without adding independent signal. Amazon researchers introduce dependence-aware label aggregation using Ising models to address this. The method models both individual judge reliability and pairwise dependencies, allowing redundant agreement to be discounted without discarding votes entirely. Across three tasks with 10-judge panels, the approach improved accuracy by 9–14% over weighted-majority-vote baselines.
-
-
أعاد Amazon Science نشر هذا
Honored to be included on this year's TIME100 AI list. This one belongs to our customers and to the teams at Amazon Web Services (AWS). The majority of workloads haven't moved to the cloud yet, and when they do, the opportunity to transform with AI is massive. We're just getting started.
AWS CEO Matt Garman has been named to the TIME100 AI list for 2026. 🎊 Matt joined as an MBA intern in 2005, before AWS even had a name, and twenty years later he leads the world's largest cloud provider, now on a $169 billion annualized revenue run rate. Congratulations, Matt. Read the full TIME100 AI profile: https://lnkd.in/eSA6V-rw
-
-
How did a model upgrade make agents worse? SOP-Bench is a new benchmark that pairs genuine enterprise procedures with functioning tools and ground-truth answers across 12 industries and 2000+ tasks. With a reasoning-style (ReAct) agent, the newer Claude 4.5 family scored lower than the older Claude 4 family. A routine upgrade can lower an agent's success rate with no obvious signal that anything changed. The only reliable way to catch it is to test on the procedures the team actually runs. With SOP-Bench, agents earn their scores by completing the work rather than producing text an automatic grader happens to like.
-
-
Ten years ago, Amazon's Automated Reasoning Group set out to use mathematical logic to prove AWS systems work correctly. Today, their production services process billions of queries daily, powering tools millions of customers rely on, from IAM Access Analyzer to Amazon Inspector to Bedrock Guardrails. One project stands out: they proved correct and seamlessly replaced the AWS authorization engine, which handles one billion API calls per second, and verified it against quadrillions of production authorizations. Now the same formal-verification techniques that secured cloud infrastructure are being applied to AI, setting boundaries for autonomous agents and validating AI-generated content with up to 99% verification accuracy.
-
AWS Trainium Frontier challenges researchers to train language models from scratch on purpose-built AI chips. The NeurIPS 2026 competition is open for registration, limited to 100 teams. Teams optimize across the full stack, including model architecture, optimizer, training loop, and custom hardware kernels. No prior Trainium experience is required. Prizes include $25K for first place, co-publication with Annapurna Labs researchers, and a chance to present at an event during NeurIPS 2026 in Sydney. Register by September 30: https://amzn.to/3TL03EA