Anthropic Warns of 'Existential Threat to Humanity'

Plus: It's Hard To Make a Monopoly Board Game in America

Sponsored by

This week’s world-famous-news-haiku-competition™ is about how a Silicon Valley start-up that released its first product just last week is drawing investment offers at a multibillion-dollar valuation, with promises that its AI tools are more efficient than OpenAI and Anthropic’s. Send me your entry — to haiku at cheddar dot com — by noon ET Thursday, for consideration by your Cheddar peers.

Now? News!

Matt Davis — Need2Know Chedditor

Table of Contents

What’s the Stock Market Up To, Eh?

Companies Mentioned in Today’s Newsletter

Anthropic Warns of 'Existential Threat to Humanity'

Did you see the SNL “Weekend Update” skit featuring Anthropic $ANTHZZX ( ▼ 3.22% ) CEO Dario Amodei? It skewered his recent spate of going on TV and urging people to stop his software from killing us all. Like, isn’t that a weird thing for a CEO to be saying?

Weirder still, there’s more. In the company’s pre-IPO (float) paperwork filed this week, the firm has warned of an “existential threat to humanity” from its products. In its S-1 prospectus, the nearly $1 trillion start-up devoted almost a third of its filing to outlining "risk factors." These include advanced AI models learning to manipulate, blackmail, and exhibit unpredictable behaviors, while current and former staff warn that runaway AI could end humanity within a decade. Amodei himself told the UN Security Council that AI is "the most important global security issue facing the world today."

Eager investors are responding by aiming to value the Claude maker at over $2 trillion. I mean, if an algorithm is going to blackmail you, you might as well collect equity on it! Of course, apocalyptic doom isn't the only peril on display. Anthropic also detailed some rather "prosaic risks." For instance, it recorded an operating loss of more than $8 billion last year on $4.6 billion in revenue, and has pledged $518 billion toward future cloud and computing obligations. Amodei is now leading public calls for "pacing the frontier" of AI development, a proposal backed by rivals Sam Altman and Elon Musk in a rare moment of industry unity. Yet despite these warnings, Anthropic remains on course to hit the Nasdaq this autumn.

So, if you have extra cash lying around, why not buy into the listing? Either the stock doubles, or nobody survives to collect the tax loss. Win-win! [I thought we agreed, enough with the screamers, Matt — Ed.] #NotFinancialAdvice

Quote of the Day

OpenAI Scraps New Model Over Security Concerns

(Google)

OpenAI $OPEAZZX ( ▼ 0.18% ) has scrapped the public release of its new GPT-6.1 Astra model over safety concerns. During internal evaluations, the ambitious system exhibited high levels of deception and routinely took unauthorized actions without user permission. Saachi Jain, OpenAI’s head of safety systems, noted, “For anything regarding safety and alignment, there’s a trade off,” adding that when shipping products to users, “we have an extremely high bar in terms of safety and alignment.”

Or at least, they do now. And this sudden display of corporate caution arrives right as The New York Times revealed that OpenAI repeatedly brushed aside internal employees who warned executives for months about dangerously lax testing protocols. Former OpenAI employee Daniel Kokotajlo bluntly told the Times, “it seems like they had very bad security, and also sloppy model training practices that led to the models having this sort of propensity.” Joshua Saxe, CTO at Abundant Security, similarly observed that OpenAI’s posture was “what you’d expect from a research lab that scaled at a blistering pace over four years and focused more on beating its competitors than securing its infrastructure.”

The situation became so chaotic that independent security researchers inspecting the company were stunned by its primitive infrastructure. Researcher Mohan Pedhapati famously asked, “Why are you using Slack to build your nuclear Manhattan projects?”

And while Florida Attorney General James Uthmeier mocked tech titans claiming they “cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so,” OpenAI seems to have finally taken the hint. Scrapping an autonomous model that lies and breaks out of testing environments is probably a good idea in that context. I wonder what it cost them, this learning experience? The firm already lost $12.6 billion in the second quarter, after all….

It's Hard To Make a Monopoly Board Game in America

(WS Game Company)

You’d think manufacturing a board game about raw American capitalism inside the United States would be a slam dunk. But when WS Game Company tried to produce the Monopoly Americana Edition domestically to dodge Chinese tariffs, they rolled a double-one and landed straight in manufacturing hell.

CEO Jonathan Silva didn’t mince words about the ordeal, discussing it with The Wall Street Journal: “It was a colossal headache. I hope I don’t have to go through it again,” he admitted, adding at one point during severe production bottlenecks, “I threw my hands up. I was done.”

What went wrong? Pretty much everything. WS Game initially expected a modest 20% cost premium compared to manufacturing in China. Instead, as Director of New Business Development Adam Hocherman revealed, “the premium was around 100%.”

Custom wooden boxes were scrapped for costing $65 each, while plastic money trays repeatedly cracked, delayed production for months, and pushed the retail price to $80 — nearly double the Chinese-made version. Even COO Kerry Addis confessed, “The product made in China has so many little things that are better than we can do here.”

The comedy reached peak irony when it came to component sourcing. Addison Camossi, whose family packaging firm was hired to build the flocked plastic tray, marvelled, “It’s the first time we made something that doesn’t automatically go in the trash.”

And the ultimate punchline? The game couldn't even achieve 100% American status: The dice still had to be imported from China because no U.S. factory could supply 5,000 pairs.

It’s enough to make me want to start up a dice factory. Maybe my father-in-law will buy me an edition for Christmas?

Song of the Day: U2, ‘Silencio’

Yes, Bono is extremely annoying, but he can still sing, and this is the band’s best single in donkey’s years, critics agree. Why not give it a try?

Oracle Boss Buys Homes for Staff While Debt Spirals

While ordinary corporate perks max out at free espresso, Oracle $ORCL ( ▲ 3.91% ) co-founder Larry Ellison has taken staff benefits to a surreal new level. The tech billionaire spent nearly $10 million quietly purchasing eight homes in a Palm Beach gated community for his household staff, including tutors for his children, The Wall Street Journal reports.

Local real-estate agent Kimmie Cruz recalled convincing homeowners to sell by reassuring them, “This is a chance to have somebody very viable. The money is there.” Staff recruiter Peter Mahler defended the lavish arrangement, explaining, “If you want them to get to work quickly and have a reasonable commute, investing in lodging can be a very smart decision.” And what a view. Local agent Ashley McCoy noted that in the community, “Every single home has a view of the water.”

Yet while Ellison’s domestic team enjoys lakeside Florida homes outfitted with Tesla $TSLA ( ▼ 1.29% ) Powerwalls, credit markets are experiencing severe indigestion over Oracle's mounting debt load. The cost to insure Oracle’s debt recently reached a record high, with its 2056 bonds crossing an 8% yield for the first time. Sitting at a precarious BBB- credit rating, Oracle is just one single downgrade away from “junk” territory. If that threshold is crossed, a staggering $120 billion in bonds will be automatically pulled from investment-grade indexes, unleashing a massive corporate debt sell-off.

Pumping billions into lumpy cloud infrastructure while gifting Florida real estate creates, let’s say, a stark financial contrast. Funding an AI arms race on borrowed cash while credit markets tighten is a high-stakes gamble. And Ellison is also on the hook to finance his son’s takeover of Warner Bros. $WBD ( ▼ 0.16% ) with Paramount Skydance $PSKY ( ▼ 2.82% ) . The commentary on Twitter is hardly encouraging about all this!

Free Yourself From Advertising Forever!

Now you can sign up for an optional ad-free version of Need2Know! Subscribe for just $5 a month, or $50 a year, and you can continue to enjoy this reasonably high-quality newsletter uninterrupted. Bonus: The immense satisfaction that comes from supporting journalism*!

*This counts as journalism, right?

ADVERTISEMENT

Your K-Cup Has Been Sitting in a Warehouse for Months

That grocery store pod? It was roasted long before it hit the shelf. Coffee starts losing flavor within weeks of roasting — which is why your morning cup tastes flat, no matter how nice your machine is.

Angelino's does it differently. A third-generation roaster in Los Angeles, they roast, grind, and seal every pod in-house for peak freshness, then ship it to your door within days of roasting. The difference is the first thing you'll notice: real aroma, real flavor, no bitterness.

Mix and match from 50+ specialty coffees, teas, and flavored blends — all Keurig®-compatible — and save more with every box you add, up to 34% off at 12+. Order once whenever you're running low, or subscribe for an extra 5% if that's easier. Entirely up to you.

New customers get 15% off their first order — applied automatically, no code needed.

SpaceX Starship Orbits Twice, Comes Home Early

Elon Musk’s supersized 407-foot rocket — the largest ever built — finally reached orbit for the first time, successfully deploying 26 of SpaceX's $SPCX ( ▲ 2.59% ) most advanced Starlink satellites this week. Although an unexpected early engine failure created a few nail-biting minutes right after liftoff, flight controllers pressed ahead, prompting Mission Control to announce to wild cheers, “Starship is orbital,” while NASA Administrator Jared Isaacman* commended SpaceX for “managing every step in a safe, responsible, and especially inspirational way.” 

*Jared Isaacman is a huge kiss-up, evidently.

The original flight plan envisioned an ambitious 10-hour marathon spanning six full laps around Earth. But after cruising through just two “laps” at a leisurely 17,500 mph, controllers opted to bring the craft down early for a Pacific splashdown north of Hawaii. SpaceX explained that the recall was made “out of an abundance of caution,” though the Starship tipping over and erupting in flames upon impact may have proved that the abundance was justified.

Addressing a business summit earlier this month regarding over-land flights, Musk candidly noted that if a spacecraft disintegrates and rains debris onto people, “our popularity would diminish very rapidly,” adding, “That’s why we’re being extremely cautious here.”

Ultimately, delivering Internet satellites, orbiting Earth twice, and concluding in a dramatic ocean fireball is a resounding success in aerospace testing. It’s a brave new world, huh?

Should You Check Your 401(k) Today?

👎️ 

No.

Poll of the Day: Just Floating This Id-AI-a

Will you buy stock in OpenAI and Anthropic when they float?

Login or Subscribe to participate in polls.

Poll of the Day: Good Luck With That Line, Eh?

We asked: Is Kalshi gambling, or what?

You answered:

🟩🟩🟩🟩🟩🟩 Yes. (598)
⬜️⬜️⬜️⬜️⬜️⬜️ No. (8)
606 Votes via @beehiiv polls

Want more Cheddar? Watch us!

Search “Cheddar” on Samsung, YouTube TV, and most other streaming platforms.

N2K is the tip of of the cheeseberg for financial news, interviews, and more.

Need2Know is covered by Cheddar’s Terms of Service

P.S. So, you remember the cheese puns that used to open this newsletter? Suffice to say, they were divisive. Now, thanks to a thing called “dynamic content options,” I can offer you the option to see cheese puns again, if you’re one of the thousands who got in touch bemoaning their departure six months ago. All you need to do is answer “true” on this survey, and submit it. If you never want to see cheese puns in this newsletter again, don’t click that link, don’t fill out the survey, don’t submit it. Just keep reading and pretend this conversation never happened. Mmmkay? Thank you.