Every application needs secrets to run: the password for its database, the key that lets it charge a card, the token that lets it send email or call an AI model. Most of those secrets end up stored with whoever hosts the application. In October, one of those hosts found out what that concentration looks like from an attacker's side.
What happened
Railway's own preliminary report, updated on 26 October, gives an unusually detailed timeline, and it reads like a siege rather than a single break-in.
On the afternoon of 20 October, monitoring paged Railway's on-call engineers about abnormal latency. Within hours they had found an attempted privilege escalation in the build infrastructure and halted the build pipelines. By that evening Railway had concluded it was dealing with, in its words, "a sustained and targeted campaign." Overnight it found further routes of access and halted builds again.
In the early hours of 21 October the attacker pulled the encrypted environment variables but could not read them. Around two hours later they obtained credentials for Railway's CI environment, and between roughly 07:25 and 09:18 UTC they ran an unauthorized query against the encrypted variables and decrypted a subset of them "utilizing a combination of the compromised credentials."
From there the response moved quickly. At 09:25 UTC Railway asked vendors including GitHub, GitLab, Anthropic, OpenAI, OpenRouter and Stripe to revoke customer keys on their side. That evening it invalidated user sessions, revoked all Railway access tokens, rotated the credentials for every storage bucket on the platform and told customers to rotate everything. Systems were rebuilt with new credentials overnight, and full service returned during 22 October. New sign-ups were still paused as of the 26 October update.
Three things are not yet known. Railway has not disclosed the root cause, saying only that it found "potential privilege escalation" in its build infrastructure followed by "subsequent routes of access." It has not ruled out that the attacker accessed customer source code. And it does not know which individual secrets were decrypted, because the query it found "alone cannot identify specifically which keys" were exposed. A final forensic report is promised.
The encryption worked, until the attacker had the keys
Railway stored customer secrets with envelope encryption, a standard and sensible design in which each secret is encrypted and the keys that decrypt them are themselves protected separately. It worked as designed for a while: the attacker's first haul of encrypted variables was useless to them.
It stopped mattering once they reached the CI credentials. Encryption at rest protects data from someone who steals a disk or a database backup. It does far less against someone already inside the systems that are allowed to decrypt, because from the platform's point of view that request looks legitimate.
None of this is a criticism peculiar to Railway; most platforms work this way. But it is worth being precise about what "your secrets are encrypted" means when a provider says it. It means a stolen copy of the database is not enough. It does not mean nobody who gets far enough inside can read them.
Why AI keys
Railway's report is direct about motive. It believes the attacker was "attempting to obtain AI provider API keys to perform distillation attacks," meaning they wanted to send very large volumes of queries to commercial AI models, on someone else's account, to train or improve a model of their own. Railway says the actor had access to "abliterated models and substantial compute resources."
That changes what a stolen key is worth. A leaked database password is valuable if the attacker wants that database. A leaked AI key is valuable to almost anyone, immediately, and its cost lands on the account holder. Railway has said it cannot refund spending done with compromised keys and has pointed customers to the AI providers.
Railway was not alone. Wiz counted nine developer and AI platforms that disclosed incidents during October: Fireworks AI, Andromeda, Zeabur, Lightning AI, OpenCode, Railway, Warp, CommandCode and Northflank. It found no single vulnerability or actor connecting them, but it did document what happens to keys after they leak. In two cases it investigated, keys traced to an unnamed compromised vendor were used for roughly 2.5 million unauthorized calls to Amazon Bedrock, with an estimated list-price cost over $100,000. Those cases are not tied to Railway, but they show the scale this kind of abuse reaches.
The Zeabur breach, a few days before Railway's, is the bleaker example. It was that platform's second breach in two months: an attacker used leaked internal credentials to get into its shared cluster, exposing around 321,810 accounts and more than 3.2 million environment-variable records. The administrative tokens involved had never expired and had not been rotated since they were created in January 2024. Zeabur has announced it is shutting down by 31 December.
This is not only a developer's problem
It is tempting to read this as a story for software teams. Most of our clients do not deploy code to Railway and have never heard of it.
But many of them have a website, a booking system, a customer portal or an internal tool that someone built for them, and that someone chose where to host it. The application almost certainly holds keys: to the payment processor, the email service, the CRM, and increasingly an AI service. Ask who holds those keys and where they are stored, and the honest answer is often that nobody at the business knows.
That is the real exposure. Not that a particular platform was breached, but that a business can have its payment or AI credentials sitting on infrastructure it did not choose, cannot see and would not think to check when a headline like this appears.
What to actually do
Find out where your keys live. Ask whoever built or maintains each of your applications a plain question: which platforms store our API keys and passwords, and which keys are they? Write the answer down. If anything of yours ran on Railway before 21 October, Railway's own advice is to rotate every stored variable, including private registry credentials, and to review logs.
Put spending limits and alerts on every AI account. Where a provider supports hard limits, use them, and set a billing alert at a level you would notice. A key that can spend without limit is a credit card left on the counter. An unexpected bill is often the first sign a key has leaked, so make sure someone actually reads those alerts.
Give each application its own narrowly scoped key. One key per application, with only the permissions it needs, means a leak from one place can be revoked without breaking everything else, and the usage logs tell you where the leak came from.
Rotate on a schedule, not only after an incident. Zeabur's tokens had gone unrotated for more than two years. A key that is changed every few months is worth far less to whoever steals it.
Ask your providers how they isolate customers. Railway is moving its builds from shared multi-tenant infrastructure to single-tenant environments as part of its response. That is a fair question to put to any platform that holds your secrets, along with how quickly it can tell you which of your keys were exposed if it is breached.
The summary
Railway did a lot right once it noticed: fast containment, vendor-side key revocation, a public timeline with timestamps. The breach still exposed secrets for up to 100,000 workspaces, because a platform that stores everyone's keys is worth a determined attacker's time in a way no single customer is.
A business cannot fix that concentration, but it can decide how much a stolen key is worth. Know where your keys are, limit what each one can do and spend, and rotate them before someone else uses them.
Sources
- Preliminary Report: Oct 21st Security Incident — Railway
- Security Disclosure: Wednesday October 21st — Railway
- Developer platform Railway urges 'all users' to revoke secret keys after cyberattack — TechCrunch
- Railway warns devs to rotate secrets after potential breach — BleepingComputer
- Railway says intruder may have accessed customer secrets — The Hacker News
- Black October for AI developer platforms — Wiz
- Zeabur shutting down after second security breach in two months — BleepingComputer
