
{/* banner prompt: Clean editorial illustration of a magnifying glass moving over a cloud-shaped invoice, small dials and toggles floating around it (min-instances, CPU, region, disk), a few of them glowing to show waste being switched off, warm white background with teal and orange accents, calm and practical, 16:9. */}

Our monthly Google Cloud bill came in at just over 600 dollars. Nothing was on fire, no traffic had spiked, and no new service had launched. The bill had simply crept up, one reasonable-looking decision at a time.

So we sat down and audited it, line by line. By the end we had a plan that cut the bill by more than half, without leaving the cloud, without moving regions, and without giving up anything we actually used. Almost all of the savings came from understanding *settings* — the dials that cloud platforms leave on their most convenient default rather than their cheapest one.

Here is what we learned.

## Read the bill like a map, not a total

The single most useful thing we did was stop looking at the total and start grouping every line item by what it actually was. Google's cost table breaks charges into dozens of tiny SKUs — "CPU (request-based billing)", "SSD backed PD capacity", "network inter-region data transfer" — and on their own they mean very little. Grouped by *cause*, a shape appears fast.

When we did that, one pattern jumped out immediately:

<Table data={
    {
        headers: ['Where the money went', 'Roughly', 'What it actually was'],
        rows: [
            ['One background workload', 'About 60%', 'A fleet of small agents writing to the database on a timer'],
            ['Web serving', 'A chunk', 'Cloud Run handling real requests'],
            ['Always-on worker', 'Around 50 dollars', 'A container that could never scale to zero'],
            ['One database server', 'Around 90 dollars', 'A virtual machine sitting almost completely idle'],
            ['Small stuff', 'The rest', 'Logs, disks, snapshots, an orphaned IP address']
        ]
    }
} />

The lesson of the audit was almost boring: **one workload was 60% of the bill.** Everything else combined — every website, every API, every bit of storage — was the minority. If you only have time to fix one thing, find your elephant first.

## Cloud Run is cheap, until a setting says otherwise

We love Cloud Run. It scales to zero, you only pay when it runs, and you never patch a server. But "you only pay when it runs" hides two settings that decide whether that promise holds.

### Minimum instances

By default a Cloud Run service scales to zero when no one is using it. Set a minimum number of instances above zero and you are asking Google to keep that many copies warm at all times — which removes cold starts, and bills you around the clock whether or not a single request arrives.

<Table data={
    {
        headers: ['Setting', 'What it does', 'What it costs'],
        rows: [
            ['Min instances = 0 (default)', 'Scales to zero when idle', 'Nothing when idle; a cold start on the first request'],
            ['Min instances = 1 or more', 'Keeps copies always warm', 'You pay for those copies 24/7, including while you sleep'],
            ['CPU only during requests (default)', 'CPU is throttled between requests', 'Cheapest for normal request and response work'],
            ['CPU always allocated', 'CPU keeps running between requests', 'Bills like an always-on VM, not per request']
        ]
    }
} />

We had a couple of services quietly holding a minimum instance warm for a workload that saw almost no traffic. Setting them back to zero cost us nothing but a cold start once in a while, and removed the charge entirely.

### The always-on trap

Here is the sharper lesson. One of our services was not a website at all. It was a background worker that holds a long-lived connection to a message queue and reacts to events. It has no incoming web requests to speak of — just a socket it needs to keep open.

That is exactly the workload Cloud Run is *worst* at. To keep the connection alive, it needs a minimum of one instance and CPU that is always allocated. In other words, it has to behave like a server that never turns off — while being billed as premium serverless compute. It cost around 50 dollars a month to do something a tiny always-on virtual machine, or a container on a machine we already ran, could do for close to nothing.

**The rule we took away:** serverless is brilliant for spiky, request-driven work that can genuinely go to zero. For anything that must stay awake all the time — a queue consumer, a poller, a persistent connection — check whether a small VM you already operate is cheaper. Often it is, by a wide margin.

## When to self-host a database instead of using a managed one

This was the big one, and it is the least obvious.

Managed databases — Cloud SQL, Firestore, DynamoDB and friends — are wonderful. Backups, failover and scaling are handled for you. But many of them bill **per operation**: so much per read, so much per write. That pricing is fantastic when your traffic is small or spiky. It turns brutal when your workload is *chatty*.

Our background fleet was as chatty as it gets. Each agent wrote a small status update to the database on a fixed timer — every 30 to 60 seconds — whether anything had changed or not. Multiply a handful of writes per minute by hundreds of agents across a month and you land at **hundreds of millions of tiny operations**. On per-operation pricing, those cheap little writes added up to the largest single line on the whole bill.

The same writes, sent to a plain MySQL server running on a virtual machine we already operate, cost effectively nothing extra. The disk was already spinning; a few hundred million small writes a month is comfortably within what one modest server handles without noticing.

So when should you reach for each?

<Table data={
    {
        headers: ['A managed database fits when', 'Self-hosting starts to win when'],
        rows: [
            ['Traffic is small, spiky or unpredictable', 'Traffic is steady and high in volume'],
            ['You would rather not run a server at all', 'You already run a VM with spare capacity'],
            ['Per-operation cost is a rounding error', 'Your workload is chatty: many tiny reads and writes'],
            ['You want managed backups and failover', 'You are happy to run your own backups'],
            ['You are early and want to move fast', 'You are steady and want predictable, flat cost']
        ]
    }
} />

There is a second reason we moved this workload onto our own database: **portability.** A plain MySQL server over a standard connection runs anywhere — another cloud, another region, your own hardware. Per-operation managed services are convenient precisely because they lock the shape of your data and your bill to one vendor.

A word of caution, because it cuts both ways: self-hosting means *you* own the backups, the patching and the 3am restart. We did not move a customer-facing production database off its managed home for a few dollars. We moved a high-volume, non-critical, internal workload where the maths was overwhelming and the risk was low. Pick your battles.

Two cheap habits also soften per-operation pricing without moving anything:

- **Write only on change.** A status update that is identical to the last one does not need to be written at all. Debouncing our timer writes alone would have removed most of the volume.
- **Do not read to decide whether to write.** It is easy to "save a write" by reading first — and end up paying for the read instead. On a chatty workload, reads can cost as much as the writes you were trying to avoid.

## Right-size relentlessly

One database server on the bill cost around 90 dollars a month. We logged in to look at it and found the load average sitting at 0.00. Two dedicated CPU cores, 99% idle, around the clock.

We had sized it once, long ago, for a load that never arrived — and then paid for that guess every month since. Swapping it for a smaller, shared-core "burstable" machine — the kind that idles cheaply and borrows extra CPU only when it briefly needs it — cut the compute cost roughly in half. It even came with more memory than before.

The habit worth building: **size machines to what they actually do, measured, not to what you once imagined they might need.** A minute with a load monitor is worth more than an hour of guessing.

## The quiet extras: region, disk, IPs and logs

Once the big items were handled, a set of small ones were each worth a few dollars, and together a meaningful slice.

**Region.** The exact same machine costs noticeably more in some regions than others — a premium of 30 to 40% is common for the more expensive locations. That premium is often worth paying: keeping a database close to your users cuts latency, and moving a production system across the world is risky. But pick your region *on purpose*, knowing what the proximity costs, rather than out of habit.

**Disk tier.** Fast SSD disks cost roughly double a "balanced" disk of the same size. A small database that is mostly idle does not need premium disk performance. Matching the disk tier to the actual workload is an easy, invisible saving.

**Idle IP addresses.** Reserved static IP addresses are billed even when they are attached to nothing. We found one reserved address doing absolutely nothing and released it. Worth a scan.

**Logs.** Log ingestion is free up to a generous monthly allowance, then billed per gigabyte. One noisy service — including a bug that threw the same error thousands of times a day — was pushing us over the line. We added a filter to drop the low-value noise (while keeping real errors) and flagged the bug to fix at the source. Verbose logging is a cost, not just a habit.

## About those "commit and save" discounts

Cloud providers offer real discounts if you commit to a resource for one or three years. They can be worth it for a steady, permanent workload. But read the fine print: the cheap shared-core machines are often *not* eligible, and — importantly — right-sizing first can beat committing. In our case, downsizing an idle machine saved more than a one-year commitment on the oversized one would have. Do the cheap, reversible optimisation before you sign a year-long contract.

## The takeaway

None of this was clever. We did not rewrite anything or adopt a new platform. We read the bill like a map, found the one workload that was most of it, and turned the right dials:

<Table data={
    {
        headers: ['Move', 'Why it worked'],
        rows: [
            ['Group the bill by cause', 'One workload was 60% of it; the rest was noise'],
            ['Mind Cloud Run min-instances and CPU', 'Warm instances and always-on CPU bill 24/7'],
            ['Self-host the chatty database', 'Per-operation pricing punishes high-volume writes'],
            ['Right-size idle machines', 'A 0.00 load average is money burning'],
            ['Sweep the small stuff', 'Region, disk tier, idle IPs and noisy logs add up']
        ]
    }
} />

Cloud platforms are a wall of dials, and the defaults are tuned for convenience and for the provider's revenue — not for your bill. The good news is that the same wall of dials is exactly what lets you fix it. Set a reminder to audit the bill every quarter. The cloud rewards attention, and quietly taxes the lack of it.

*If you run anything in the cloud and have never grouped your bill by cause, that is the single most valuable hour you can spend this month.*


df14e3d9271520b989cd074edc113525f817c67a06a7dcc5b44d3319e182b71f

Practical lessons from auditing a Google Cloud bill: Cloud Run min-instances and CPU billing, when to self-host a database instead of a managed one, and region and machine right-sizing.

A real 600-dollar-a-month cloud bill, audited line by line, and the Cloud Run settings, database choices and machine sizes that quietly drove the cost.

How We Cut Our Google Cloud Bill in Half


# The Dark Side of Cloud Computing: Unexpected Costs and Precautions

## Introduction
Cloud computing has revolutionized the way businesses and individuals manage data and applications. With benefits like scalability, cost-efficiency, and accessibility, it’s no wonder that cloud services have become a cornerstone of modern technology. However, these advantages come with hidden risks that can lead to significant challenges for users.

This article focuses on one of the most common issues: unexpected costs in cloud computing. By examining real-life cautionary tales, we’ll explore why these costs occur and how you can avoid them.

## Cautionary Tales from the Cloud

### Story 1: Data Storage Surge and Budget Shock

A most recent story is that a X user posted her cloud service bill on the social network platform. 

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/the-dark-side-of-cloud-computing-stories-and-precautions/2025-01-24-18-26-29.png)
(Source: [X](https://x.com/tamarajtran/status/1880036092906467841))

It was a shocking ~$70,000 bill in one day as she stated.

[@tamarajtran](https://x.com/tamarajtran) has a project relying Google's Firebase service, and her bill was normally $50/month. However, due to a bug in the code, around 1PB of data was unexpectedly stored in the Google Cloud Storage (GCS), which lead to the surging bill.

Fortunately, this story had a happy ending this matter was resolved with the help of the Google staff. [@tamarajtran](https://x.com/tamarajtran) no longer has to worry about the bill.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/the-dark-side-of-cloud-computing-stories-and-precautions/2025-01-26-15-20-14.png)
(Source: [X](https://x.com/tamarajtran/status/1880719936190042560))

### Story 2: "Wrong" AWS Bucket Name and Unexpected Bill

In April 2024, software engineer Maciej Pocwierz encountered an unexpected $1,300 charge on his AWS account due to nearly 100 million unauthorized S3 PUT requests within a single day. The issue arose because a popular open-source tool used a default (placeholder) S3 bucket name identical to his. Consequently, numerous misconfigured systems attempted to store their backups in his private bucket, leading to the excessive charges. At that time, AWS charged for unauthorized requests, even if they were denied.

According to the [report](https://cybernews.com/news/amazon-bucket-name-can-cause-massive-bill/),
> The software engineer even demonstrated a simple exploit: anyone knowing a bucket name, even without an AWS account, could try to initiate a connection, which will be denied, but the owner will be charged for each request.

In response to such incidents, AWS announced in May 2024 that it would no longer bill customers for unauthorized requests resulting in specific HTTP error responses, aiming to prevent similar unexpected expenses in the future. 

### Story 3: Insane Bill

This story was discussed on the Hackers News [article](https://news.ycombinator.com/item?id=28637647). Basically, a user recounted an incident from approximately 2017 where they inadvertently incurred a $15,000 charge on Microsoft Azure. While experimenting with Azure's video processing services, they attempted to disable and detach a service that didn't meet their needs. Despite believing they had successfully shut it down, the process continued running unnoticed, as it didn't appear in the Azure console. The good news was that upon discovering the substantial charge, the user contacted Azure support, who acknowledged the oversight and waived the fee.

### Story 4: Unexpected Bill from Years Ago

This is my own story. I used a cloud service about five years ago on a Pay As You Go (PAYG) plan. Essentially, if you don’t pay, you don’t go (no pay, no service). I was charged daily until I stopped using the service, leaving my account at a zero balance as shown in their console.

Years later, I received a bill claiming that I still owed them money for a particular month. This is completely illogical because my account was set up to automatically deduct any charges from my credit card. If there was a gap in the billing, there must be something wrong with their system. And if there was something wrong with their billing system, how could they claim that this is a legitimate bill?

## Why Unexpected Costs Happen

- **Complex Pricing Models**: Cloud providers often use intricate pricing structures that can be difficult to predict.
- **Lack of Monitoring**: Without tools to track usage, it’s easy to exceed budgeted amounts.
- **Unoptimized Resources**: Keeping unused or over-provisioned resources running can lead to unnecessary expenses.
- **Data Transfer Costs**: Moving data between regions or out of the cloud can incur significant fees.
- **No Hard Limits**: Some cloud providers don’t enforce hard limits, leading to unexpected costs.

## How to Avoid Unexpected Costs

1. **Understand Pricing Models**: Familiarize yourself with your cloud provider’s pricing structure, including hidden fees like data transfer costs.
2. **Set Budget Alerts**: Use tools to set spending limits and receive alerts when approaching them.
3. **Optimize Resources**: Regularly audit your cloud usage to identify and shut down unused or over-provisioned resources.
4. **Monitor Usage**: Implement usage monitoring tools to gain real-time insights into your cloud consumption.
5. **Regularly Audit**: Regularly audit your cloud usage and configurations to identify potential issues and take proactive measures.
6. **Stay Within Free Tiers**: If using a free tier, ensure you understand the limits and monitor your usage carefully.
7. **Better Architecture**: Optimize your cloud architecture to minimize over-provisioned resources and reduce data transfer costs.
8. **Simplify Cloud Usage**: Focus on essential cloud services and avoid unnecessary features.
9. **Use Free Services**: Take advantage of free services and credits offered by cloud service providers.
Points 1 to 6 highlight precautions that help minimize risks on the cloud service provider's side, while the final three points focus on actions we can take to reduce the risk of unexpected costs on our end.

By designing a better architecture, we can reduce unnecessary cloud usage and make smarter choices when selecting services that suit our business needs. For instance, instead of relying on Google Cloud Storage, we could create an NFS share drive using a virtual machine in the cloud to store our data. This approach allows us to control costs by setting a hard limit on the virtual machine's disk size, mitigating the risk of unlimited storage usage. However, this method increases system complexity, so we must carefully weigh the trade-off between added complexity and reduced cost risks.

Modern cloud providers offer a wide range of features, but using too many of these services—or encountering bugs in our code or third-party code—can lead to unexpected expenses. To mitigate this, it's essential to simplify our cloud usage and focus on core, essential services.

Additionally, some free services can effectively support business operations. For example, Cloudflare provides free DNS name resolution and cloud front services, which can also help mask the true IP addresses of our servers. Leveraging such free options where appropriate can further reduce costs without compromising functionality.


## What to do If You Run Into Unexpected Costs

### Challenges in Resolving Unexpected Costs

When things like unexpected costs happen, it can be challenging to to contact the provider to resolve the issue:
- **Lack of Documentation**: Some providers may not provide clear documentation on how to address unexpected costs, leading to delays in resolving the issue.
- **Lack of Training**: Some providers may not provide training on how to resolve unexpected costs, making it challenging for non-technical employees to address the issue.
- **Complex Contact Processes**: Many businesses have complex contact processes, making it challenging to identify the right contact person for resolving the issue.
- **Unresponsive Support**: Some providers may not provide timely support, making it challenging for businesses to resolve issues.

### Resolving Unexpected Costs Through Multiple Communication Channels

When evaluating a cloud service provider, it’s crucial to consider their reputation for customer support, as reaching their support team can sometimes be challenging. Reliable and responsive customer service can make a significant difference in resolving issues effectively.

To maximize your chances of getting assistance, try contacting the provider through multiple channels, including their support, sales, and customer service teams. Use every method available—email, phone, and social media. However, keep in mind that phone support is often reserved for premium customers with many cloud providers.

If you’ve exhausted all official communication channels, consider sharing your story on social media. Social platforms can amplify your message, potentially connecting you with the support team and allowing you to highlight your experience. This approach can also provide valuable insights into the provider’s operations and how best to address your concerns.

## Conclusion

Cloud computing offers immense potential, but unexpected costs can quickly overshadow its benefits. By understanding how these costs arise and taking proactive measures, you can ensure that your cloud investments remain cost-effective and sustainable.

While the cloud provides flexibility and scalability, vigilance and preparation are essential to avoid budget shocks. Take the time to review your current strategies and implement safeguards to keep your cloud usage under control.

Please note this article does not aim to diminish the benefits of cloud services. Instead, it seeks to highlight potential risks and provide guidance on managing them effectively.

## Contact

If you have any questions or need further assistance in regarding finding a cloud service suitable for your business or optimizing your cloud usage, feel free to reach out to me on:

- X (formally twitter): [Eric Tang](https://x.com/_e_tang)
- email: [eric@tyo.com.au](mailto:eric@tyo.com.au)


## References

- [User beware: choosing the wrong AWS bucket name can lead to a massive bill](https://cybernews.com/news/amazon-bucket-name-can-cause-massive-bill/)

3532f454d63beeea93457b2fca79330246b0fc5cea0c4cc155dc3b8331706507

This article focuses on one of the most common issues: unexpected costs in cloud computing. By examining real-life cautionary tales, we’ll explore why these costs occur and how you can avoid them.

This article focuses on one of the most common issues: unexpected costs in cloud computing. 

Tag: Cloud

Tag: Cloud

How We Cut Our Google Cloud Bill in Half

The Dark Side of Cloud Computing: Unexpected Costs and Precautions