TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Four Ways to Cut Your Cloud Costs

Reducing storage, messaging queue, and transit and computing costs and minimizing data consumption can help you cut your cloud costs.

• By Yotam Segev
• June 22, 2023

Increased productivity and a better user experience have materialized in the sprint to the cloud, but cost savings have often proven to be elusive. According to Gartner Group, “Organizations with little or no cloud optimization plans rush into cloud technology investments. They end up overspending on cloud services by up to 70% without deriving the expected value from it.” While organizations continue to adopt cloud services to unlock business opportunities, executives struggle to mitigate cloud data costs. To fully realize the benefits of the cloud, they must balance opportunity and cost optimization. The question is, how? Let’s look at the primary costs of cloud services -- storage, messaging queues, and transit and compute -- and ways to mitigate them.

Reduce Storage Costs

For Further Reading: How To Get the Upper Hand on Cloud Cost Management The Importance of Seeing Cloud Costs in Business Context The True Cost of Moving to the Cloud

Redundant data increases storage volumes, which come at a premium. Address this by removing redundant data or identifying data that can be moved from active storage to “cold” storage or archives. When data retention is required -- for example for financial or transaction data that regulations stipulate must be retained for 7-10 years -- moving backups from standard storage to glacier storage can result in significant savings. Standard S3 storage starts at $0.023 per GB per month, whereas Glacier Flexible Retrieval (archive storage that still allows for data analysis) starts at $0.0036 per GB per month. If you can simply archive the data, Glacier Deep Archive drops your costs down to $0.00099 per GB.

A good way to determine what data can move is to classify and perform a risk assessment. Modern data security platforms provide the ability to dynamically discover and automatically classify your cloud data with high degrees of accuracy. This is important because automation and continuous refinement are the only ways to ensure that you have a complete view of your data and what it represents.

Once you have this baseline, a data risk assessment provides insight into:

• Which security, privacy, or regulatory frameworks require specific actions. Do data sovereignty rules require the data to remain in a given cloud region, or does a compliance framework dictate that certain data must be retained for a given period of time?


• How that data is accessed and used. In some cases you will find that no person or application has accessed the data in months (or longer!). This allows you to check with the data owner to see if they are aware that the data exists and if there is a business purpose or justification to maintain it.


• Does that data represent a redundant or shadow copy of the data. If so, then deletion can be appropriate instead of migrating to a less expensive storage medium.

Minimize Data Consumption

In addition to transferring data to the proper storage tier, another way to reduce cloud storage costs is to reduce overconsumption. Three types of data are at the center of data minimization efforts:

• Stale data. Data that is no longer used may cost more than the value it delivers to the business.
• Ghost data. Snapshots of data stores that no longer exist can comprise up to 30% of cloud storage volume.
• Copy data. Backups are vital for ensuring data resilience and business continuity. However, copies often persist due to a lack of visibility and clear governance policies.

Maintaining an accurate data inventory is the first step in determining how often to take snapshots, how long to keep them, and when they should be kept in active versus archived storage. It is also a key step in compliance with GDPR as well as industry-specific regulations (such as HIPAA, PCI DSS, and Sarbanes-Oxley), and can have a significant impact on risk reduction. According to GDPR, “The principle of ‘data minimization’ means that a data controller should limit the collection of personal information to what is directly relevant and necessary to accomplish a specified purpose. They should also retain the data only for as long as is necessary to fulfill that purpose.”

Trim Messaging Queue Costs

Processing costs for data also represent hidden cloud costs that can increase quickly. Many cloud architectures include automation to process and manage data. “Zombie” data processing jobs, replicas or clusters that were created for temporary use but never decommissioned, or data transformation processes can rack up significant messaging fees. Consider a basic AWS service such as SQS message queues. Given that 90% of architectures use microservices, and a common architectural component in a composable/microservice architecture is message queues to keep services in sync, the seemingly low $0.35 per million requests price for this service can represent thousands of dollars in monthly fees when any changes, messages, status notifications, or log requests are processed through a queue. Understanding what data needs to be managed and how data is transiting through services can easily eliminate 60% or more of these messages if you identify which messages actually require processing.

Decrease Transit and Computing Costs

Observability is a critical component of any scalable cloud architecture. Services that record API activity and monitor resources and applications are frequently used to understand performance, usage dynamics, and costs. These are typically used in conjunction with S3 storage, EC2 computing environments, database services, notification services, and event-driven functions-as-a-service. The cost to transfer data to and from these services, as well as the computing cost to process the data to extract useful information can quickly increase cloud fees. To avoid budget overruns or runaway spending, maintaining a business objective–aligned inventory and budget is critical.

A Final Word

Cloud cost savings are possible by first gaining visibility into the data landscape with a comprehensive cloud data inventory and then working to clean it up. This will reduce storage, messaging queue, and transit and compute costs overall and help you minimize data use.