TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

4 Ways to Control Cloud Costs in the Age of Generative AI

Generative AI applications are adding to enterprise cloud costs. Here are 4 ways to control them.

• By Brian Adler
• June 13, 2024

As organizations harness the power of artificial intelligence to drive innovation and efficiency, they face a pressing concern: how to strike the delicate balance between leveraging generative AI and optimizing cloud costs.

For Further Reading: Four Ways to Cut Your Cloud Costs How To Get the Upper Hand on Cloud Cost Management The Importance of Seeing Cloud Costs in Business Context

In the recently released Flexera 2024 State of the Cloud Report, the numbers paint a vivid picture of where the priorities of IT leaders lie. Today, more than half of all data is in the public cloud; more than one-third of respondents indicate all nonsensitive data will move to the cloud, and nearly one-fifth say they will move all sensitive data to the cloud.

With this move of data to the cloud, cloud infrastructure and the associated expenses play pivotal roles in strategic initiatives. Managing cloud spending reigns supreme as the top cloud challenge, cited by a staggering 84% of respondents. More than a quarter of respondents allocate over $12 million annually to cloud expenditures, reflecting a notable 21% year-over-year increase in organizations spending $1 million or more per month on cloud services.

As organizations navigate the complexities of cloud cost management, another budgetary challenge emerges: investment in generative AI. Many generative AI chatbots or assistants need to query vast amounts of data stored across cloud environments, driving cloud costs to unpredictable levels for many organizations. A quarter of respondents are already using generative AI public cloud services extensively, with an additional 38% experimenting with generative AI capabilities. IT leaders are making speculative bets on this emerging technology, with considerable investments in opportunities with a potential for a strong return. However, this is introducing new pressures on cost optimization and budgets, particularly those earmarked for cloud infrastructure.

How can your organization navigate this intricate landscape to maximize the benefits of both generative AI and cloud computing while efficiently managing costs?

1. Cultivate cost awareness across the organization

First and foremost, prioritize building a cost-conscious culture within your organization. IT professionals are presented with some serious challenges to get spending under control and identify value where they can. Educating teams on cloud cost management strategies and fostering accountability can empower them to make informed decisions that align with business objectives.

Organizations are increasingly implementing FinOps frameworks and strategies in their cloud cost optimization efforts as well. This promotes a shared responsibility for cloud costs across IT teams, DevOps, and other cross-functional teams. Cloud computing is a top operating expense for most organizations, and with a FinOps framework, your business can gain a better understanding of its costs and improve decision-making processes.

One of the biggest problems today is how much cloud spending is wasted. Respondents reported that their public cloud waste stands at 27%. This is where the FinOps team can step up and identify opportunities for cost savings and pinpoint wasteful spending. With 51% of organizations reporting utilizing a FinOps team and 20% reporting they will have one by next year, the industry is certainly seeing the benefits of having FinOps practices to manage their cloud costs. The cost savings can then be used to power innovation, such as generative AI initiatives.

2. Harness the power of analytics and automation

Implementing robust monitoring and optimization tools is essential. By leveraging analytics and automation, your organization can gain real-time insights into cloud usage patterns and identify opportunities for optimization. Whether it's rightsizing resources, implementing cost allocation tags, or leveraging spot instances, proactive optimization measures can yield substantial cost savings without sacrificing performance.

Advanced analytics can also provide predictive insights, enabling your enterprise to anticipate future resource requirements and adjust cloud provisioning accordingly. By harnessing the power of data-driven decision-making, you can optimize your cloud spending while ensuring scalability and agility to meet evolving business demands.

When it comes to generative AI, IT leaders must be prepared to swiftly pivot when making speculative investments. Although some projects may yield promising results and tangible returns, others may not make good business sense. In such instances, business leaders must hold themselves accountable. Dealing with emerging technologies requires leaders to establish strict assessment processes, supported by robust data, to effectively measure return on investment. This is still a burgeoning area, so your IT leaders may need to get creative in their evaluation criteria and processes.

The advice here is to set goals and performance metrics ahead of any implementation. With continuous monitoring, your leaders should be able to avoid runaway costs associated with generative AI and strike a balance between experimentation and value.

3. Allocate strategic workload placement

Not all workloads are created equal; understanding their unique requirements can inform decisions regarding deployment models -- whether it's through on-premises, public cloud, or hybrid environments. By strategically allocating workloads based on factors such as performance, security, and cost, you can optimize cloud spending while harnessing the full potential of generative AI.

For example, mission-critical workloads that require high availability and stringent security measures may be best suited for on-premises or private cloud environments, and less sensitive workloads can leverage the cost-efficiency and scalability of public cloud platforms. By adopting a workload-centric approach to cloud deployment, your organization can achieve optimal performance and cost-effectiveness across your IT infrastructure.

4. Embrace continuous optimization

Finally, optimizing cloud costs is not a one-time endeavor but a continuous process. As your business needs evolve and technologies advance, you must remain vigilant in monitoring cloud spending and identifying opportunities for further optimization.

Regularly revisit and refine cost management strategies to ensure your enterprise stays agile and responsive in an ever-changing digital landscape. By embracing a culture of continuous optimization, your organization can adapt to shifting market dynamics, optimize cloud spending, and maximize the value of generative AI and cloud computing investments.

The Bottom Line: Strike a Balance

Striking a balance between generative AI and cloud cost optimization requires a holistic approach that encompasses people, processes, and technology. By prioritizing cost consciousness, leveraging advanced analytics and automation, adopting strategic workload placement strategies, and embracing continuous optimization, your organization can unlock the full potential of its investments in both generative AI and cloud computing -- ushering in a new era of innovation and efficiency.