TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Intelligence and Efficiency Will Guide Unstructured Data Management in 2023

With more data than ever in the cloud, it’s no wonder that data management strategies will change in 2023. Here are 5 trends that reflect those changes.

• By Kumar Goswami
• December 5, 2022

This year IT and data professionals went on a wild ride, needing to act on projects that may have been sidelined from pandemic-related and supply chain disruptions. Many organizations spent nearly two years focused on remote work, business continuity, and customer retention projects. Although uncertainty persists and IT budgets may tighten in 2023, the intelligent application of technology remains the force behind business innovation and productivity. Here’s how I see the trends for data management and storage shaping up in 2023.

For Further Reading: Executive Q&A: Data Management Best Practices for Changing Times The 3 Vs and Unstructured Data Analytics Executive Q&A: The Value of Improved Data Management

Trend #1: IT leaders will spend on technologies that will best support data-driven initiatives across the enterprise

As IT leaders look to make smart bets with threats of a recession looming, in 2023 they will focus on strategic cloud and analytics investments to support new data-driven business initiatives. Cloud computing and AI/ML technologies have the highest potential for cutting costs and delivering insights that can create new value from data, using the affordable power of cloud computing. Automating workflows to curate and deliver data to cloud-native analytics tools will help IT organizations efficiently leverage massive stores of unstructured data while reducing the manual effort required for data curation by data analysts and researchers. Data workflow automation is becoming a new requirement of unstructured data management platforms.

I also predict greater adoption of adaptive, intelligent automation that learns from enterprise environments to improve results. This functionality could be particularly useful in areas such as cybersecurity and governance, data life cycle management (moving the right data to the right place at the right time), and customer experience management. Intelligent automation helps enterprises avoid the dreaded data swamps, compliance breaches, and outsized data storage costs, and shortens time to value from BI, AI/ML, and cloud services initiatives.

Trend #2: Cloud file storage will continue to grow but will require more intelligent data migrations

File storage in the cloud is the top enterprise storage spending priority, according to the Komprise 2022 State of Unstructured Data Management Report. However, as enterprises improve cloud practices and rein in spending overall, it will become important to better understand file data characteristics to optimize data placement in the cloud. Rather than a lift-and-shift dump of data to cloud file storage, organizations need proper planning and assessment of migrations, with deep insight into the financial and user impacts of moving data. This knowledge will allow organizations to fully leverage the diversity of cloud storage classes, each of which is designed with specific workloads and access patterns in mind. IT and business leaders should also be sure that migration strategies and tools allow them to easily leverage cloud-native data services including big data analytics, indexing and search, data workflows, and data protection, including backup and replication.

Trend #3: Edge data growth will push demand for intelligent edge processing

Explosive data growth along with consumer adoption of disruptive digital products such as self-driving cars is pushing demand for edge storage and consequently is changing data management requirements to deliver visibility into edge data. This visibility will be instrumental in managing data in place at the edge through enrichment (such as tagging) and extraction of just the right data sets for analysis.

Smarter edge data management will avoid overspending on storing extraneous data in cloud data lakes and warehouses by filtering and deleting non-valuable data at the edge first. Edge analytics tools will quickly process the data without the need to send large files back and forth to cloud or on-premises data centers, saving time and money. The right edge analytics and data management program can deliver real-time insights to improve customer experiences or detect issues quickly, such as a manufacturing defect or a ransomware breach.

Trend #4: Analytics will become a core component of unstructured data management

The global AI software market is expected to reach a whopping $135 billion by 2025, at a growth rate outpacing the overall software market, according to Gartner. As enterprise demand for these new tools accelerates, data management strategies need to follow suit. Unstructured data, which comprises at least 80 percent of all data generated, is the fuel needed to power modern ML engines. A majority (65 percent) of organizations in the Komprise 2022 State of Unstructured Data Management survey indicate that they plan to or are already delivering unstructured data to their big data analytics platforms. This trend is indicative of the ongoing evolution of data management from driving storage cost efficiencies to a broader mission of supporting data analysts, data scientists, and department heads across the enterprise.

Storage and IT managers will need to prepare by getting full visibility into data across silos, understanding data characteristics and metadata to enable rapid classification and search, and then moving it into the optimal storage tier to feed the data lake and analytics platforms preferred by their end users. IT will need to work closely with stakeholders from security, legal, data governance, research, and data science teams, as well as business unit leaders, to fulfill the requirements of new, unstructured data analytics programs.

Trend #5: The IT self-service trend will expand to unstructured data management

Enterprise IT departments are drowning in data requests along with their daily responsibilities. It’s time for end users and departments to play a greater role in managing their own files and data storage. With the appropriate security guardrails in place, storage professionals will benefit by sharing data management analytics with departments. By doing so, IT teams can collaborate more closely with departments to deliver data services while meeting cost savings and governance goals.

At the same time, departments can ensure that their data is managed appropriately according to business needs and is always easily accessible and available. For example, users can identify data sets with certain characteristics (such as project or age) to move to cloud storage to cut costs or support research initiatives. The democratization of unstructured data management will ultimately create tighter alignment and collaboration between IT and business units, which can only benefit the enterprise for the long term.