Gregory Piatetsky of KDNugget, a leading data science website, recently published a look back on the year, and predictions for the coming year. Carla Gentry, Zuar’s Product Evangelist, was tapped as a contributor to the article. The full article with Carla’s abridged assessments and predictions can be viewed here, and below are her unabridged insights.


Carla on 2021

What were the main developments in AI, Data Science and machine learning in 2021?

What an interesting and exhausting year 2021 has been, as Covid continues to be a factor in all we do, companies have pivoted on staffing, remote verses in-house, data in the cloud or on-site, data lakes or data warehouse, etc. One thing that has become noticeably clear is the fact that we are at a crossroads with data and its ability to make a difference. To add a little context, I read the other day that data sprawl has become a real and costly problem inside organizations, and it is hurting innovation. According to a new Hakkoda survey, 6% of business and IT leaders labeled their data organization and processes a “dumpster fire.”

Enter 2022, a new year and a new chance to get things right and start using all the data that you have collected. However, as we discovered, that is not always easy. Now that we have social media data, Salesforce, Google, a multitude of APIs to go along with unstructured data, structured data, relational data such as SQL, Db2, PostgreSQL and non-relational databases such as MongoDB and Hadoop, as well as NoSQL, low code, no code, NetSuite... arggggg.... It is maddening to think of joining all these data platforms to attain knowledge, but that is where we stand!

Carla on 2022

We will continue this data siloed path since each department will always have its own agenda and needs, as companies grow the need for HR, Sales, Accounting, Marketing, etc. to have usable data still exist – but what we have not faced before is HOW, as a company, to query across all platforms.

Companies that can bring multiple data platforms together will be vital for businesses who do not have the talent, time, or ability to tie all their data together to make executive decisions. Throwing good money at bad ideals is no longer acceptable, and ROIs must be attained. With retail, housing, finance data collections growing rapidly, collaboration between departments is more important than ever.

AI and machine learning will continue to be buzzwords, but will businesses use them effectively? Classification, recommendation engines and chatbots are wonderful and can be of significant use, but will they move beyond that or will it continue to be marketing hyperbole? Time will tell, but I am leaning toward the latter unfortunately.

Let us embrace all this innovative technology, but let us also keep in mind that data itself is useless unless you do something useful with it!  According to McKinsey data, a mid-sized company with $5 billion of operating costs might spend $250 million on data management. Let us make sure we spend money for the RIGHT reasons, and use all this wonderful information that we attain to make a difference in 2022.

We will continue along this path of siloed data since each department will always have its own agenda, budget and needs. Companies that can bring multiple data platforms together will be vital for businesses who do not have the talent, time, or ability to tie all their data together to make executive decisions. Throwing good money at bad ideas is no longer acceptable; ROIs must be attained. Let us embrace innovative technology, but let us also keep in mind that data itself is useless unless you do something with it.

As technology continues to be an integral part of most organizations, the most efficient and cost-effective way to stay ahead of the curve would be to find reliable business intelligence partners. Those who can ensure you stay data driven and have actionable visualization across all departments. Just having tons of data spread across multiple platforms does not ensure success, you need answers to vital questions that are part of your daily planning. Pushing pretty but uninformative dashboards and filters are no longer acceptable.

Considering the departmentalization of most companies, your HR department could be using PeopleSoft, the accounting department using QuickBooks, Finance using SAP, the Marketing department has unstructured data from social media posts, images, emails, product reviews, surveys, etc. as well as data from lead generation software, APIs, HubSpot and Web analytic data, all stored on various databases and finally your sales department could be using Salesforce to log and analyze metrics vital to ensure the company reaches their projection for growth.

This data collection obsession has caused some companies to have an overwhelming number of data sources. In fact, according to a 2021 global CDO study, data fragmentation and complexity distract from innovation enterprise infrastructure will be cloud-first and multi-hybrid for many years, with systems spread across on-premise and multi-cloud environments. The findings showed that:

  • Nearly 80% of organizations surveyed store more than half of their data in hybrid and multi-cloud infrastructures
  • 79% of organizations are using more than 100 data sources, with 30% using more than 1,000 sources
  • 37% of data leaders are barely keeping the lights on when it comes to data management as opposed to driving strategy or innovation with data

The fact is, too much data can be a hazard if it is not structured properly and migrated to be effective for all departments. Siloed data equals siloed analytics which makes sales data as far as dollars and cents collected and owed only available on QuickBooks, for example, but the metrics as far as detailed information from sales is in Salesforce. Data Migration is KEY for 2022.

Conclusion

The BI industry has a 'buyer' verses 'users' problem. They are NOT the same people. The 'buyers' are the budget holders and decision makers who ultimately choose, to the bewilderment of the 'user' who ends up not using the tools purchased. Despite being aware of the harm data silos can cause and the disadvantages of siloed data, it still happens. So protect your data and your organization by partnering with a vendor like Zuar, who will work with you one-on-one to ensure that you attain the company-wide querying ability that you need.

Data Transformation: Definition, Types, Process & Benefits | Zuar
What is data transformation? What are the benefits of data transformation? What are the processes? Learn all this and more!
How Automated Data Analytics Can Improve Team Productivity | Zuar
Not sure If data automation Is worth the effort? Learn more about automation, the methods, the tools, and more.
ETL vs ELT: What’s the Difference? | Zuar
At Zuar, we advocated using ELT instead of the more traditional ETL due to the ease of eliminating errors and auditing data with ELT. Here’s why...