Gartner identifies four emerging challenges to achieving value from AI securely and at scale

According to a Gartner survey of 451 senior technology leaders in the second quarter of 2024, 57% of chief information officers (CIOs) said they were responsible for leading their company's artificial intelligence (AI) strategy. However, four emerging challenges make it difficult for CIOs to realize the value of AI.

“CIOs often feel like they are in the middle of the hype due to the constant race to innovate among technology vendors, but they feel like they are in the trough when it comes to driving AI outcomes – the reality is that value is elusive,” said Mary Mesaglio, distinguished research vice president at Gartner.

“CIOs can actually control the pace of AI outcomes,” said Hung LeHong, distinguished research vice president and Gartner Fellow . “If the company’s AI goals are modest and the industry has not yet been disrupted by AI, CIOs can adopt a relatively steady pace, namely AI-steady pace. If the company’s AI goals are more ambitious or the industry is being reshaped by AI, CIOs need to adopt a faster pace, namely AI-accelerated pace. Whether moving forward at AI-steady pace or AI-accelerated pace, the ultimate goal must be to drive value and results.”

In the opening keynote of the recent Gartner IT Symposium/Xpo, Gartner analysts discussed how to address four emerging challenges to securely realize the value of AI at scale in front of more than 8,000 CIOs and IT executives.

Using AI may not always lead to business benefits

The premise of using generative artificial intelligence (GenAI) to create business value is that GenAI tools must be used continuously in the workflow. In the second quarter of 2024, Gartner surveyed more than 5,000 digital workers in the United States, the United Kingdom, India, Australia and China. The results showed that employees using GenAI can save an average of 3.6 hours per week, but not all employees using GenAI can get the same degree of benefits.

“This is the real challenge with AI productivity,” LeHong said. “The productivity gains from GenAI are not evenly distributed but will vary across employees, not only because of differences in individual interest and adoption, but also based on job complexity and experience level.”

Organizations adopting AI at an accelerated pace are seeking not only productivity gains but also improvements at other levels, including operational and process improvements (such as automating key business processes or reinventing jobs to collaborate with chatbots) and transformative improvements at the business level (such as outcomes that create new revenue streams or reshape the enterprise value proposition).

“In this context, CIOs should manage AI benefits like an investment portfolio, determining how much to invest in each benefit area and balancing the risk and reward of the entire portfolio,” Mesaglio said.

AI costs could quickly spiral out of control

According to a Gartner survey of more than 300 CIOs conducted in June and July 2024, more than 90% of CIOs said cost management limits their ability to create value with AI. In fact, Gartner believes that cost is as important as security or illusion in AI risk.

Without an understanding of how GenAI costs scale, CIOs’ cost calculations could be off by as much as 5 to 10 times, according to Gartner estimates.

“CIOs need to understand the cost structure of AI, not only the cost components and pricing model options, but also how to reduce costs and negotiate with vendors,” LeHong said. “CIOs should create proofs of concept that focus on evaluating cost scalability in addition to testing the feasibility of the technology.”

Ubiquitous data and AI bring new challenges and risks

AI and data are widely used in enterprises and are no longer centralized assets directly controlled by IT departments. A Gartner survey of more than 300 CIOs found that in the future, on average, only 35% of AI capabilities will be built by internal IT teams. This means that enterprises need to adopt new methods to manage and protect data access, govern AI input and output, and securely realize AI value.

Describing the future AI technology stack, LeHong said: "Here we will mention the concept of 'technology architecture combination'. The bottom of the combination is all the assets from IT, which are usually centralized assets; the top is the assets from all over the enterprise, which are usually decentralized assets; in the middle is the trust, risk and security management (TRiSM) technology that ensures the security of all assets. This combination framework helps enterprises deal with various types of AI and data."

“CIOs should design a technology architecture portfolio that can handle AI complexity while remaining open to new opportunities,” Mesaglio said. “Organizations taking a moderate pace of AI (ten AI projects or fewer) will manage their technology architecture portfolio through human teams and committees. Organizations taking an accelerated pace of AI will ensure the security of large-scale AI applications by introducing TRiSM technologies—a set of technologies for establishing trust, monitoring risk, and managing security.”

Using AI is a double-edged sword for employee performance and well-being

Some employees may enjoy AI tools, while others may feel threatened or resentful. Strong reactions to AI may lead to unintended behavioral outcomes that negatively impact employee performance, such as envy of colleagues using AI and over-reliance on AI tools.

Yet few organizations are actively managing these behavioral outcomes: Only 20% of CIOs in the Gartner survey, conducted in June and July 2024, said they were committed to mitigating the potential negative impacts of GenAI on employee well-being.

“Most companies don’t care much about how their employees feel about AI, but this is important because AI can lead to all kinds of unintended behavioral outcomes,” Mesaglio said. “The key is to address this issue through change management by clearly identifying the right people to own the behavioral outcomes and managing them as rigorously as technical and business outcomes.”