Gartner: Enterprises face new security threats when deploying generative AI

At the recent 2024 China Technology Industry Executive Exchange Conference, Gartner released its latest research on the security of generative artificial intelligence (GenAI). With the rapid development and application of GenAI technology, more than 58% of technology providers around the world have embedded GenAI technology in their products. Although this technology has brought innovation opportunities to enterprises, it has also introduced new security challenges.

Shaoqing Zeng, senior research director at Gartner, said: "GenAI technology has great potential in improving business efficiency and developing new services, but it also expands the security attack surface of enterprises, especially in terms of prompt engineering and data security. GenAI technology provides attackers with new ways, such as using AI-generated content to conduct social engineering attacks or undermining the integrity of AI systems by contaminating training data. In addition, attackers can use GenAI to improve the automation and intelligence level of attack tools, which poses a challenge to traditional security defense measures. At the same time, the complexity of GenAI systems themselves may also introduce new vulnerabilities and weaknesses, requiring enterprises to adopt more advanced technologies and strategies to ensure the security of the system."

随着GenAI技术融入日常业务操作，企业面临的攻击面显著扩大。这不仅影响到数据处理，还直接威胁到AI模型的安全性。为了应对这些挑战，企业必须采取先进的安全措施和技术。这包括但不限于实施综合的数据监控系统、加强训练数据的安全性，以及部署专门的安全解决方案来保护AI模型免受操纵和攻击。更重要的是，企业需要在AI开发和部署的每个阶段整合安全最佳实践，确保从设计之初就考虑到安全因素。

Zeng Shaoqing pointed out: "In the rapidly developing field of GenAI, passive security defense alone can no longer meet protection needs. Enterprises must establish an automatic mobile target defense system during the design and operation of AI systems to prevent possible security threats. This forward-looking security strategy is the key to protecting enterprises from emerging technology risks."

At the same time, attackers are using GenAI's capabilities to improve the sophistication and efficiency of their attacks, posing new challenges to traditional security defense systems. Attackers use GenAI to conduct more automated and intelligent attacks, which not only improves the efficiency of attacks, but also increases the concealment and complexity of attacks. To meet these challenges, enterprises must adopt AI-based defense technologies that can monitor and analyze abnormal behaviors in real time to identify and block potential threats.

Zeng Shaoqing pointed out: "Attackers can use GenAI to quickly develop new attack methods, which requires our defense systems to not only respond quickly, but also be smart enough to identify and adapt to these new threats. Enterprises investing in AI-driven security solutions will be key to combating this new trend."

At the same time, the popularity of GenAI technology has brought about the complexity of data security management, and enterprises must adapt to this change to prevent growing security risks. GenAI can process and analyze large amounts of data, but if it is not properly managed, it may also lead to data leakage or malicious use. Therefore, enterprises need to implement strict data governance measures to ensure that all data processed using AI meets the latest security standards and compliance requirements. In addition, it is also crucial to strengthen the monitoring and auditing of AI operations to ensure data integrity and privacy protection.

Shaoqing Zeng said: "In this data-driven era, ensuring data security is a major challenge facing companies using GenAI technology. We must recognize that data is not only an asset, but also a potential risk point. The use of advanced encryption technology, access control and real-time monitoring systems are an integral part of protecting data security.