Using model guardrails to regulate GenAI’s behavior and output

Currently, many organizations are working hard to design and implement generative artificial intelligence (GenAI) solutions, hoping to improve the versatility and creativity of solutions and drive business value. The 2023 Gartner Enterprise Artificial Intelligence (AI) Survey revealed the three main implementation methods of GenAI use cases. 74% of respondents customized existing GenAI models to meet the needs of their own use cases, and 65% of respondents tried to train customized GenAI models themselves.

However, implementing GenAI is not an easy task. The pursuit of creativity and versatility often increases the complexity, uncertainty, and possibility of generating unexpected results of GenAI solutions, which has become a major problem for GenAI enterprise adoption. The more creative and versatile the GenAI solution is, the higher the possibility of unexpected behaviors and outputs (such as hallucinations, harmful content beyond the scope of application, etc.) (see Figure 1).

Figure 1: Striking a balance between creativity and versatility

For enterprise organizations that choose to build their own GenAI solutions based on GenAI models, their data and analysis (D&A) leaders responsible for AI work should use three types of guardrail tools: open source guardrails, commercial guardrails, and self-built guardrails to control the input and output of GenAI models, verify and correct the input and output of GenAI models, and improve the reliability of the models.

Evaluate and optimize the creativity and versatility of GenAI solutions

GenAI models can be both creative and versatile. Organizations often need to use GenAI solutions in a wide range of scenarios, and these scenarios have different requirements for solution creativity and versatility. Therefore, it is necessary to determine the positioning of GenAI solutions based on the deployment purpose and functional requirements in specific scenarios, and use guardrail tools to establish control strategies and mechanisms based on the specific needs in terms of creativity and versatility.

D&A leaders responsible for AI efforts should determine the appropriate business risk tolerance based on the scenarios and methods in which the GenAI solution will be used, depending on:

• Importance of business use cases

• For internal use or for customers

• Whether human supervision is introduced

Then, based on business risk tolerance, guardrail tools should be used to manage model inputs and outputs, establish strict or loose control mechanisms, and ultimately achieve the best balance between creativity and versatility.

Use model guardrails to validate and correct model inputs and outputs

A practical way to manage the creativity and versatility of GenAI models is to use guardrail tools. Guardrails (which establish a protective layer between GenAI models and applications and end users) can monitor and manage all traffic to the model, including user input and model/application output (see Figure 2).

Figure 2: Guardrails are deployed between users and GenAI models

The following are two typical guardrails:

• Model guardrails to control end-user input: All user requests must be filtered through guardrails to remove unexpected requests, including requests that are beyond the scope of the GenAI solution and requests that violate the acceptable use policy. In this way, guardrails can control the versatility of the solution within a manageable range, just like establishing a safety fence.

  
  

• Model guardrails to control GenAI outputs: All model outputs must be validated with guardrails, but different use cases have different requirements for model creativity and thus require different levels of control over the model. However, in the case of developing a GenAI-powered enterprise search engine or a customer-facing chatbot, model outputs must be more rigorously validated and controlled to moderate model creativity and ensure that end users can get reliable and expected results.

It is important to note that guardrails are not a panacea and cannot completely solve the behavior and accuracy issues of GenAI solutions. GenAI solutions must strike a balance between accuracy and the risk tolerance of the enterprise organization.

In addition, as GenAI develops rapidly, guardrail technology is also constantly changing and improving. Given the unpredictability of neural networks, guardrail technology is currently a practical way to verify and correct the output of GenAI models. In the long run, before the basic GenAI models become sufficiently reliable and trustworthy, guardrails provide a transitional solution that can help enterprises and organizations promote GenAI adoption.