Is creating an in-house LLM right for your organisation?

Business leaders are feeling the pressure to incorporate generative AI into their strategies, aiming to achieve the best outcomes for their organisations and stakeholders. According to Gartner, 38% of leaders cited customer experience and retention as the primary focus of their generative AI investments, highlighting its crucial role in the future of business. However, before jumping into the world of large language models (LLMs), it’s essential to evaluate if these tools align with your business needs.

While off-the-shelf LLMs are readily available and easy to access, their effectiveness can be limited by several challenges. These models often provide a generic customer experience that may lack industry-specific context. Moreover, outsourcing embedding models can be costly, and sharing data externally raises significant privacy concerns. Training an in-house AI model can mitigate these issues, fostering creativity and innovation within your team as they explore the model’s potential across various projects. If you determine that a domain-specific AI is necessary for your business, here are five key questions to guide you in creating your own in-house model.

Question 1: What business problem are you trying to solve with AI?

Before diving into LLMs, take a step back to clearly define the problem you want to address. Once identified, determine the specific natural language tasks you need, such as summarisation, named entity recognition, semantic textual similarity, or question answering.

It’s important to distinguish between downstream tasks and domain awareness. While popular LLMs like GPT, Llama, and PaLM excel in downstream tasks (e.g., summarisation or question answering), they often lack the industry-specific knowledge required for many applications. Success in downstream tasks does not guarantee that the model will possess the domain awareness needed for your specific industry.

Question 2: Are there existing industry-specific AI tools?

During the research phase of your AI strategy, it’s crucial to evaluate available tools, especially those tailored to your industry. However, even industry-specific tools might lack the nuances necessary for your business. Ensure that any AI model you consider can understand the context and respond accurately in the language most relevant to your users.

Context is key because generative AI can still "hallucinate", producing inaccurate information on certain topics. This is why the Biden-Harris Administration has emphasised the importance of safe, secure, and trustworthy AI in its executive order. While this order applies to government agencies, private-sector businesses should also consider adopting similar standards.

Read more: The evolution of AI and its Impact on various industries.

Question 3: Is your data ready for AI training?

The quality of your organisation’s data is the most critical factor in training your own LLM. Companies with high-quality data are at a significant advantage, as data is essential at every stage of the AI development process, from training to testing and re-training. High-quality data typically requires minimal curation and retraining, which is crucial for success.

However, many companies find their data isn’t AI-ready, encountering issues like noisy data, poor labelling, or hidden repetitions that add little value. Organising and preparing data can be time-consuming, potentially taking years before it’s fully ready for AI training.

Question 4: Do you have the necessary experts to train AI models?

Experts play a vital role in data generation and quality control during the AI training process. While synthetic data sets exist, they require human evaluation to ensure their reliability. Select experts with deep industry knowledge to fine-tune your model, whether they are in-house or outsourced. Their expertise will be crucial in labelling, testing, and retraining data to achieve accurate, reliable results.

Question 5: What are your time constraints?

Developing an in-house AI model is both time-consuming and costly. Factors such as the complexity of the business problem, data quality, and the availability of experts and AI engineers all influence the project's duration and success. The process involves trial and error, and optimising hyperparameters like learning rates and epochs can add extra time.

Despite careful planning, there’s always a risk that new LLM solutions may emerge, rendering your model outdated. Balancing timing with quality is essential, given the rapid pace of AI development.

In conclusion, there’s no one-size-fits-all approach to AI. For business leaders, the decision to train an LLM from scratch may seem daunting, but with the right data and a domain-specific problem, the investment can yield significant long-term benefits.