Why right-sizing LLMs is so important for channel partners

Why right-sizing LLMs is so important for channel partners

Ramprakash Ramamoorthy, Director of AI research, ManageEngine

Integrating Large Language Models into enterprise IT poses significant challenges. A critical success factor is the concept of right-sizing and developing LLMs of optimal size and configuration to match business needs. Channel partners, who play a role in technology implementation, struggle with this, says Ramprakash Ramamoorthy at ManageEngine, and provides a way forward.


Key takeaways

  • Integrating LLMs into enterprise IT poses significant challenges and channel partners struggle with this.
  • A critical success factor is right-sizing, selecting LLMs of optimal size and configuration to match business needs.
  • At the heart of the AI revolution are deep LLMs, trained on vast datasets, often with billions or trillions of parameters.
  • LLMs are based on transformer networks, allows them to perform natural language tasks such as generation, translation, sentiment analysis.
  • While scale contributes to capabilities, it also brings challenges in terms of computational resources for training and deployment.
  • In contrast to large models are micro LLMs, or small language models, which are specialised models for particular domains or tasks.
  • Micro LLMs, fine-tuned versions of larger models, align closely with industry and customer needs.
  • Micro LLMs offer advantages like improved domain accuracy, lower computational costs, reduced latency, enhanced data privacy.
  • Utilising a buy-and-build strategy, combining foundational models with smaller models, can deliver rapid implementation and lower risks.
  • LLMs can be categorised based on their training and use cases, including general-purpose, instruction-tuned, and dialog-tuned models.
  • Concept of right-sizing LLMs is paramount for enterprises aiming to leverage this technology.
  • Right-sizing involves selecting models that balance size, complexity, training data to meet business needs.
  • Contrary to the belief that bigger is better, smaller, specialised models perform specific tasks more efficiently.
  • This approach reduces energy consumption and ensures sustainable AI integration.

The enterprise landscape is undergoing a transformation driven by rapid advancements in AI, particularly in the realm of Large Language Models, LLMs. In the United Arab Emirates and globally, businesses are increasingly aware of LLMs’ potential to revolutionise operations, from enhancing customer interactions to optimising data analysis. These AI systems, capable of understanding and generating human-like text, provide opportunities to boost efficiency, drive innovation, and gain a competitive edge.

Integrating LLMs into enterprise IT poses significant challenges. A critical factor for success is the concept of right-sizing, selecting or developing LLMs of optimal size and configuration to match specific business needs. Channel partners, who play a crucial role in technology adoption, often struggle with this.

At the heart of the current AI revolution are deep LLMs, trained on vast datasets, often with billions or trillions of parameters. Their architecture, based on transformer networks, allows them to perform a variety of natural language tasks such as generation, translation, and sentiment analysis. While their scale contributes to powerful capabilities, it also brings challenges in terms of computational resources for training and deployment.

In contrast to these large, general-purpose models, the concept of micro LLMs, or small language models, are specialised models designed for particular domains or tasks. Often fine-tuned versions of larger models, micro LLMs align more closely with industry or customer-specific needs. These models offer advantages like improved domain accuracy, lower computational costs, reduced latency, and enhanced data privacy via local deployment.

For channel partners, understanding and utilising a buy-and-build strategy, combining foundational models with fine-tuned, smaller models, can deliver rapid implementation and lower risks. LLMs can also be categorised based on their training and use cases, including general-purpose, instruction-tuned, and dialog-tuned models.

The concept of right-sizing LLMs is paramount for enterprises aiming to leverage this technology effectively. It involves selecting models that balance size, complexity, and training data to meet business needs without incurring unnecessary resource use.

Contrary to the belief that bigger is always better, smaller, specialised models often perform specific tasks more efficiently. This strategic approach not only reduces costs and energy consumption but also ensures sustainable AI integration.

To address these issues, channel partners can follow a structured roadmap:

#1 Define business objectives and use cases

The initial and most crucial step involves close collaboration with enterprise customers to clearly identify the specific business problems that LLMs are intended to solve and to define the desired outcomes and measurable KPIs for AI implementation. The focus should always be on the value proposition and the unique needs of the client rather than simply adopting the latest technology for its own sake.

#2 Assess data requirements and quality

Channel partners need to guide their enterprise customers in evaluating the availability, accessibility, quality, and relevance of the data that will be necessary for either fine-tuning existing LLMs or for informing the selection of appropriate pretrained models. This includes emphasising the importance of data cleaning, preprocessing, and understanding the implications of data privacy regulations.

#3 Evaluate LLM options and sizes

A structured framework for evaluating a range of LLM models is necessary. This should include large, general-purpose models and smaller, domain-specific micro LLMs, and open-source and proprietary options. The evaluation criteria should encompass model capabilities, performance benchmarks relevant to the identified use cases, the cost of usage and deployment, and the associated infrastructure requirements. Platforms like BytePlus ModelArk can provide valuable insights into the impact of model size on performance and efficiency.

#4 Implement right-sizing

Channel partners can leverage various techniques to adapt LLMs to specific tasks and potentially reduce the need for excessively large models. Effective prompt engineering can guide model output with carefully crafted instructions. Implementing retrieval-augmented generation can ground LLM responses with relevant information from external knowledge sources, improving accuracy and reducing hallucinations.

Targeted fine-tuning on domain-specific datasets can further optimise model performance for specific applications. A recommended approach is to begin with larger models for initial proof-of-concept and then progressively explore smaller models while optimising prompts and utilising techniques like few-shot learning to maintain performance.

#5 Prioritise security, privacy, ethics

Implementing robust security measures to protect sensitive enterprise data throughout the LLM life cycle is vital. This includes secure data handling practices, access controls, and ensuring compliance with relevant data privacy regulations. Addressing potential biases in LLM outputs is also a crucial ethical consideration.

#6 Adopt pilot projects and iterations

Start with focused pilot projects to test feasibility and effectiveness of LLM solutions for specific use cases before undertaking broader deployments is a prudent strategy. Continuous monitoring of model performance, gathering user feedback, and iteratively refining the models and implementation strategies based on real-world results are essential for long-term success.

#7 Foster learning and upskilling

Invest in training and development programs to equip enterprises with the necessary expertise in LLM technology, right-sizing techniques, and best practices for enterprise implementation is crucial. Ensuring enterprise customers stay informed about the latest advancements in the rapidly evolving field of AI is an ongoing necessity.

#8 Consider partnerships

Channel partners should explore collaborations with specialised AI vendors or consultants to augment their in-house expertise and gain access to advanced tools and platforms for LLM development and deployment.

LLMs offer vast potential for enterprises in the UAE. Channel partners are key to unlocking this potential, but they must overcome challenges in model right-sizing.

By following a structured roadmap, investing in expertise, and aligning with strategic AI practices, channel partners can guide enterprise customers in adopting cost-effective, efficient LLM solutions. This positions them as trusted advisors and paves the way for innovation and growth in the age of enterprise AI.


How ManageEngine leverages narrow AI and selective LLMs

ManageEngine’s IT management tools, that leverage right-sized LLMs, can enhance enterprise IT operations. They can improve network anomaly detection, enable precise predictive maintenance, and streamline IT support ticket categorisation. They also facilitate faster, more accurate retrieval of information from knowledge bases.

ManageEngine employs a balanced AI strategy, using narrow AI for routine tasks and LLMs for complex challenges, ensuring high performance and cost-efficiency.

ManageEngine exemplifies this strategy with its proactive AI approach. The company focuses on preventing IT issues and optimising operations using narrow, purpose-built AI and selectively deployed LLMs. This aligns with right-sizing principles, matching the appropriate AI model to each task. ManageEngine’s practical, value-driven philosophy is reflected in its AIOps platform, which uses machine learning to automate tasks and provide actionable insights.

Zoho Corp, ManageEngine’s parent company, strengthens this vision with significant investment in in-house LLMs, such as Zia LLM. Emphasising contextual AI, data privacy, and customer control, Zoho’s initiatives, like Zia Agents and AI Bridge, demonstrate a strong commitment to responsible AI adoption. Collaborations like the one with NVIDIA to develop industry-specific LLMs further reinforce this.

In the context of ManageEngine’s IT management tools, right-sized LLMs can enhance various operations. They can improve network anomaly detection, enable precise predictive maintenance, and streamline IT support ticket categorisation.

They also facilitate faster, more accurate retrieval of information from knowledge bases. ManageEngine employs a balanced AI strategy, using narrow AI for routine tasks and LLMs for complex challenges, ensuring high performance and cost-efficiency.


How channel partners can become strategic advisors in enterprise AI

Channel partners can advance opportunities in LLM right-sizing by providing strategy consulting. Services can include identifying suitable use cases, selecting optimal models, fine-tuning with proprietary data, and providing deployment support.

By following a structured roadmap, investing in expertise, and aligning with strategic AI practices, channel partners can guide enterprise customers in cost-effective LLM solutions. This positions them as trusted advisors and paves the way for growth in enterprise AI.

By integrating right-sizing into their offerings through AI-readiness assessments, data preparation guidance, prompt engineering, and model performance monitoring, channel partners can deliver continuous support.

This ensures enterprise customers stay updated on advancements and adapt strategies accordingly.

Several roadblocks hinder effective rightsizing.

  • Limited understanding of model architecture, difficulty assessing model suitability for specific tasks, and a lack of tools for domain-specific evaluations.
  • Preparing quality training data is complex and resource-intensive, and many enterprises lack infrastructure and expertise.
  • Fine-tuning requires specialised knowledge, and data privacy concerns can deter cloud-based deployments.
  • Even deploying moderate sized models requires significant computational resources.
  • Monitoring LLM output and mitigating distortion also pose challenges.

Browse our latest issue

Intelligent Tech Channels

View Magazine Archive