Building Domain-Specific LLMs: Efficient Fine-Tuning

General-purpose LLMs are powerful, but domain-specific tasks demand tailored performance. In this session, we will explore how to efficiently adapt LLMs for industry use cases (e.g., legal, healthcare, aviation) using modern fine-tuning techniques like LoRA, adapter layers, and instruction tuning. Participants will learn how to customize models using minimal compute while maintaining high relevance and accuracy.

Key Takeaways:

• Why domain-specific fine-tuning matters
• Overview of PEFT methods (LoRA, Adapters, Prefix Tuning)
• Instruction tuning & Chain-of-Thought prompting
• Data prep, hyperparameter tuning, evaluation metrics
• Toolkits: Hugging Face, PEFT library, Azure ML workflows

Prerequisites:

Basic understanding of LLMs and transformers

Python + ML experience (recommended but not mandatory)

Pradeep Kumar, a Dubai-based Software and AI/ML Engineer, brings over 12 years of expertise in designing innovative solutions across industries. Holding a Master’s in AI and ML from Liverpool John Moores University, he leverages cutting-edge technologies to optimize processes and drive innovation. Pradeep is the creator of the popular newsletter Air Around AI, sharing insights on the latest AI trends and developments. His technical expertise and thought leadership make him a prominent figure in the ever-evolving AI landscape. You can reach him on LinkedIn.