How to Build LLMs on Your Company’s Data While on a Budget

How to Build LLMs on Your Company’s Data While on a Budget

Step-by-Step Tutorial:

1. Introduction to the Session:

   • The session is about building LLMs on your own data.
   • The speaker, Sean Owen, is a principal practice lead on ML and data science at Databricks with extensive industry experience.

2. Understanding the Use Case:

   • The use case involves using large language models (LLMs) on your data without breaking the bank.
   • Examples include customizing language models for specific data sets like gardening information.

3. Exploring Different Options:

   • Consider using search engines, specialist sites, or AI models like Dolly for answering questions about your data.
   • Customizing an LLM for your private data sets can be achieved by fine-tuning a model or supplying relevant information at runtime.

4. Choosing the Right Tools:

   • Options for language engines include Dolly, MPT, Falcon, or other open-source models.
   • Use tools like Chroma for vector databases to assist in semantic similarity searches.

5. Training and Fine-Tuning the Model:

   • Select a suitable LLM model based on your data and fine-tune it using tools like deep speed or hugging face trainer.
   • Consider memory constraints, GPU options, and training settings like batch size and gradient checkpointing.

6. Data Preparation and Indexing:

   • Prepare your data set with questions and answers or relevant text chunks for indexing.
   • Ensure the data fits the model's context window length and structure for effective training.

7. Fine-Tuning Process:

   • Fine-tune the model with your data set to induce the desired behavior for generating responses.
   • Monitor the training process, evaluate the model's performance, and adjust settings as needed to avoid overfitting.

8. Generating Answers:

   • Use the fine-tuned model to generate answers to questions by feeding context from a vector database.
   • Experiment with different generation settings like beam search, temperature, and nucleus sampling for varied responses.

9. Testing and Optimization:

   • Test the model's responses, analyze the results, and optimize the training process to improve performance.
   • Address challenges like overfitting, training loss patterns, and memory constraints for better outcomes.

10. Closing Remarks and Next Steps:

    • Explore demo sites like dbdemos.ai for a full demonstration of the process.
    • Consider further training and certification opportunities at Databricks to enhance your skills in ML and data science.