improve

Author: abhishekkrthakur

docs/source/tasks/extractive_qa.mdx

@@ -1,36 +1,45 @@
-# Extractive Question Answering
+---
+title: "AutoTrain Extractive Question Answering - Train QA Models Easily"
+description: "Learn how to train extractive question answering models using AutoTrain. Simple guide for both local and cloud training with popular models like BERT."
+---
 
-Extractive Question Answering is a task in which a model is trained to extract the answer to a question from a given context. 
-The model is trained to predict the start and end positions of the answer span within the context. 
-This task is commonly used in question-answering systems to extract relevant information from a large corpus of text.
+# Extractive Question Answering with AutoTrain
 
+Extractive Question Answering (QA) enables AI models to find and extract precise answers from text passages. This guide shows you how to train custom QA models using AutoTrain, supporting popular architectures like BERT, RoBERTa, and DeBERTa.
 
-## Preparing your data
+## What is Extractive Question Answering?
 
-To train an Extractive Question Answering model, you need a dataset that contains the following columns:
+Extractive QA models learn to:
+- Locate exact answer spans within longer text passages
+- Understand questions and match them to relevant context
+- Extract precise answers rather than generating them
+- Handle both simple and complex queries about the text
 
-- `text`: The context or passage from which the answer is to be extracted.
-- `question`: The question for which the answer is to be extracted.
-- `answer`: The start position of the answer span in the context.
+## Preparing your Data
 
-Here is an example of how your dataset should look:
+Your dataset needs these essential columns:
+- `text`: The passage containing potential answers (also called context)
+- `question`: The query you want to answer
+- `answer`: Answer span information including text and position
 
+Here is an example of how your dataset should look:
 
 ```
 {"context":"Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.","question":"To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?","answers":{"text":["Saint Bernadette Soubirous"],"answer_start":[515]}}
 {"context":"Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.","question":"What is in front of the Notre Dame Main Building?","answers":{"text":["a copper statue of Christ"],"answer_start":[188]}}
 {"context":"Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.","question":"The Basilica of the Sacred heart at Notre Dame is beside to which structure?","answers":{"text":["the Main Building"],"answer_start":[279]}}
 ```
 
-
 Note: the preferred format for question answering is JSONL, if you want to use CSV, the `answer` column should be stringified JSON with the keys `text` and `answer_start`.
 
 Example dataset from Hugging Face Hub: [lhoestq/squad](https://huggingface.co/datasets/lhoestq/squad)
 
-
 P.S. You can use both squad and squad v2 data format with correct column mappings.
 
-## Training Locally
+## Training Options
+
+### Local Training
+Train models on your own hardware with full control over the process.
 
 To train an Extractive QA model locally, you need a config file:
 
@@ -75,12 +84,14 @@ $ autotrain --config config.yaml
 
 Here, we are training a BERT model on the SQuAD dataset using the Extractive QA task. The model is trained for 3 epochs with a batch size of 4 and a learning rate of 2e-5. The training process is logged using TensorBoard. The model is trained locally and pushed to the Hugging Face Hub after training.
 
-## Training on the Hugging Face Spaces
+### Cloud Training on Hugging Face
+Train models using Hugging Face's cloud infrastructure for better scalability.
 
 ![AutoTrain Extractive Question Answering on Hugging Face Spaces](https://raw.githubusercontent.com/huggingface/autotrain-advanced/main/static/ext_qa.png)
 
 As always, pay special attention to column mapping.
 
-## Parameters
+
+## Parameter Reference
 
 [[autodoc]] trainers.extractive_question_answering.params.ExtractiveQuestionAnsweringParams

docs/source/tasks/llm_finetuning.mdx

@@ -1,16 +1,27 @@
-# LLM Finetuning
-
-With AutoTrain, you can easily finetune large language models (LLMs) on your own data. 
-You can use AutoTrain to finetune LLMs for a variety of tasks, such as text generation, text classification, 
-and text summarization. You can also use AutoTrain to finetune LLMs for specific use cases, such as chatbots, 
-question-answering systems, and code generation and even basic fine-tuning tasks like classic text generation.
-
-Config file task names:
-- `llm`: generic trainer
-- `llm-sft`: SFT trainer
-- `llm-reward`: Reward trainer
-- `llm-dpo`: DPO trainer
-- `llm-orpo`: ORPO trainer
+---
+title: "LLM Finetuning with AutoTrain Advanced"
+description: "Complete guide to fine-tuning Large Language Models (LLMs) using AutoTrain Advanced. Learn how to prepare data, train models, and deploy them for text generation, chatbots, and more."
+keywords: "llm finetuning, language model training, autotrain, hugging face, nlp, machine learning"
+---
+
+# LLM Finetuning with AutoTrain Advanced
+
+AutoTrain Advanced makes it easy to fine-tune large language models (LLMs) for your specific use cases. This guide covers everything you need to know about LLM fine-tuning.
+
+## Key Features
+- Simple data preparation with CSV and JSONL formats
+- Support for multiple training approaches (SFT, DPO, ORPO)
+- Built-in chat templates
+- Local and cloud training options
+- Optimized training parameters
+
+## Supported Training Methods
+AutoTrain supports multiple specialized trainers:
+- `llm`: Generic LLM trainer
+- `llm-sft`: Supervised Fine-Tuning trainer
+- `llm-reward`: Reward modeling trainer  
+- `llm-dpo`: Direct Preference Optimization trainer
+- `llm-orpo`: ORPO (Optimal Reward Policy Optimization) trainer
 
 ## Data Preparation
 
@@ -145,6 +156,27 @@ Chat models can be trained using the following trainers:
 
 The only difference between the data format for reward trainer and DPO/ORPO trainer is that the reward trainer requires only `text` and `rejected_text` columns, while the DPO/ORPO trainer requires an additional `prompt` column.
 
+## Best Practices for LLM Fine-tuning
+
+### Memory Optimization
+- Use appropriate `block_size` and `model_max_length` for your hardware
+- Enable mixed precision training when possible
+- Utilize PEFT techniques for large models
+
+### Data Quality
+- Clean and validate your training data
+- Ensure balanced conversation samples
+- Use appropriate chat templates
+
+### Training Tips
+- Start with small learning rates
+- Monitor training metrics using tensorboard
+- Validate model outputs during training
+
+### Related Resources
+- [AutoTrain Documentation](https://huggingface.co/docs/autotrain)
+- [Example Fine-tuned Models](https://huggingface.co/models?pipeline_tag=text-generation&sort=downloads) 
+- [Training Datasets](https://huggingface.co/datasets?task_categories=task_categories:text-generation)
 
 ## Training