I trained a local 7B model to be better than GPT-4 at writing SQL
Fine-tuned Mistral 7B on 100k SQL query pairs with DPO. It now beats GPT-4 on Spider benchmark by 12%. Runs on a single RTX 4090. Model weights and training code included.
Content
# SQLMaster-7B
```
Spider Dev Set:
GPT-4-turbo: 78.2%
SQLMaster-7B: 87.6% ← +12%
Claude 3.5: 82.1%
```
```python
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("sqlmaster/7b-v1")
```
Model weights on HuggingFace: `sqlmaster/7b-v1`#Fine-tuning#SQL
49.3k
0