BERT and the fine-tuning era (2018)
Google researchers released BERT, a pretrained language model that developers fine-tuned on specific tasks such as classification and entity extraction. It beat prior task-specific systems on several standard benchmarks.
Fine-tuned encoders became the default for document extraction and classification.
Generative models later absorbed many of those tasks through prompting, but specialized models remained cheaper for high-volume extraction.
d1.2 makes a similar bet on task-specific scoring, but sells it as a hosted API rather than a downloadable checkpoint.
