Both model specification and validation methods have optimization embedded into the process.
Gallery
Automated Machine Learning
Hard vs Soft
β’
Weaker models with well-tuned hyperparameters can outperform fancier models β On the State of the Art of Evaluation in Neural Language Models (Melis et al. 2018).
β’
Graduate Student Descent (GSD) - A graduate student fiddles around with the hyperparameters until the model works (joke!)
β’
This process can be done by computer by testing multiple solutions and taking logs of the results.
Automated Methods
β’
Built-in with frameworks
β’
Popular algos:
β£ (Microsoft), β£ and β£ can be tested.
β’
For now the purpose of AutoML is not as a replacement tool for data scientist, but as a tool to improve efficiency.
D. Automation