Automated machine learning

VTU COE PG Reports / 29 Jul, 2024

1. Auto-ML– Automated machine learning (Auto-ML) platforms are gaining popularity and taking over various aspects of the data science lifecycle. These platforms automate tasks such as data sourcing, feature engineering, conducting machine learning experiments, evaluating and choosing the most effective models, and deploying them into production environments.

   
   

   
   

   AutoML 

   Automated Machine Learning, or AutoML, is one of the latest trends that is driving the democratization of data science. A huge part of a data scientist’s job is spent on data cleansing and preparation, and each of these tasks are repetitive and time-consuming. AutoML ensures that these tasks are automated, and it involves building models, creating algorithms and neural networks.  

   AutoML is essentially the process of applying ML models to real-world issues by leveraging automation. AutoML frameworks help data scientists in data visualization, model intelligibility and model deployment. The main innovation in it is hyperparameters search, utilized for preprocessing components, model type selection, and for optimizing their hyperparameters.

   What is AutoML?

   Automated Machine Learning provides methods and processes to make Machine Learning available for non-Machine Learning experts, to improve efficiency of Machine Learning and to accelerate research on Machine Learning.

   Machine learning (ML) has achieved considerable successes in recent years and an ever-growing number of disciplines rely on it. However, this success crucially relies on human machine learning experts to perform the following tasks:

   • 1.Preprocess and clean the data.
   • 2.Select and construct appropriate features.
   • 3.Select an appropriate model family.
   • 4.Optimize model hyperparameters.
   • 5.Design the topology of neural networks (if deep learning is used).
   • 6.Postprocess machine learning models.
   • 7.Critically analyze the results obtained.

   As the complexity of these tasks is often beyond non-ML-experts, the rapid growth of machine learning applications has created a demand for off-the-shelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML.

2. AutoML to Advance and Improve Research

   Making a science of model search argues that the performance of a given technique depends on both the fundamental quality of the algorithm and the details of its tuning and that it is sometimes difficult to know whether a given technique is genuinely better, or simply better tuned. To improve the situation, Bergstra et al. proposed reporting results obtained by tuning all algorithms with the same hyperparameter optimization toolkit. Sculley et al.’s ICLR’18 workshop paper Winner’s Curse argues in the same direction and gives recent examples in which correct hyperperameter optimization of baselines improved over the latest state-of-the-art results and newly proposed methods.

   Hyperparameter optimization and algorithm configuration provide methods to automate the tedious, time-consuming and error-prone process of tuning hyperparameters to new tasks at hand. We for example provide packages for hyperparameter optimization:

   • SMAC3 – a python re-implementation of the SMAC algorithm
   • DEHB: Differential Evolution combined with HyperBand