[SOLVED] Why Azureml studio handles the pip packages and conda packages to create environments and not only one?

Issue

I'm following the azure path for the dp-100 certification and notice that to create an environment some packages are installed with conda and others with pip:

# Ensure the required packages are installed
packages = CondaDependencies.create(conda_packages=['scikit-learn','pip'],
                                    pip_packages=['azureml-defaults'])
sklearn_env.python.conda_dependencies = packages

Why is that? will it be any change if I change that for:

packages = CondaDependencies.create(conda_packages=['scikit-learn','pip','azureml-defaults'])
sklearn_env.python.conda_dependencies = packages

or

packages = CondaDependencies.create( pip_packages=['azureml-defaults','scikit-learn'])                                  
sklearn_env.python.conda_dependencies = packages

If so... how should I decide for which packages to use conda and for which to use pip?

For more context this is the complete block of code:

from azureml.core import Experiment, ScriptRunConfig, Environment
from azureml.core.conda_dependencies import CondaDependencies

# Create a Python environment for the experiment
sklearn_env = Environment("sklearn-env")

# Ensure the required packages are installed
packages = CondaDependencies.create(conda_packages=['scikit-learn','pip'],
                                    pip_packages=['azureml-defaults'])
sklearn_env.python.conda_dependencies = packages

# Create a script config
script_config = ScriptRunConfig(source_directory='training_folder',
                                script='training.py',
                                environment=sklearn_env) 

# Submit the experiment
experiment = Experiment(workspace=ws, name='training-experiment')
run = experiment.submit(config=script_config)
run.wait_for_completion()

And this is the source

Solution

If so... how should I decide for wich packages use conda and for wich use pip?

According to documentation:

If your dependency is available through both Conda and pip (from PyPi), use the Conda version, as Conda packages typically come with pre-built binaries that make installation more reliable.

For example:

from azureml.core.environment import CondaDependencies

   myenv = Environment(name="myenv")
   conda_dep = CondaDependencies()
   conda_dep.add_conda_package("scikit-learn")

References: Understanding Conda and Pip, Unable to access python packages installed in Azure ML and Azure ML not able to create conda environment

Answered By - DeepDave-MT

Answer Checked By - Clifford M. (WPSolving Volunteer)