This are notes that I had taken while configuring VM on GCP, running mlflow server in it, and connecting it to my local Vs code.

These notes were written in 2025, so some GCP Console UI details and links may have changed. I'm running Windows 11 with WSL2 for development, and the terminal commands assume you're using PowerShell in VS Code.

Download and Install Google cloud SDK

Log in to your google account if not already using gcloud init.

This displays a link to log in page or directly opens up a browser window for log in.
Log in to your google account credentials.

This command is used for the initial setup and configuration of your gcloud CLI. It not only authenticates your account but also guides you through other common setup steps.

Use gcloud auth login when you only need to re-authenticate your gcloud CLI session (e.g., if your credentials expire) or switch to a different authenticated account without changing other configuration settings. It's a quick way to ensure your gcloud commands are running with the correct permissions.
This command gets your OAuth2 credentials and saves them locally.

Go to Google Cloud Console, Select your project.

Create the VM using Console (Web UI)

Navigation bar -> Compute Engine -> VM instances -> Create Instance

• Name : Name of instance (eg mlflow-server)
• Region: Region in which you want VM to live. Select one appropriately.
• Zone: Select appropriate zone
• Select machine type (e2-medium is good enough for developement, you can customize RAM)

Go to Left Panel

• OS and Storage

  • All defualts are good, click change if you want to change something.
  • eg. Operating system: Ubuntu, size of disk (10 GB is enough), etc.

• Network tab:

  • Deselect all Allow HTTP traffic, Allow HTTPS traffic, Allow Load Balancer Health Checks.
  • Add network tag (eg ssh-access, mlflow-server, etc): this comes handy later in selecting VM based on tag while creating a firewall rule.
  • Network interface: Select default network

• Security

  • Access scopes -> Select Set access for each API (to give least privilege to VM)
    • Enable for Cloud SQl, Storage (Full).
    • Note: These are the least necessary for mlflow development, if running any extra application that requires other resources like Bigquery, Modify access scope accordingly.

  Manage Aceess

  • Check Control VM access through IAM permissions.

Other default settings are good enough.

• Click Create

Enable os login

enable-osconfig : TRUE  
enable-oslogin : TRUE  

Note:
Google attaches a service account to newly created instance.
This account acts like an authentication API for VM.
You can create new service accounts explicitly for other resources in Security -> Service account section.

Reserve internal IP of VM instance

Create a PostgreSQL instance

Navigation bar-> Cloud SQL -> Instances -> Create Instance -> Choose PostgreSQL -> Fill options accordingly.

• Select Enterprise
• Select Editor Preset -> Sandbox for experimental work.
• Instance ID is name of your instance.
• Set passoword for default user postgres.
  • Choose strong password. Don't let @ be in password or use URL-encoding.
  • You can create other users and databases within this instance later through console.
• Choose zone, its better to have SQL instance in same zone as VM.
• Machine : Dedicated 1 vCPU, 3.75 GB is good enough.
• Storage: Default SSD 10 GB is enough
• Connections: Uncheck public ip, check Private IP and select default network.
  • Having default network to which VM is also connected allows easy connectivity between the two without firewall rule.
  • This way your SQL instance is not open to internet and only listens to default network.
• Enable Private Path
• Enable Vertex AI Integration in Flags and paramters (optional).
• Click Create Instance

Create a bucket to store artifatcs

Navigation bar -> Cloud Stoarge -> Buckets -> Create

• Name: Should be globally unique, better prefix with your project_id (eg project-name-123456-artifacts).
• Label (optional, to identify bucket)
• Choose where to store your data:
  • Select Region -> Your VM's region (its better if both bucket and VM live in same region).
• Choose how to store data: Select according to use case, Standard is good enough for development.
• Check Enable Hierarchical namespace on this bucket (optional)
• Choose how to control access to objects.
  • Check Enforce public access prevention on this bucket.
  • Access control: Uniform is good enough
• Choose how to protect object data: Defaults are good enough.

We set bucket as artifact storage later.

Enable IAP source

Security -> IAP (Identity Aware Proxy) -> SSH AND TCP RESOURCES tab

-> Select your VM instance, this will open info panel on right side.
-> Click ADD PRINCIPAL.
-> New Principles: Enter your google account id (your gmail id)
-> Assign Roles : Cloud IAP -> IAP Secured Tunnel User
-> Click save.

If you want to give any other user access to VM thorugh IAP, give that user permission similarly.

Allow traffic from IAP to VM

Create new firewall rule:

• Enter IPV4 source range 35.235.240.0/20 (IAP's range)
• target: VM tags or it's service account. (tags are preferred)
• Allow Ingress: tcp : 5000 (mlflow), 22 (ssh), and all tcp ports if IAP requires.
• Allow all tcp only if IAP requires (as specified in console), otherwise its NOT RECOMMENDED.
• To allow all tcp ports, just check tcp and don't enter any value for port.
• Priority of ports 22, 5000 should be lower (say 50) than all tcp rule (say 100). (Lower priority dominates.)

Grant access by assigning roles

Navigation bar -> IAM

• Select VM sevice account -> Edit button (on right side).
  • Assign following roles: Storage Object Reader and Writer (To handle Bucket operations)
• Select your google account
  • Assign Compute Instance Admin (v1), IAP-secured Tunnel User, Compute OS Admin Login (To login as a admin user with sudo previlege)
    • For standard user without sudo previlege, assign Compute OS Login instead of Compute OS Admin Login. (but not both simultaneously).

Now on Local VS code or powershell:

1. Make sure OpenSSH client is installed.
2. Make sure gcloud.cmd is in PATH system environment variable. Take help of LLM's on how to do this on your system.

Connect to VM using Remote-SSH extension

Install Remote-SSH extention in VS code if not already installed.

1. connect via ssh: Enter this command in powershell

gcloud compute ssh <VM-NAME> --tunnel-through-iap --zone=<zone>  --project "<PROJECT-ID>"

1. Create config file in folder where key is created. Add below line to config file

Host <alias>  # give any name eg. my-instance
    User <your-username-on-gcp>  # eg user_example_gmail_com
    IdentityFile <Path to private key generated in above step>  
    ProxyCommand "<path to gcloud.cmd>" compute start-iap-tunnel <VM-NAME> <PORT> --listen-on-stdin \
            --project=<project-id> --zone=<your-instance-zone>

After you are in VM, install python and mlflow.

sudo apt update
sudo apt install python3 python3-pip

pip3 install mlflow psycopg2-binary
pip install google-cloud-storage

start mlflow server

mlflow server \
    --backend-store-uri "postgresql://postgres:<your_postgres_password>@<cloud_sql_private_ip>:5432/<database-name>" \
    --default-artifact-root="gs://bucket-name"

Alternatively you can export these arguments as environment variables in bashrc file:

nano ~/.bashrc

# Add below lines at end of file
export MLFLOW_TRACKING_URI="postgresql://postgres:<your_postgres_password>@<cloud_sql_private_ip>:5432/<database-name>"
export MLFLOW_ARTIFACT_ROOT="gs://bucket-name"

# Save file and execute below command
source ~/.bashrc

# Then run this
mlflow server --backend-store-uri=$MLFLOW_TRACKING_URI --default-artifact-root=$MLFLOW_ARTIFACT_ROOT

This will start mlflow server on port 5000. Open it in local browser.

If above option does not work, you can manually create an IAP tunnel between ports of VM and your local machine.

Manually create an IAP tunnel between ports of VM and local machine.

1. SSH into VM

gcloud compute ssh <VM-NAME> --tunnel-through-iap --zone=<zone>  --project "<PROJECT-ID>"

Install mlflow and required packages.

1. Start mlflow server export variables in bashrc file as shown above.

mlflow server --backend-store-uri=$MLFLOW_TRACKING_URI --default-artifact-root=$MLFLOW_ARTIFACT_ROOT

Tunnel VM's port 5000 to your local machine's 5000 port using start-iap-tunnel:

• Start iap tunnel between ports: In local VS code, in new powershell:

gcloud compute start-iap-tunnel <VM_NAME> 5000 --local-host-port=localhost:5000 --project=<project-id> --zone=<zone>

After this command terminal displays

Testing if tunnel connection works.
Listening on port [5000].

and freezes, means tunnel has been made. Open localhost:5000 port in your browser now to see mlflow ui.

Create new shell, ssh into VM using gcloud compute ssh as above to execute any other scripts in VM.