C1M1: AI foundations on GC

Cloud
ML
Author

Chi Zhang

Published

January 5, 2025

Keywords

Vertex AI, Gemini multimodal, autoML, BigQuery ML, Healthcare Data Engine

C1M1

Google Cloud infrastructure

Networking and security -> Compute and storage -> Data and AI products

Compute

  • compute engine, GKE (kubernetes), App engine, cloud run, cloud run functions
  • Processing power: CPU, GPU, TPU (tensor processing unit)

Storage

  • Cloud SQL, Spanner, Firestore, Bigtable, BigQuery
  • structured data: transactional workload, analytical workload (SQL: BQ; NoSQL: Bigtable)

Data and AI products

Data-to-AI workflow: ingestion and process, storage, analytics (BQ), AI/ML (VertexAI)

Vertex AI: AutoML, Workbench, Colab Enterprise, Vertex AI Studio, Model Garden

ML models

Supervised learning

  • classification (logistic reg)
  • regression (linear reg)

Unsupervised learning

  • clustering (K-means)
  • association (association rule learning)
  • dimensionality reduction (PCA)

BigQuery

Can create ML model inside BQ

C1M2: AI development options

Preconfigured (pre-trained APIs) - low-code / no-code (BQML, AutoML on vertex AI) - DIY (custom training)

Pre-trained APIs

  • Speech, text, languages
  • image and video
  • document and data
  • conversational AI

AutoML inside vertex AI, with UI and no code

Custom training: e.g. tensorflow. All hosted on vertexAI

  • lowest level: hardware (CPU, GPU, TPU)
  • low-level TF APIs: core tensorflow (C++, python)
  • TF model libraries: tf.layers, tf.losses, …
  • High-level TF APIs: tf.keras, tf.data

C1M3: AI development workflow

AutoML with vertex AI: no-code approach through UI, user-friendly

Vertex AI workbench or colab: code-based, using SDKs, experienced users

Image data

  • classification (single-label or multi-label)
  • object detection
  • segmentation

Tabular data

  • regression/classification
  • forecasting

Text data

  • classification
  • entity extraction
  • sentiment analysis

Video

  • object tracking
  • classification

Evaluation; Deploy and monitor

MLOps and workflow automation

ML development

  • upload data
  • engineer feature
  • train model
  • evaluation result

Operations

  • deploy
  • monitor
  • release

C1M4: Generative AI

Interface: Vertex AI studio UI - backend models: Gen AI (multimodal, language, image, code, speech)

Gemini multimodal

Prompt design

Prompt: a request to a model for a desired outcome. Input (required), context (optional), example (optional)

  • zero-shot: single command to LLM without any example
  • one-shot: provide a single example
  • few-shot: a few examples

Prompt design does not change parameters of pre-trained model, and does not require ML background or code skills

Results might not be stable

Model tuning

Parameter-efficient tuning

  • adapter tuning
  • reinforcemnet
  • distilling