Manual - Gen AI Development

Ollama

[Ollama]((https://github.com/ollama/ollam) runs large language models locally.

Install Ollama via brew on Mac OSX:

brew install --cask ollama

Run the Ollam application and follow the step-by-step setup guide to install the Ollama command line tool.

To run a model locally:

# The models I am currently running locally:
 
ollama run phi3
ollama run llama3
ollama run gemma

Google Vertex AI

To run Jupyter notebooks in VS Code with google-cloud-sdk:

brew install --cask google-cloud-sdk
gcloud config set project YOUR_PROJECT_ID
 
# To install or remove components at your current SDK version [475.0.0], run:
# gcloud components install COMPONENT_ID
# gcloud components remove COMPONENT_ID
 
# To update your SDK installation to the latest version [475.0.0], run:
# $ gcloud components update
 
# Log into Google Cloud
gcloud auth application-default login
 
# Verify the configuration
gcloud config list

If you receive an error (ModuleNotFoundError: No module named 'imp') when running any of the above commands, try:

brew upgrade google-cloud-sdk
 
# As of 13/05/2024, gloud components update does not support python3.12+)
# We need to use python 3.11 to run the command
 
export CLOUDSDK_PYTHON=$(which python3.11)
gcloud components update
 
# gloud does work with python3.12+,
# we can safely unset the environment variable
 
unset CLOUDSDK_PYTHON
gcloud version