Manual - Gen AI Development

Ollama

[Ollama]((https://github.com/ollama/ollam) allows you to run large language models locally.

Instal Ollama via brew on Mac OSX:

brew install --cask ollama

Run the Ollam application and follow the step-by-step setup guide to install the Ollama command line tool.

To run a model locally:

# The models I am currently running locally:
 
ollama run phi3
ollama run llama3
ollama run gemma

Google Vertex AI

To run Jupyter notebooks in VS Code with google-cloud-sdk:

brew install --cask google-cloud-sdk
gcloud config set project YOUR_PROJECT_ID
 
# To install or remove components at your current SDK version [475.0.0], run:
# gcloud components install COMPONENT_ID
# gcloud components remove COMPONENT_ID
 
# To update your SDK installation to the latest version [475.0.0], run:
# $ gcloud components update
 
# Log into Google Cloud
gcloud auth application-default login
 
# Verify the configuration
gcloud config list

If you receive an error (ModuleNotFoundError: No module named 'imp') when running any of the above commands, try:

brew upgrade google-cloud-sdk
 
# As of 13/05/2024, gloud components update does not support python3.12+)
# We need to use python 3.11 to run the command
 
export CLOUDSDK_PYTHON=$(which python3.11)
gcloud components update
 
# gloud does work with python3.12+,
# we can safely unset the environment variable
 
unset CLOUDSDK_PYTHON
gcloud version

If you receive PermissionDenied: 403 Your application is authenticating by using local Application Default Credentials. The aiplatform.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see [https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds](https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds) . [reason: "SERVICE_DISABLED", use the command below to set up the quota project.

gcloud auth application-default set-quota-project YOUR_PROJECT