How to Execute LLMs Model Remotely at No Cost Using Google Colab T4 GPU
Harness ollama + google colab 🦙
5 min readFeb 8, 2024
Have you ever worried about not having enough GPU running LLMs?
Have you ever considered Private LLMs?
As a seasoned data scientist, I’m a big fan of Google Colab (Oops, sorry Microsoft), and it requires no setup to use, while providing access free of charge to computing resources including Nvidia’s T4 GPUs.
You can see what GPU you’ve been assigned at any time by executing the following cell.
# Use the IPython magic command to execute the nvidia-smi command and capture the output
gpu_info = !nvidia-smi
# Join the list of strings into a single string, separated by newline characters
gpu_info = '\n'.join(gpu_info)
# Check if the output contains the word 'failed,' indicating a failure in connecting to a GPU
if gpu_info.find('failed') >= 0:
print('Not connected to a GPU')
else:
# Print the GPU information
print(gpu_info)