Introduction

Getting started with large models like the Ollama 3.2 can feel overwhelming, but with the right setup, running a 1-billion parameter model on Linux is straightforward. In this guide, you'll learn how to set up your environment and run the model efficiently.

Prerequisites

Before diving into the code, make sure you have the following:

A Linux machine (Ubuntu 20.04+ recommended)
Python 3.8+
NVIDIA CUDA Toolkit (for GPU acceleration)
At least 8GB of RAM (though 16GB is recommended for better performance)

Note: Having a GPU is highly recommended for running large models like Ollama. For CPU-only usage, performance will be significantly slower.

Step 1: Installing Ollama

To install Ollama on Linux, follow these steps:

Open your terminal and ensure your system is updated:
```
sudo apt update && sudo apt upgrade
```
Install Python and dependencies (virtual environment recommended):
```
sudo apt install python3-pip python3-venv
```

Clone the Ollama repository:

git clone https://github.com/ollama/ollama
cd ollama

Create a virtual environment and activate it:

python3 -m venv ollama_env
source ollama_env/bin/activate

Install the required packages:
```
pip install -r requirements.txt
```

Step 2: Downloading the 1B Parameter Model

You can download the 1B Parameter Model from the official Ollama repository. Run the following command:

curl -O https://ollama.org/models/ollama-3.2-1b-param.tar.gz
tar -xvzf ollama-3.2-1b-param.tar.gz -C /path/to/your/models/

Ensure you have enough space on your disk for the model files, which can exceed 5GB.

Step 3: Running the Model

Once installed, you can now run the model with a simple script. Here's an example Python script to load and run the Ollama 1B model:

import ollama

# Load the model
model = ollama.load_model("/path/to/your/models/ollama-3.2-1b")

# Example input for inference
input_data = "What is the capital of France?"

# Run the model inference
output = model.run(input_data)

print("Model Output:", output)
"The real magic happens when the model starts predicting!"

Performance Tips

Running large models on Linux can be resource-intensive. Here are some performance tips:

Use a GPU: Leverage CUDA for faster computation.
Optimize Memory Usage: If you face memory constraints, try splitting your input into smaller batches.
Monitor System Resources: Use tools like htop and nvidia-smi to monitor CPU and GPU usage during model execution.

Troubleshooting

Issue	Solution
Out of Memory (OOM)	Reduce batch size or upgrade your hardware.
Model not loading	Ensure paths are correct and model is unzipped.
Slow performance	Check if CUDA is properly configured.

You did it!

With the steps outlined in this guide, you should now be able to run your first Ollama 3.2 1B parameter model on Linux. Remember to take advantage of GPU acceleration for faster results, and keep your system resources in check to avoid performance bottlenecks.

For further information, visit the official Ollama website.

back to blog posts

Running Your First Ollama 3.2 1B Parameter Model on Linux (or Apple macOS)