# Run Ollama and OpenCode on an AWS EC2 instance

These are my personal notes on how to run Ollama and OpenCode on an EC2 instance and connect remotely with Visual Studio Code for code development.

## Motivation

As part of the [AWS Community Builders Program](https://builder.aws.com/community/community-builders), I receive lots of free AWS credits, so this is an experiment for me to test open source models for coding.

### Advantages

*   You can gain access to powerful hardware without cost of ownership.
    
*   You only pay for what you use. You stop the instance when you are not using it.
    
*   You can easily upgrade and downgrade your instance.
    
*   You have full control over the instance, so you can install anything you wish.
    
*   You run the agents in an isolated environment and not on your personal machine. The agent cannot access your personal files or your credentials.
    
*   It is a throwaway instance, if the agent messes up your installation you just delete it and provision a new one.
    
*   Your conversations are secure and you do not have to worry about the Terms and Conditions of AI model providers.
    
*   You can test different hardware before you commit to built/buy your own machine
    

### Disadvantages

*   The resources of your instance will most of the time be underutilised.
    

## Instance provisioning

When setting up my instance, I prefer to get a cheap instance type. Once it is all set up, then I can scale it up to one with more resources. I initially provision an instance with the following settings:

```plaintext
Name: devbox
AMI: Deep Learning Base AMI with Single CUDA (Ubuntu 24.04)
Architecture: 64-bit (x86)
Instance type: g4dn.xlarge
Key pair name: select an existing one or create one
Security group: create or select an existing one
EBS Volume: 160 GB 
EBS Volume Encryption: enabled

Advanced Details:
  IAM Instance profile: select or create a new role
```

Some comments on the above choices:

*   We are using a Deep Learning Base AMI that has the NVIDIA drivers preinstalled.
    
*   You will use the key to SSH to the instance.
    
*   The security group should only allow full access to your own IP address.
    
*   Choose your own volume size based on how big are the models that you are going to download and what other files you will need to store on the instance (codebases for example).
    
*   The role should have the default permissions and no extra permissions. In the future, if your agent needs to access the AWS API you can add permissions using the least privilege principal.
    

## Connecting to your instance

After launching the instance, wait for it to become available. In the meantime, grab its public IP address and set up your SSH config `~/.ssh/config` by adding an entry like this one

```plaintext
Host devbox
  HostName <public IP of EC2 instance>
  User ubuntu
  IdentityFile <full path to your .pem file>
```

This way we can simply do `ssh devbox` and connect to the instance. If you stop and start the instance again you will need to update this entry with the new IP address. If you cannot connect, check your security group rules.

## Installation

### Ollama

Install [Ollama](https://ollama.com/) with

```plaintext
curl -fsSL https://ollama.com/install.sh | sh
```

and check that it is running with

```plaintext
systemctl status ollama
```

Pull a model

```plaintext
ollama pull gemma4:12b
```

Start a quick conversation with

```plaintext
ollama run gemma4:12b
```

and then ask your question. You can exit with `/bye`.

```plaintext
>>> How much VRAM is available on a g4dn.xlarge EC2 instance?
...

>>> /bye
```

### btop

Install `btop` with GPU support

```plaintext
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
sudo apt update
sudo apt install g++-14 lowdown -y
```

Run the script found [here](https://gist.github.com/talmo/b592fb5372630ea823935ec7a047ab26) with to install btop with GPU support

```plaintext
sudo su -c "bash <(wget -qO- https://gist.githubusercontent.com/talmo/b592fb5372630ea823935ec7a047ab26/raw/78593355307ddf00ca08899d47f20733082b35f0/install_btop_gpu.sh)" root
```

### OpenCode

Install [OpenCode](https://opencode.ai/) with

```plaintext
curl -fsSL https://opencode.ai/install | bash
```

## Download a codebase

Let's download some code on our instance so that we can test our agent.

```plaintext
mkdir ~/Repos/
cd ~/Repos/
git clone https://github.com/anomalyco/opencode.git
```

## Connect with VS Code

You can easily connect to your remote instance with VS code running locally on your machine. For this you will need the extension developed by Microsoft named `Remote - SSH`

Once the extension is downloaded, open the commands menu (command + shift + P on my maC) and run `Remote -SSH: Connect to Host...` . The setting from your `~/.ssh/config` should be visible and you can simply select `devbox` (or any other name you used). Once connected, you can open the folder of the `opencode` project that we cloned earlier, i.e. on the path `/home/ubuntu/Repos/opencode/` (or any other project you have cloned).

Finally, from the menu of VS Code click `Terminal > New Terminal` and congratulations, you have your IDE ready for development.

### (Optional) Port forwarding

You can do port forwarding from the terminal, or more easily you can port-forward port 11434 from within VS Code.

![](https://cdn.hashnode.com/uploads/covers/5fa5600a0163314ab6d7de2d/4f1c427a-677f-42f2-8308-928515b44728.png align="center")

This would be useful if you want to connect a harness running locally to models served on the EC2 instance.

If you did do the above, you can navigate to [http://localhost:11434](http://localhost:11434) and you should see the following message in your browser

```plaintext
Ollama is running
```

## Option 2 (preferred): Configure context length at model level

```plaintext
mkdir ~/OllamaCustom
```

Create file `~/OllamaCustom/gemma4_12b` with the definition of our custom model so that we can increase the context

```plaintext
cat <<EOF > ~/OllamaCustom/gemma4_12b
FROM gemma4:12b
PARAMETER num_ctx 262144
EOF
```

and create the custom model

```plaintext
ollama create custom-gemma4-12b -f ~/OllamaCustom/gemma4_12b
```

Then run `ollama ls` and confirm the custom model is listed.

## Option 2: Configure global Ollama Context Length

Stop the Ollama service and edit the configuration

```plaintext
sudo systemctl stop ollama
sudo systemctl edit ollama
```

Add the following to the configuration

```plaintext
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=262144"
```

and start the service again

```plaintext
sudo systemctl start ollama
```

## Configure OpenCode

Create the directory

```plaintext
mkdir -p ~/.config/opencode/
```

and the file that holds the OpenCode configuration (`~/.config/opencode/opencode.json`)

```plaintext
cat <<EOF > ~/.config/opencode/opencode.json
{
  "\$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (Local)",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "custom-gemma4-12b": {
          "name": "Custom Gemma 4 12B",
          "limit": {
            "context": 262144,
            "output": 8192
          }
        }
      }
    }
  }
}
EOF
```

## Run OpenCode

Start OpenCode in the terminal of your VS Code that is connected to the EC2 Instance.

```plaintext
opencode
```

Type`/models` and then you can select the local model

```plaintext
Custom Gemma 4 12B - Ollama (Local)
```

Having the OpenCode repo open, you can ask the following and watch the agent do its magic.

```plaintext
Review the variants of the README in each language and fix any inconsistencies in the html code. Do not fix text translated in the specific language, this is expected.
```

## Debugging

You can view the logs of the running Ollama service with

```plaintext
journalctl -u ollama -f
```

## Scale up the instance

Shut down the instance with

```plaintext
sudo shutdown now
```

From the AWS Console, change the instance to `p5.2xlarge` and start the instance again.

Important: remember to update the IP address of the instance in your `~/.ssh/config`

## Use a bigger model

SSH to the instance again and create a new file for the new custom model under `~/OllamaCustom/qwen3-coder-30b`

```plaintext
cat <<EOF > ~/OllamaCustom/qwen3-coder-30b
FROM qwen3-coder:30b
PARAMETER num_ctx 262144
EOF
```

then create the Ollama custom model with

```plaintext
ollama create custom-qwen3-coder-30b -f ~/OllamaCustom/qwen3-coder-30b
```

and update the `~/.config/opencode/opencode.json` by adding the model

```plaintext
        "custom-qwen3-coder-30b": {
          "name": "Custom Qwen3 Coder 30B",
          "limit": {
            "context": 262144,
            "output": 65536
          }
        }
```

Start `opencode` again and select the new model with `/models`

## Checking performance

Check with `ollama ps` while running a model. We can see that for `g5.2xlarge` Gemma 4 12B can run entirely on the GPU

```plaintext
NAME                        ID              SIZE      PROCESSOR    CONTEXT    UNTIL
custom-gemma4-12b:latest    f361b9dbdeb7    8.3 GB    100% GPU     262144     4 minutes from now
```

while Qwen 3 Coder 30B runs partly on GPU and partly on CPU

```plaintext
NAME                             ID              SIZE     PROCESSOR          CONTEXT    UNTIL
custom-qwen3-coder-30b:latest    e1b819cdf614    45 GB    52%/48% CPU/GPU    262144     4 minutes from now
```

## Scaling options

If you need more VRAM for larger models or longer context windows, moving to a higher tier instance is easy. For example, while `g4dn` is a great starting point, you can use `g5`/`g6`/`g6e` instances that offer even more GPU memory.

| Instance Type | System RAM | GPU Memory (VRAM) |
| --- | --- | --- |
| g4dn.xlarge | 16 GB | 16 GB |
| g4dn.2xlarge | 32 GB | 16 GB |
| g5.xlarge | 16 GB | 24 GB |
| g5.2xlarge | 32 GB | 24 GB |
| g6.xlarge | 16 GB | 24 GB |
| g6.2xlarge | 32 GB | 24 GB |
| g6e.xlarge | 32 GB | 48 GB |
| g6e.2xlarge | 64 GB | 48 GB |

Note: While VRAM is consistent across a family, larger instance sizes (e.g., moving from xlarge to 2xlarge) provide more system RAM. This extra memory is crucial for supporting larger context windows (Key-Value caches), providing headroom for the OS, and allowing model offloading if weights exceed available VRAM.

## Conclusion

This setup provides a powerful, isolated environment for experimenting with local models and agentic coding tools like OpenCode. By offloading the heavy lifting to an EC2 instance, you can experiment freely without compromising your local machine's resources or data privacy.