These are my personal notes on how to run Ollama and OpenCode on an EC2 instance and connect remotely with Visual Studio Code for code development.
Motivation
As part of the AWS Community Builders Program, I receive lots of free AWS credits, so this is an experiment for me to test open source models for coding.
Advantages
You can gain access to powerful hardware without cost of ownership.
You only pay for what you use. You stop the instance when you are not using it.
You can easily upgrade and downgrade your instance.
You have full control over the instance, so you can install anything you wish.
You run the agents in an isolated environment and not on your personal machine. The agent cannot access your personal files or your credentials.
It is a throwaway instance, if the agent messes up your installation you just delete it and provision a new one.
Your conversations are secure and you do not have to worry about the Terms and Conditions of AI model providers.
You can test different hardware before you commit to built/buy your own machine
Disadvantages
- The resources of your instance will most of the time be underutilised.
Instance provisioning
When setting up my instance, I prefer to get a cheap instance type. Once it is all set up, then I can scale it up to one with more resources. I initially provision an instance with the following settings:
Name: devbox
AMI: Deep Learning Base AMI with Single CUDA (Ubuntu 24.04)
Architecture: 64-bit (x86)
Instance type: g4dn.xlarge
Key pair name: select an existing one or create one
Security group: create or select an existing one
EBS Volume: 160 GB
EBS Volume Encryption: enabled
Advanced Details:
IAM Instance profile: select or create a new role
Some comments on the above choices:
We are using a Deep Learning Base AMI that has the NVIDIA drivers preinstalled.
You will use the key to SSH to the instance.
The security group should only allow full access to your own IP address.
Choose your own volume size based on how big are the models that you are going to download and what other files you will need to store on the instance (codebases for example).
The role should have the default permissions and no extra permissions. In the future, if your agent needs to access the AWS API you can add permissions using the least privilege principal.
Connecting to your instance
After launching the instance, wait for it to become available. In the meantime, grab its public IP address and set up your SSH config ~/.ssh/config by adding an entry like this one
Host devbox
HostName <public IP of EC2 instance>
User ubuntu
IdentityFile <full path to your .pem file>
This way we can simply do ssh devbox and connect to the instance. If you stop and start the instance again you will need to update this entry with the new IP address. If you cannot connect, check your security group rules.
Installation
Ollama
Install Ollama with
curl -fsSL https://ollama.com/install.sh | sh
and check that it is running with
systemctl status ollama
Pull a model
ollama pull gemma4:12b
Start a quick conversation with
ollama run gemma4:12b
and then ask your question. You can exit with /bye.
>>> How much VRAM is available on a g4dn.xlarge EC2 instance?
...
>>> /bye
btop
Install btop with GPU support
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
sudo apt update
sudo apt install g++-14 lowdown -y
Run the script found here with to install btop with GPU support
sudo su -c "bash <(wget -qO- https://gist.githubusercontent.com/talmo/b592fb5372630ea823935ec7a047ab26/raw/78593355307ddf00ca08899d47f20733082b35f0/install_btop_gpu.sh)" root
OpenCode
Install OpenCode with
curl -fsSL https://opencode.ai/install | bash
Download a codebase
Let's download some code on our instance so that we can test our agent.
mkdir ~/Repos/
cd ~/Repos/
git clone https://github.com/anomalyco/opencode.git
Connect with VS Code
You can easily connect to your remote instance with VS code running locally on your machine. For this you will need the extension developed by Microsoft named Remote - SSH
Once the extension is downloaded, open the commands menu (command + shift + P on my maC) and run Remote -SSH: Connect to Host... . The setting from your ~/.ssh/config should be visible and you can simply select devbox (or any other name you used). Once connected, you can open the folder of the opencode project that we cloned earlier, i.e. on the path /home/ubuntu/Repos/opencode/ (or any other project you have cloned).
Finally, from the menu of VS Code click Terminal > New Terminal and congratulations, you have your IDE ready for development.
(Optional) Port forwarding
You can do port forwarding from the terminal, or more easily you can port-forward port 11434 from within VS Code.
This would be useful if you want to connect a harness running locally to models served on the EC2 instance.
If you did do the above, you can navigate to http://localhost:11434 and you should see the following message in your browser
Ollama is running
Option 2 (preferred): Configure context length at model level
mkdir ~/OllamaCustom
Create file ~/OllamaCustom/gemma4_12b with the definition of our custom model so that we can increase the context
cat <<EOF > ~/OllamaCustom/gemma4_12b
FROM gemma4:12b
PARAMETER num_ctx 262144
EOF
and create the custom model
ollama create custom-gemma4-12b -f ~/OllamaCustom/gemma4_12b
Then run ollama ls and confirm the custom model is listed.
Option 2: Configure global Ollama Context Length
Stop the Ollama service and edit the configuration
sudo systemctl stop ollama
sudo systemctl edit ollama
Add the following to the configuration
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=262144"
and start the service again
sudo systemctl start ollama
Create the directory
mkdir -p ~/.config/opencode/
and the file that holds the OpenCode configuration (~/.config/opencode/opencode.json)
cat <<EOF > ~/.config/opencode/opencode.json
{
"\$schema": "https://opencode.ai/config.json",
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (Local)",
"options": {
"baseURL": "http://localhost:11434/v1"
},
"models": {
"custom-gemma4-12b": {
"name": "Custom Gemma 4 12B",
"limit": {
"context": 262144,
"output": 8192
}
}
}
}
}
}
EOF
Run OpenCode
Start OpenCode in the terminal of your VS Code that is connected to the EC2 Instance.
opencode
Type/models and then you can select the local model
Custom Gemma 4 12B - Ollama (Local)
Having the OpenCode repo open, you can ask the following and watch the agent do its magic.
Review the variants of the README in each language and fix any inconsistencies in the html code. Do not fix text translated in the specific language, this is expected.
Debugging
You can view the logs of the running Ollama service with
journalctl -u ollama -f
Scale up the instance
Shut down the instance with
sudo shutdown now
From the AWS Console, change the instance to p5.2xlarge and start the instance again.
Important: remember to update the IP address of the instance in your ~/.ssh/config
Use a bigger model
SSH to the instance again and create a new file for the new custom model under ~/OllamaCustom/qwen3-coder-30b
cat <<EOF > ~/OllamaCustom/qwen3-coder-30b
FROM qwen3-coder:30b
PARAMETER num_ctx 262144
EOF
then create the Ollama custom model with
ollama create custom-qwen3-coder-30b -f ~/OllamaCustom/qwen3-coder-30b
and update the ~/.config/opencode/opencode.json by adding the model
"custom-qwen3-coder-30b": {
"name": "Custom Qwen3 Coder 30B",
"limit": {
"context": 262144,
"output": 65536
}
}
Start opencode again and select the new model with /models
Check with ollama ps while running a model. We can see that for g5.2xlarge Gemma 4 12B can run entirely on the GPU
NAME ID SIZE PROCESSOR CONTEXT UNTIL
custom-gemma4-12b:latest f361b9dbdeb7 8.3 GB 100% GPU 262144 4 minutes from now
while Qwen 3 Coder 30B runs partly on GPU and partly on CPU
NAME ID SIZE PROCESSOR CONTEXT UNTIL
custom-qwen3-coder-30b:latest e1b819cdf614 45 GB 52%/48% CPU/GPU 262144 4 minutes from now
Scaling options
If you need more VRAM for larger models or longer context windows, moving to a higher tier instance is easy. For example, while g4dn is a great starting point, you can use g5/g6/g6e instances that offer even more GPU memory.
| Instance Type |
System RAM |
GPU Memory (VRAM) |
| g4dn.xlarge |
16 GB |
16 GB |
| g4dn.2xlarge |
32 GB |
16 GB |
| g5.xlarge |
16 GB |
24 GB |
| g5.2xlarge |
32 GB |
24 GB |
| g6.xlarge |
16 GB |
24 GB |
| g6.2xlarge |
32 GB |
24 GB |
| g6e.xlarge |
32 GB |
48 GB |
| g6e.2xlarge |
64 GB |
48 GB |
Note: While VRAM is consistent across a family, larger instance sizes (e.g., moving from xlarge to 2xlarge) provide more system RAM. This extra memory is crucial for supporting larger context windows (Key-Value caches), providing headroom for the OS, and allowing model offloading if weights exceed available VRAM.
Conclusion
This setup provides a powerful, isolated environment for experimenting with local models and agentic coding tools like OpenCode. By offloading the heavy lifting to an EC2 instance, you can experiment freely without compromising your local machine's resources or data privacy.