NVIDIA Jetson Orin Nano Complete Setup Guide & Running LLM
There is a confusion what is difference between Jetson Orin Nano vs Jetson Orin nano Super. hardware prospective both are same. A software upgrade JetPack 6.2 is required to upgrade Jetson Orin Nano to Jetson Orin nano Super which increase 70% more performance. Which we are going to perform in this video.
Sites
1. https://developer.nvidia.com/embedded/learn/get-started-jetson-orin-nano-devkit#intro
2. https://www.jetson-ai-lab.com/initial_setup_jon.html#__tabbed_1_1
Below image shows that this Jetson Orin Nano Developer Kit unit has the old UEFI firmware
Escape key for setup and ALT+ Delete key any time if you want come out
UEFI New firmware or firmware 36.x. Direct upgrade to Super
Preparing Micro SSD for Jetson Orin Nano
1 Check the version of the firmware. If it is 36.x you can directly upgrade to JetPack 6.2.
2. Download JetPack 6.2 from NVIDIA site and extract
3. Download and run Etcher.
4. Write the extracted image to Micro SSD Card
5. Insert the SSD card to Jetson Orin Nano and install the image
Note - After installation the image you may face issue the browser is not starting. To fix this, you can downgrade and hold the snapd package to a previous version using terminal commands, which should resolve the issue for browsers like Chromium and Firefox.
1. Open the terminal: on your Jetson device.
2. Download the previous version of snapd:
snap download snapd --revision=24724
3. Acknowledge the new snapd version:
sudo snap ack snapd_24724.assert
4. Install the previous snapd version:
sudo snap install snapd_24724.snap
5. Hold the snapd service to prevent automatic updates:
sudo snap refresh --hold snapd
Now the browser should start to work.
How to RDP to Jetson Orin Nano.
1. Open the terminal: on your Jetson device and run below commands
$ wget https://www.nomachine.com/free/arm/v8/deb -O nomachine.deb
$ sudo dpkg -i nomachine.deb
2. On your Windows/ Mac/ Linux PC, Go to https://www.nomachine.com/ and select the appropriate version and download and install.
3. Follow up the installation wizard.
4. Double click the application follow the on screen instruction, it will search the Jetson Orin Nano. You can now remote to Jetson Orin Nano.
Running your LLM locally
Pages
https://www.jetson-ai-lab.com/tutorial_ollama.html
https://github.com/dusty-nv/jetson-containers
1. Go to https://github.com/dusty-nv/jetson-containers docs - Setup.md
2. Clone The repo – Copy below command
git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh
3. Open the terminal in jetson nano and past the above copied command.
4. Go to https://www.jetson-ai-lab.com/tutorial_ollama.html Copy and Run the command
jetson-containers run --name ollama $(autotag ollama)
5. Run the below command to run the LLM
ollama run llama3.2:3b
6. Now the AI is installed locally and ready to use in terminal window.
To run LLM in browser-based interface like ChatGPT Open WebUI to be installed. In the NVIDIA Jetson AI Lab - Select Open WebUI
1. Run below command to pull the docker image of Open WebUI
sudo docker run -d --network=host \
-v ${HOME}/open-webui:/app/backend/data \
-e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:main
2. Open a Web browser on a PC connected to the same network as your Jetson and access the following address. http://IP of jetson nano:8080 through web browser on locally.
