Install CUDA, cuDNN ,TensorRT on RTX4060 NVIDIA GPU for Machine Learning AI

Image by Nana Dua

Install CUDA, cuDNN ,TensorRT on RTX4060 NVIDIA GPU for Machine Learning AI

I've new computer notebook then I want to use with Machine Learning AI.

My Computer Notebook Spec.

  • CPU: Intel Core i5-12450H (2.00GHz up to 4.40GHz, 8C(4P+4E)/12T, 12MB Intel Smart Cache)
  • RAM: 16GB (2x8GB) DDR4-3200 SO-DIMM
  • SSD 512GB NVMe PCIe 4.0
  • Graphics: NVIDIA GeForce RTX 4060 Laptop 8GB GDDR6
  • Windows 11 Home Edition




I want to use Dual-boot Windows and Ubuntu for Machine Learning AI Application.

Install CUDA, cuDNN ,TensorRT for Ubuntu

First Install Ubuntu alongside Windows

Step 1 : Create Ubuntu USB Drive Boot

We decide to use Ubuntu 22.04.3 LTS then load Ubuntu https://ubuntu.com/download/desktop


Step 2 : Boot with USB Drive then install Ubuntu


Step 3 : Select NVIDIA Driver

I've found nvidia-driver-525 and nvidia-driver-535 work fine with my notebook.

nvidia-driver-545 is not work with RTX4060.


  • Compatible NVIDIA GPU: Verify that your NVIDIA GPU is supported by the desired CUDA version you plan to install. Consult the CUDA Toolkit Compatibility Notice for guidance: https://docs.nvidia.com/deeplearning/frameworks/support-matrix/


      • check with nvidia-smi

      Step 4 : Install CUDA

      we select CUDA 12.0


      check with nvcc --version

      CUDA Install guide

      https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html


      Step 5 : Install cuDNN



      cuDNN Install guide

      https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html


      Step 6 : Install TensorRT

      Some Application use TensorRT for better performance.

      Download TensorRT

      https://developer.nvidia.com/tensorrt



      TensorRT install guide

      https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html






      Install CUDA, CUDNN ,TensorRT for Windows11

      STEP  1 : First check NVIDIA GPU Driver. 

      Normally your notebook have installed. 

      STEP 2 : Install CUDA 

       https://developer.nvidia.com/cuda-downloads

      We use CUDA 11.8 only.


      Install from EXE file. It has many CUDA program to install.


      STEP 3 : Install cuDNN

      Download file https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/


      Select cuDNN version as you want. we use cuDNN 8.9.0 CUDA 11

      Install Documents

      https://docs.nvidia.com/deeplearning/cudnn/installation/windows.html

      https://docs.nvidia.com/deeplearning/cudnn/installation/windows.html#installing-on-windows




      STEP 4 : Install TensorRT

      Install Guide
      https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html


      Now ( Feb 2024 ) TensorRT support up to CUDA 12.1 and cuDNN 8.9.0 only.

      Download TensorRT https://developer.nvidia.com/tensorrt-download


      Document Guide for Windows
      https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-zip


      pip list and import tensorrt







      Adun Nantakaew อดุลย์ นันทะแก้ว
      LINE : adunnan


      ความคิดเห็น

      แสดงความคิดเห็น

      โพสต์ยอดนิยมจากบล็อกนี้

      Llama3 Typhoon v1.5 (scb10x) LLM

      รูปภาพ
      Llama3 Typhoon v1.5 (scb10x) LLM   Typhoon-1.5 models come in 8B and 72B sizes These models are built on the 8B Llama3 and 72B Qwen base models,  8B weights released under the Meta Llama 3 Community License ; 72B weights released under the Tongyi Qianwen License Performance To gain insight into Typhoon’s performance, we evaluated it using multiple-choice exam: Language & Knowledge Capabilities: We assessed Typhoon on multiple-choice question answering datasets, including ThaiExam , M3Exam , and MMLU . The ThaiExam dataset was sourced from standard examinations in Thailand, including ONET, TGAT, TPAT, and A-Level. M3Exam is a benchmark for Southeast Asian countries, including Thailand. MMLU is a standard benchmark for language models in English. Typhoon-1.5X is an eXperimental model designed for application use cases, featuring improved capabilities in Retrieval-Augmented Generation (RAG) , constrained generation , and reasoning in order to achieve competitive performa...
      อ่านเพิ่มเติม

      Eval Llama v3.1 8B ,70B Model with Thai Language

      รูปภาพ
      Eval Llama v3.1 8B ,70B  ด้วย ข้อสอบภาษาไทย   เทียบกับ llama3-typhoon ทดสอบ การใช้งาน llama v3.1 ที่ว่า support ภาษาไทย จะทำข้อสอบ ภาษาไทยได้ดีแค่ไหน มาดูกัน ข้อสอบ • 04_investment_consult • 05_facebook_beleble_th_200 • 06_xcopa_th_200 • 07_xnli2.0_th_200 • 08_onet_m3_thai • 09_onet_m3_social • 10_onet_m3_math • 11_onet_m3_science • 12_onet_m3_english • 13_onet_m6_thai • 14_onet_m6_math • 15_onet_m6_social • 16_onet_m6_science • 17_onet_m6_english Code ทดสอบ มาจาก  OpenThaiGPT - Thai Exams Eval https://github.com/OpenThaiGPT/openthaigpt_eval การทดสอบด้วยการทำข้อสอบแบบ Multiple Choice ภาษาไทย ทดสอบบนชุดข้อสอบที่ไม่เคยเห็นมาก่อน ทดสอบแบบ Zero-shot Learning, โค้ดการทดสอบและเนื้อหาข้อสอบสามารถทำตามได้ที่นี่ ข้อมูลของ ข้อสอบ https://docs.google.com/spreadsheets/d/1ZtP5Jkx0IvCWNPQhMKitZszGnLKqvEDEf0OKdmQiXjA/edit#gid=1181424412 ผลการทดสอบ ผลการทดสอบ Average Code การทดสอบ code การ plot Graph Adun Nantakaew อดุลย์ นันทะแก้ว LINE : adunnan Facebook : https://www.facebook.co...
      อ่านเพิ่มเติม

      LLAMA-CPP-PYTHON on RTX4060 GPU

      รูปภาพ
      LLAMA-CPP-PYTHON on NVIDIA RTX4060 GPU We try to use llama-cpp-python library with many OS. Windows 11 Ubuntu 22.04 and Colab ( ubuntu 22.04 also ) Llama-cpp-python library with RTX4060 GPU on Windows11 Install NVDIA GPU Driver. we use driver version 537.24 then check nvidia-smi command for check your GPU. Install CUDA Toolkit.    https://developer.nvidia.com/cuda-downloads we use CUDA 12.1 version make sure your cuda is work fine with nvcc command. We use Python 3.11.7 check python and pip with command line. Install Pytorch  https://pytorch.org/get-started/locally/ check pip list and check cuda available on your GPU with python shell. if false it cannot use with NVIDIA GPU. find your problem.  Install llama-cpp-python  https://pypi.org/project/llama-cpp-python/ with CUDA 12.1 use this command. on Jul 2,2024 version 0.2.81 but we use only 0.2.75 it's work fine. ( 0.2.81 not test on Windows11 yet )  pip install llama-cpp-python \ --extra-index-url htt...
      อ่านเพิ่มเติม