https://github.com/intel/ipex-llm

rafavi

[url=https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/install_windows_gpu.md]https://github.com/intel/ipex-llm[/url]
Install ipex-llm

With the llm environment active, use pip to install ipex-llm for GPU:

Choose either US or CN website for extra-index-url:

• For US:
  conda create -n llm python=3.11 libuv
• conda activate llm
• pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
  
  
• For CN:
  conda create -n llm python=3.11 libuv
• conda activate llm
• pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/cn/
  
  
  
  

Note

If you encounter network issues while installing IPEX, refer to this guide for troubleshooting advice.

Verify Installation

You can verify if ipex-llm is successfully installed following below steps.

Step 1: Runtime Configurations

• Open the Miniforge Prompt and activate the Python environment llm you previously created:
  conda activate llm
  
  
• Set the following environment variables according to your device:
  
  
  • For Intel iGPU and Intel Arc™ A770:
    set SYCL_CACHE_PERSISTENT=1
    
    
    
    
  
  

Tip

For other Intel dGPU Series, please refer to this guide for more details regarding runtime configuration.

Step 2: Run Python Code

• Launch the Python interactive shell by typing python in the Miniforge Prompt window and then press Enter.
• Copy following code to Miniforge Prompt line by line and press Enter after copying each line.
  import torch from ipex_llm.transformers import AutoModel,AutoModelForCausalLM    tensor_1 = torch.randn(1, 1, 40, 128).to('xpu') tensor_2 = torch.randn(1, 1, 128, 40).to('xpu') print(torch.matmul(tensor_1, tensor_2).size())
  
  
  It will output following content at the end:
  torch.Size([1, 1, 40, 40])
  
  

  Tip:

  If you encounter any problem, please refer to here for help.

• To exit the Python interactive shell, simply press Ctrl+Z then press Enter (or input exit() then press Enter).