Open links in new tab
  1. Github
    https://github.com/pytorch/xla

    pytorch/xla: Enabling PyTorch on XLA Devices (e.g. Google TPU)

    • Current CI status:
      Note: PyTorch/XLA r2.1 will be the last release with XRT available as a legacy runtime. Our main release build will not include X… See more

    Getting Started

    PyTorch/XLA is now on PyPI!
    To install PyTorch/XLA a new TPU VM:
    To … See more

    Github
    Available docker images and wheels

    Python packages
    PyTorch/XLA releases starting with version r2.1 will be available on PyPI. You can now install the main build with pip install torch_xla. … See more

    Github
    Providing Feedback

    The PyTorch/XLA team is always happy to hear from users and OSS contributors! The best way to reach out is by filing an issue on this Github. Questions, bug reports, feature req… See more

    Github
    Disclaimer

    This repository is jointly operated and maintained by Google, Facebook and a number of individual contributors listed in the CONTRIBUTORS file. For questions directed at … See more

    Github
    PyTorch on XLA Devices
    PyTorch
    PyTorch on XLA Devices — PyTorch/XLA master documentati…
    PyTorch
    Feedback
    Table of Contents
    Getting Started
    Available docker images and wheels
    Providing Feedback
    Disclaimer
     
  1. PyTorch
    https://pytorch.org/xla/master/accelerators/tpu.html

    Learn about TPUs — PyTorch/XLA master documentation

  2. Google Cloud
    https://cloud.google.com/blog/products/a…

    Introducing PyTorch/XLA 2.3 | Google Cloud Blog

    Apr 26, 2024 · PyTorch/XLA offers the best of both worlds: the user experience and ecosystem advantages of PyTorch, with the compiler performance of …

    • Estimated Reading Time: 3 mins
    • Github
      https://github.com/openxla/xla

      GitHub - openxla/xla: A machine learning compiler for GPUs, …

    • Hugging Face
      https://huggingface.co/blog/pytorch-xla

      Hugging Face on PyTorch / XLA TPUs

      Feb 9, 2021 · Importing torch_xla initializes PyTorch / XLA, and xm.xla_device() returns the current XLA device. This may be a CPU, GPU, or TPU depending on your environment, but for this blog post we’ll focus primarily on TPU.

    • PyTorch
      https://pytorch.org/xla/release/2.1/index.html

      PyTorch on XLA Devices

    • People also ask
      What is PyTorch XLA?
      PyTorch/XLA is a Python package that uses the XLA deep learning compiler to connect the PyTorch deep learning framework and Cloud TPUs. You can try it right now, for free, on a single Cloud TPU VM with Kaggle! Take a look at one of our Kaggle notebooks to get started: To install PyTorch/XLA stable build in a new TPU VM:

      pytorch/xla: Enabling PyTorch on XLA Devices (e.g. Google

      github.com
      Does XLA/TPU recompile a graph?
      XLA/TPU yield degraded performance with too many recompilations. XLA compilation is expensive. PyTorch/XLA automatically recompiles the graph every time new shapes are encountered. Usually models should stabilize within a few steps and you can see huge speedup for the rest of training.

      PyTorch on XLA Devices — PyTorch/XLA master documentation

      pytorch.org
      How to train XLA TPU in parallel?
      In a typical XLA:TPU training scenario we’re training on multiple TPU cores in parallel (a single Cloud TPU device includes 8 TPU cores). So we need to ensure that all the gradients are exchanged between the data parallel replicas by consolidating the gradients and taking an optimizer step.

      Hugging Face on PyTorch / XLA TPUs

      huggingface.co
      Does PyTorch / XLA run on Cloud TPUs?
      PyTorch / XLA will run most standard PyTorch programs with minimal modifications, falling back to CPU to execute operations that are not yet supported on TPUs. With the help of a detailed reportthat PyTorch / XLA generates, PyTorch users can find bottlenecks and adapt their programs to run more efficiently on Cloud TPUs.

      PyTorch is now GA on Google Cloud TPUs | Google Cloud Blog

      cloud.google.com
      What is XLA compiler?
      The XLA compiler takes models from popular ML frameworks such as PyTorch, TensorFlow, and JAX, and optimizes them for high-performance execution across different hardware platforms including GPUs, CPUs, and ML accelerators. If you want to use XLA to compile your ML project, refer to the corresponding documentation for your ML framework:

      openxla/xla: A machine learning compiler for GPUs, CPUs, and ML accel…

      github.com
      What is XLA device?
      device (string or torch.device) – The xla device that will be mapped to the real device. A string representation of the hardware type (CPU, TPU, GPU) of the given device. Retrieves the replication ordinal of the current process. The ordinals range from 0 to xrt_world_size () minus 1.

      PyTorch on XLA Devices — PyTorch/XLA master documentation

      pytorch.org
      Feedback
    • Google Cloud
      https://cloud.google.com/blog/products/ai-machine...

      What’s new with PyTorch/XLA 2.5 | Google Cloud Blog

    • Google Cloud
      https://cloud.google.com/blog/products/ai-machine...

      PyTorch / XLA now generally available on Cloud TPUs - Google …

    • Google Colab
      https://colab.research.google.com/github/kozodoi/...

      2020-10-30-pytorch-xla-tpu.ipynb - Colab - Google Colab

    • PyTorch
      https://pytorch.org/xla/release/1.9/index.html

      PyTorch on XLA Devices — PyTorch/XLA master documentation

    • Some results have been removed