Tensors and operations

This is an introductory TensorFlow tutorial shows how to:

Import the required package
Create and use tensors
Use GPU acceleration

Import TensorFlow

To get started, import the tensorflow module. As of TensorFlow 2.0, eager execution is turned on by default. This enables a more interactive frontend to TensorFlow, the details of which we will discuss much later.

library(tensorflow)

Tensors

A Tensor is a multi-dimensional array. Similar to array objects in R, tf$Tensor objects have a data type and a shape. Additionally, tf$Tensors can reside in accelerator memory (like a GPU). TensorFlow offers a rich library of operations (tf$add, tf$matmul, tf$linalg$inv etc.) that consume and produce tf.Tensors. These operations automatically convert native R types, for example:

tf$add(1, 2)

## tf.Tensor(3.0, shape=(), dtype=float32)

tf$add(c(1, 2), c(3, 4))

## tf.Tensor([4. 6.], shape=(2,), dtype=float32)

tf$square(5)

## tf.Tensor(25.0, shape=(), dtype=float32)

tf$reduce_sum(c(1, 2, 3))

## tf.Tensor(6.0, shape=(), dtype=float32)

# Operator overloading is also supported
tf$square(2) + tf$square(3)

## tf.Tensor(13.0, shape=(), dtype=float32)

Each tf$Tensor has a shape and a datatype:

x = tf$matmul(matrix(1,ncol = 1), matrix(c(2, 3), nrow = 1))
x

## tf.Tensor([[2. 3.]], shape=(1, 2), dtype=float64)

x$shape

## (1, 2)

x$dtype

## <dtype: 'float64'>

The most obvious differences between arrays and tf$Tensors are:

Tensors can be backed by accelerator memory (like GPU, TPU).
Tensors are immutable.

R arrays compatibility

Converting between a TensorFlow tf.Tensors and an array is easy:

TensorFlow operations automatically convert R arrays to Tensors.

Tensors are explicitly converted to R arrays using the as.array, as.matrix or as.numeric methods. There’s always a memory copy when converting from a Tensor to an array in R.

# TensorFlow operations convert arrays to Tensors automatically
1 + tf$ones(shape = 1)

## tf.Tensor([2.], shape=(1,), dtype=float32)

# The as.array method explicitly converts a Tensor to an array
as.array(tf$ones(shape = 1))

## [1] 1

GPU acceleration

Many TensorFlow operations are accelerated using the GPU for computation. Without any annotations, TensorFlow automatically decides whether to use the GPU or CPU for an operation—copying the tensor between CPU and GPU memory, if necessary. Tensors produced by an operation are typically backed by the memory of the device on which the operation executed, for example:

x <- tf$random$uniform(shape(3, 3))

# List devices
tf$config$experimental$list_physical_devices()

## [[1]]
## PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')
## 
## [[2]]
## PhysicalDevice(name='/physical_device:XLA_CPU:0', device_type='XLA_CPU')

# What device is x placed
x$device

## [1] "/job:localhost/replica:0/task:0/device:CPU:0"

Device Names

The Tensor$device property provides a fully qualified string name of the device hosting the contents of the tensor. This name encodes many details, such as an identifier of the network address of the host on which this program is executing and the device within that host. This is required for distributed execution of a TensorFlow program. The string ends with GPU:<N> if the tensor is placed on the N-th GPU on the host.

Explicit Device Placement

In TensorFlow, placement refers to how individual operations are assigned (placed on) a device for execution. As mentioned, when there is no explicit guidance provided, TensorFlow automatically decides which device to execute an operation and copies tensors to that device, if needed. However, TensorFlow operations can be explicitly placed on specific devices using the tf$device context manager, for example:

print("On CPU:0:")
with(tf$device("CPU:0"), {
  x <- tf$ones(shape(1000, 1000))
  print(x$device)
})

print("On GPU:0:")
with(tf$device("GPU:0"), {
  x <- tf$ones(shape(1000, 1000))
  print(x$device)
})