Tracing

Tracing records operations, scripting parses Python

Tracing in TorchScript involves recording the operations performed by a PyTorch model during execution. This process creates a serialized representation of the model that can be run independently of the original Python code. Scripting, on the other hand, involves translating the PyTorch model into a TorchScript format that can be executed in a different environment, such as C++.

Example

Suppose you have a PyTorch model for image classification. By tracing this model with a sample input, TorchScript records the operations performed. Later, you can convert the traced model into TorchScript code, which can then be deployed in a C++ application for inference.

Remember this

Understanding the difference between tracing and scripting is crucial for optimizing model deployment and ensuring compatibility across different programming environments. Tracing captures the model's behavior, while scripting transforms it for execution in other languages.

Text adapted from Wikipedia, licensed under CC BY-SA 4.0.

Related concepts

torch.compile does in PyTorch 2.0: traces and optimizes the computation graph

torch.compile optimizes computation graph by tracing and compiling it for efficiency

loop unrolling does: trades code size for reduced loop overhead

Loop unrolling optimizes execution speed by reducing loop control instructions

Lambda calculus

Lambda calculus represents data using only functions

Encoder vs decoder: encoder sees all tokens bidirectionally, decoder sees only past tokens

Encoder: Sees all tokens bidirectionally; Decoder: Sees only past tokens

log-probabilities are used instead of probabilities: avoids numerical underflow

Why can't we just add up tiny chances over time?

consistent hashing solves: minimizes key redistribution when servers are added/removed

Consistent hashing minimizes key redistribution when servers are added/removed

Swipe through 100 ML concepts daily

Open Pocket Polymath