What to study for each PTCA domain
PyTorch Fundamentals
Covers PyTorch core concepts and tensors, including creation, shapes, dtypes, reshaping, broadcasting, and in-place operations, plus autograd with requires_grad, backward, and .grad. It also addresses training, testing, and using models with the training loop, eval and train modes, no_grad, saving and loading state dicts, and device basics across CPU, CUDA, and MPS.
Key skills
tensors and shapes, autograd and backward, training loop, eval and no_grad, state dicts, CPU CUDA and MPS devices
Model Development
Covers PyTorch neural network building blocks through nn.Module, including __init__ and forward, and layers such as Linear, Conv2d, Embedding, pooling, and recurrent modules. It also addresses activations, dropout, and batch normalization, parameter inspection and freezing, Sequential containers, and classifier heads with logits and losses.
Key skills
nn.Module, Linear and Conv2d, Embedding, activations, dropout, batch normalization, Sequential, transfer learning freezing
Performance & Optimization
Covers precision and execution optimization including mixed precision with autocast and GradScaler, torch.compile, channels-last, TF32, pruning, and gradient checkpointing. It also addresses performance measurement with profiling, throughput benchmarking, and CUDA synchronization, and distributed training with DistributedDataParallel, DistributedSampler, and FSDP.
Key skills
mixed precision and GradScaler, torch.compile, profiling, pinned memory, DDP and FSDP, gradient checkpointing and accumulation
Data Handling
Covers datasets and the Dataset protocol with __len__ and __getitem__, DataLoaders for batching, shuffling, workers, and collation, and transforms including ToTensor, Normalize, and training-only augmentation. It also addresses training data practices such as train validation and test splits, padding variable-length sequences, and handling imbalanced data with weighted sampling.
Key skills
Dataset protocol, DataLoader batching and workers, transforms and Normalize, augmentation, collate and padding, weighted sampling