Heuristics AI

TIP/ADVICE : Best for all inference engineering beginners. Keep asking this question throughout "Will this enable my inference to become more efficient/cheap/secure?" This field is at an intersection of HPC, Hardware and Low level systems. Hence it is very very easy to drift into a lot of rabbit holes that may be irrelevant especially if you are on a timeline. Otherwise if you find something exciting (HPC given how broad it is, has abundant flashy niches) by all means explore, you never know what you will find yourself extremely passionate about. Above tip is only for people who have decided on Inference Engineering and have to touch various topics from a lot of adjacent fields without spreading too thin or loosing focus, that simple question saves you. Hope it helps 🙂

https://docs.google.com/spreadsheets/d/164yL7peCLfcXkT-oAjaikSKdD7ptiCdS9nsMyNwOtzs/edit?gid=0#gid=0

ADVICE well. strict rigid roadmaps never work. But the general path looks like this : 1) Understanding Parallel Programming 2) CUDA/Triton (GPU kernel writing) - PMPP book. Write non-trivial kernels while scoring them 3) Read and understand Flash Attention, vLLM, PagedAttention, continous batching, scheduler, kv-cache and Quantisation. Try to implement something from scratch. Dive into research papers for deeper understanding. A lot can be done here. 4) Merge PRs into Open Source projects / Build your own inference engine At any point if you get your first job in this field or adjacent to it, take it. Then it's just going for depth and/or very specific rabbit holes @heuristics_ai

https://github.com/thomasjpfan/profiling-pytorch-modal

https://arpanpathak.github.io/gpu-parallel-book/

https://handbook.modular.com LLM Inference Handbook

https://developer.nvidia.com/blog/ai-model-co-design-hardware-friendly-llm-design/

RL READING LIST for LLMs https://x.com/Grad62304977/status/1967548295816819184?s=20

https://www.linkedin.com/posts/pritam-ai_machinelearning-ai-careeradvice-activity-7469028707634937856-XmVl?utm_source=social_share_send&utm_medium=android_app&rcm=ACoAADO3fjQBXyXMHpZ-0dAyklxmiO-XwV6FJLQ&utm_campaign=copy_link

https://www.linkedin.com/posts/pritam-ai_machinelearning-ai-careeradvice-activity-7469028707634937856-XmVl?utm_source=social_share_send&utm_medium=android_app&rcm=ACoAADO3fjQBXyXMHpZ-0dAyklxmiO-XwV6FJLQ&utm_campaign=copy_link

https://buff.ly/9aVIalL

https://buff.ly/jJ9O0RL

Heuristics AI pinned «Greg Brockman May 3, 2016 My path to OpenAI https://blog.gregbrockman.com/my-path-to-openai»

Greg Brockman May 3, 2016 My path to OpenAI https://blog.gregbrockman.com/my-path-to-openai

Everything you need to know about Pytorch Distributed Data Parallel(DDP) https://jino-rohit.github.io/blogs/10_ddp.html

https://www.makingsoftware.com/ best site on software engineer

Transformers from Scratch https://brandonrohrer.org/transformers.html

https://kyunghyuncho.me/teaching-fundamentals-of-machine-learning/

https://developers.googleblog.com/torchtpu-running-pytorch-natively-on-tpus-at-google-scale/ TorchTPU: Running PyTorch Natively on TPUs at Google Scale APRIL 7, 2026

TinyLoRA – Learning to Reason in 13 Parameters https://arxiv.org/abs/2602.04118