Tabiiy Intellekt

Muhandislar uchun Amaliy Kompositsion O'ylash/Fikrlash. Ushbu kitob ETH Tsyurixda Amaliy Kategiryalar Nazariyasi kursining darsligi hisoblanadi. Murakkab tizimlarni rejalashtirishda, ularni tushunib yetishda va umumlashtirishda kategoriylar nazariyasining…

"Your impact is directly proportional to how much pain you cause to infra." https://x.com/cHHillee/status/2082601096250589620

https://x.com/__alpoge__/status/2079028340955197566?s=20

Kimi K3 va GLM 5.2 tufayli Fable joyida qolar ekan

The ML practitioners have at most two life stages: the never ending attempt to close the gestalt of gpu-poorness; and joining a big lab (which might never happen).

Alifbo o'zgarishining bir tarafi bor. Zamonaviy LLMlar yangi alifboga mostlashmagan bo'ladi. Kiruvchi va chiquvchi satrlarni mostlashtiruvchi dasturlar yozib qo'yganda ham, masalan, tokenlash eng effectiv bo'lmaydi. Masalan, o'zbekchada ishlaydigan LLMlarni olib fine-tune qilish kerak bo'ladi yangi alifbo uchun. Bu ham yana bir xarajat-da endi.

🏆 IOI 2026 Call for Volunteers IOI kabi katta tadbirlarda volontyorlar juda ham muhim. Ularning vazifalariga tashkiliy ishlar bilan yordamlashish, boshqa davlatlik ishtirokchilarga gid (guide) bo'lish va boshqalar kirishi mumkin. Asosiy talablar ingliz tilida erkin muloqot qila olish, ma'suliyatli va "open-minded" bo'lish. Ko'pgina IOI'larda qatnashgan holda ishontirib ayta olamizki, zerikib qolmaysiz. Shuningdek, volontyorlarga sertifikat va IOI Merchandise ham beriladi. Ko'proq ma'lumot uchun: - https://t.me/xalqaro_it_olimpiada/539 🚀 Arizalar: - https://forms.gle/7Gt6TFXPB3o5KPU28 Muddat: 5-iyul 🧑‍💻 @cp_uz

QAT orqali BNN train qilish Bu semester universitetda Tensor Processor Design for Image Recognition degan qiziq kurs olgan edim. Kurs final project sifatida CNN'ni FPGA'da run qilishni optimallashtirish edi. To'liqroq aytganda MNIST digit detection datasetni…

QAT orqali BNN train qilish Bu semester universitetda Tensor Processor Design for Image Recognition degan qiziq kurs olgan edim. Kurs final project sifatida CNN'ni FPGA'da run qilishni optimallashtirish edi. To'liqroq aytganda MNIST digit detection datasetni hamma 70k rasmlarini pynq-z2 board'da eng tez inference qilish. Talablar, accuracy >93%, network architecture berilgan(3 conv layers + max pool + linear layer), conv netning hamma layerlari chipning PL(processing logic) qismida ishlashi kerak va berilgan float32 ko'rinishdagi datasetni diskdan o'qishdan keyingi hamma amallar benchmark timing ichiga bo'lishi kerak. Bu jamoaviy loyiha edi, men va bir koreys kursdoshim birga optimize qilib oxirida barcha guruhlar ichida eng tez 133ms natijaga erishdik. To'liq loyiha ko'p qismlardan iborat, bugun CNN training qismi haqida yozaman. FPGA juda low level qurilma, sodda qilib aytganda juda ko'p customizable logic gatelar. floating point amallar juda sekin ~10 clock cycle, fixed point yoki butun sonli amallarga nisbatan 1 clock cycle. Shuning uchun float32 yoki bfloat16 tipdagi network qurishni ma'nosi yo'q. Integer quantization qilamiz, xo'sh necha bitgacha tusha olamiz? 8 bit? 4 bit? 2 bit? 1 bit. MNIST dataset juda sodda bo'lgani uchun to'liq binary neural network train qilib yuqori accuracy oldim, va to'liq project shuni ustiga qurildi. MNIST uchun 1 bitlik CNN'ni PTQ(post training quantization) orqali ham qilsa bo'lishi kerak, chunki dataset sodda. Lekin tavakkal qilib o'tirmay QAT(quantization aware training) orqali qilib qo'ya qoldim. Nomi Binary Neural Network bo'lgani bilan weightlar va activationlar shunchaki 01 emas. Ha xotirada 01 ko'rinishida bo'ladi lekin, men matematik tarafdan 0->1, 1->-1 sifatida ishlatdim. Shunda matematikasida hamma weightlar va activationlar -1 yoki +1 bo'ladi. weight/activationlarni +-1 ga yaxlitlash oson, hard=torch.where(soft >= 0, torch.ones_like(soft), -torch.ones_like(soft)). torch.sign(0)=0 bo'lgani uchun to'g'ri kelmaydi. Bitta muammo bor, bu amal differentiable emas. Backpropagation orqali train qilib bo'lmaydi. Sodda yechim bu STE(straight through estimator). Faqat backward pass uchun binarize qilish amalimizni diffrentiable qilib ko'rsatamiz, aniqroq aytganda binarize'(x) = x if -1 < x < 1 else 0. def binarize(x: torch.Tensor) -> torch.Tensor: soft = F.hardtanh(x, -1, 1) hard = torch.where(soft >= 0, torch.ones_like(soft), -torch.ones_like(soft)) return (hard - soft).detach() + soft Shunda forward pass'da `hard`ni o'zi qoladi (-soft + soft = 0), lekin backward pass'da faqat soft ko'rinadi (.detach() gradient flowni kesib qo'yadi). Endi hamma weightlar va activationlarni ishlatishdan oldin binarize qilib ishlatsak forward pass'da hammasi +-1 bo'ladi va backpropagation bilan train qilish ham ishlaydi. O'zim menga eng yoqadigan mavzulardan biri backpropagation haqida yozmoqchi edim. Shu case orqali backpropagation qanday ishlashini bilish muhimligi, uni o'zimizni foydamizga shunaqa tricklar uchun ishlatsa bo'lishini ko'rsata oldim deb o'ylayman.

https://arxiv.org/pdf/2601.14888 5 kilometrdan ko’rgandim knowledge distillation kvantizatsiya uchun mosroqligini.

Yana bir chiroyli ish: https://haeggee.github.io/posts/magnitude-direction-decoupling Avvalroq, model parameterlarini chegaralash uchun streaming power iteration / lanczos dan foydalanib ko'rgan edim. Shunaqa eksperimentlar qilganimda WD juda bir xalaqit beradi, suvni bulg'alaydi. Endi mana WD dan qutulsa bo'larkan.

Yoshligimda eshitgandim, qaymoqqa sut qo'shib sotishardi. Endi top LLMlarni interpolyatsiya qilib yangi SOTA e'lon qilishyaptikan. Rio-3.5-Open-397B ≈ 0.6 x Nex-N2_pro + 0.4 x Qwen https://github.com/nex-agi/Nex-N2/issues/4

I guess a very important video, given we're autoriding AI technology optimization. Given even claude has broken, pink self-image of itself, we will probably be doing the same talk again after 20 years or less. https://youtu.be/tlQ7EoJDTQY

Fable 5 (link) yasab berdi https://claude.ai/public/artifacts/e1e784a3-87b9-4fcc-a8cb-284138e9f16d

Fable 5 (link) yasab berdi https://claude.ai/public/artifacts/e1e784a3-87b9-4fcc-a8cb-284138e9f16d

LLMlardan oldingi internetning man yoqtirmaydigan tomonlaridan biri uning ortiqcha isrofgarchiligi edi. Oddiy bir twitterga o'xshagan sayt ochsangiz, googleda qidirsangiz, ma'lumotlar tekst bo'lsa ham sahifalar yuzlab kilobaytlar, ba'zan megabaytlar bo'ladi, haliyam shunday. LLMlar bizga yana optimal traffikni qaytarayapti, endi sarflaydigan vaqtimiz, ishlatadigan internet trafigimiz va oladigan ma'lumotimizga (ko'proq) proporsional. Eski internetga shunday bir jazo kerak edi va bu juda yaxshi.

Jesse's leaning over the laptop, grinning. JESSE: Yo, I got it. Asked the AI for the whole plan. It's like ninety-eight percent sure, Mr. White. WALT: Run it again. JESSE: I just ran it. WALT: Run it again. JESSE: It's gonna say the same thing, man. It's confident. WALT: So are you. Ask it like we're broke. Ask it like we're getting watched. JESSE: ...Why would I lie to the robot? WALT: Do it. (He does it. Three more times. The grin's gone.) JESSE: Okay so. Now it's saying four different things. WALT: Good. JESSE: Good?! It contradicted itself, yo! WALT: Where. JESSE: ...The route. Two say the route's fine. Two say it gets us pinched. WALT: That's the part you didn't think about. JESSE: So we toss those? WALT: No. JESSE: ...We keep the bad ones? WALT: You tell it where it's wrong. The route. Tell it. JESSE: And then it just agrees with me. It always agrees with me. WALT: Then make it earn the agreement. Again. JESSE: This is taking forever, man. WALT: You wanted ninety-eight percent in three seconds. JESSE: ...Yeah? WALT: That's the part that gets people killed. (Jesse types. Slower this time.) JESSE: ...Okay. This one's actually good. WALT: Now it's good. JESSE: It said the same thing the first one did, basically! WALT: No. The first one was lucky. Basically, use LLMs like Walt 🗣

Iqrorlar qandaydir mikromexanik darajada ammallarga teng bo’lsa kerak. Kamida iqror qilishni shunday desa bo’ladi. Iqror yana kelajakdagi amallar uchun “bashorat”, amallar esa o’tmish haqida. Shunda qaysi birida signal ko’proq (shovqinga qaraganda)? Masalan, yosh insonni olsak, unda kam amallar bor, bu amallar shovqinli, iqrorlar balki katta kelajak integrali uchun yaxshiroq bashoratchi. Yoshi kattaroq odamda esa amallarning o’tmish integrali ancha katta va signali ko’p, kelajak esa (iqrorlar bilan birga) ahamiyati kamroq. Kontext: https://t.me/iqtisodchi_kundaligi/4168

Second price auction for compute. I am bidding like 2.92 and winning. Some mf comes and outbids and kills my runs and pauses his bid. SECOND PRICE AUCTIONS DO NOT WORK FOR COMPUTE PLATFORMS