Language Models

Inside Sinusoidal Position Embeddings: A Sense of Order

In the groundbreaking 2017 paper “Attention Is All You Need”, Vaswani et al. introduced Sinusoidal Position Embeddings to help Transformers encode positional information, without recurrence or convolution. This elegant, non-learned

Language Models, LLMs, NLP

Inside RoPE: Rotary Magic into Position Embeddings

Self-attention, the beating heart of Transformer architectures, treats its input as an unordered set. That mathematical elegance is also a curse: without extra signals, the model has no idea which

Language Models, LLMs, NLP

SmolLM3 Blueprint: SOTA 3B-Parameter LLM

In the evolving landscape of open-source language models, SmolLM3 emerges as a breakthrough: a 3 billion-parameter, decoder-only transformer that rivals larger 4 billion-parameter peers on many benchmarks, while natively supporting

Language Models, LLMs

Getting Started with Qwen3 – The Thinking Expert

Alibaba Cloud just released Qwen3, the latest model from the popular Qwen series. It outperforms all the other top-tier thinking LLMs, such as DeepSeek-R1, o1, o3-mini, Grok-3, and Gemini-2.5-Pro. Unlike

Generative AI, Language Models, LLMs, NLP

Beginner’s Guide to Embedding Models

As artificial intelligence continues to advance, Embedding Models have become fundamental to how machines interpret and interact with unstructured data. By translating inputs like text, images, audio, and video into