How Closely Should Language Models Read Old Context?
Block-hybrid attention that reads the active block exactly while compressing past blocks in diffusion language models

I like building things, improving things, and solving problems — headed toward entrepreneurship.
Block-hybrid attention that reads the active block exactly while compressing past blocks in diffusion language models
Where AI Hardware Is Headed