Tags: inference cost

The cost of AI infrastructure is increasingly driven by memory expenses, with DRAM prices jumping roughly 7x in the past year. As hyperscalers expand data centers, managing prompt caching and memory orchestration is emerging as a key competitive factor. Companies that master cache optimization can reduce token usage and inference costs, opening new avenues for profitability in AI applications. Read more

DeepSeek announced a new experimental AI model featuring Sparse Attention technology that dramatically lowers inference costs for long‑context tasks. The model, released on Hugging Face and accompanied by a research paper on GitHub, uses a lightning indexer and fine‑grained token selection to focus computational resources on the most relevant excerpts. Preliminary tests suggest API call prices can be cut by as much as 50 percent in long‑context scenarios. The open‑weight release invites third‑party validation and positions DeepSeek as a notable player in the ongoing effort to make transformer‑based AI more cost‑effective. Read more

DeepSeek announced its experimental model V3.2‑exp, featuring a new Sparse Attention mechanism that dramatically lowers inference expenses for long‑context tasks. The architecture employs a lightning indexer to prioritize excerpts and a fine‑grained token selector to feed a limited attention window, allowing the model to process extensive context with reduced server load. Preliminary tests suggest API calls in long‑context scenarios could cost up to half as much as before. The model is open‑weight and freely available on Hugging Face, inviting independent verification and broader adoption. Read more