DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85%

DSpark can make decoding faster, but acceptance quality still determines how much speed the system actually realizes.
Tech Times

Qualcomm Closes $3.9 Billion Modular Deal: Meta Validates Full-Stack CUDA Challenge

Qualcomm confirmed a $3.92 billion all-stock deal to buy AI software startup Modular, paired with a Meta Platforms CPU ...

OpenAI unveils first custom AI inference chip, Jalapeño, with Broadcom — and its development was sped-up with OpenAI's own models

The companies attributed this speed to a deep software-hardware co-development process that actively used OpenAI’s own models ...
GitHub

Midscene Python

midscene-python/ ├── midscene/ # Core framework │ ├── core/ # Core framework │ │ ├── agent/ # Agent system │ │ ├── insight/ # AI inference engine │ │ ├── ai_model/ # AI model integration │ │ ├── yaml ...
GitHub

GitHub - facebookincubator/AITemplate: AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore ...

High performance: close to roofline fp16 TensorCore (NVIDIA GPU) / MatrixCore (AMD GPU) performance on major models, including ResNet, MaskRCNN, BERT, VisionTransformer, Stable Diffusion, etc. Unified ...