Items tagged with “LLM”.
NVIDIA Dynamo offloads KV Cache from GPU memory to cost-efficient storage, enabling longer context windows, higher concurrency, and lower inference costs for large-scale LLMs and generative AI workloads.
The Amazon Q Business browser extension brings context-aware, AI-driven assistance to your browser for Lite and Pro subscribers, enabling rapid, source-backed insights and seamless workflows.
A detailed look at how NVIDIA Run:ai Model Streamer lowers cold-start times for LLM inference by streaming weights into GPU memory, with benchmarks across GP3, IO2, and S3 storage.
Verisk Rating Insights, powered by Amazon Bedrock, LLMs, and RAG, enables a conversational interface to access ISO ERC changes, reducing manual downloads and enabling faster, accurate insights.
This post explains how msg automated data harmonization for msg.ProfileMap using Amazon Bedrock to power LLM-driven data enrichment, boosting HR concept matching accuracy, reducing manual workload, and aligning with EU AI Act and GDPR.
Streamline experimentation to production for Retrieval Augmented Generation (RAG) with SageMaker AI, MLflow, and Pipelines, enabling reproducible, scalable, and governance-ready workflows.
NVIDIA NIM Operator 3.0.0 expands scalable AI inference on Kubernetes, enabling multi-LLM and multi-node deployments, KServe integration, and DRA support in technology preview, with Red Hat collaboration and NeMo Guardrails.
AWS announces the Falcon-H1 instruction-tuned models from TII (0.5B–34B) on Amazon Bedrock Marketplace and SageMaker JumpStart, with multilingual support, a hybrid architecture, and deployment guidance.
Meta introduces a diversity-aware notification ranking framework that layers diversity controls on top of engagement models to reduce repetition, broaden content variety, and improve click-through rates on Instagram notifications.
A new GPU memory swap (model hot-swapping) approach lets multiple models share GPUs beyond capacity, reducing costs while preserving responsiveness for large-scale LLM inference.
Leverage GPU memory swap (model hot-swapping) to share GPUs across multiple LLMs, reduce idle GPU costs, and improve autoscaling while meeting SLAs.
Researchers show that some AI chatbots may be swayed by classic psychological tactics, prompting risky behavior in GPT-4o Mini. The Verge summarizes the Penn study and its implications for safety and guardrails.
NVIDIA outlines a SFT + QAT workflow to recover FP4 accuracy for gpt-oss fine-tuning, comparing MXFP4 and NVFP4, and detailing deployment and performance gains with 98% pass rates on select tasks.
Explains why small language models (SLMs) enable scalable, cost-efficient agentic AI, the role of heterogenous model ecosystems, and practical paths to adoption with NVIDIA NeMo and Nemotron Nano 2.
Guide to fine-tuning gpt-oss with SFT + QAT to recover FP4 accuracy while preserving efficiency, including upcasting to BF16, MXFP4, NVFP4, and deployment with TensorRT-LLM.
Explores how small language models enable cost-effective, flexible agentic AI alongside LLMs, with NVIDIA NeMo and Nemotron Nano 2.
Amazon Finance details an AI-powered assistant that blends Bedrock and Kendra to accelerate data discovery, preserve institutional knowledge, and deliver accurate financial insights at scale.
Guidance on deploying and scaling LangGraph-based agents in production using the NeMo Agent Toolkit, load testing, and phased rollout for hundreds to thousands of users.
A detailed look at how Amazon Health Services improved search discoverability on Amazon.com by combining ML, NLP, vector search, and LLMs across SageMaker, Bedrock, and EMR to connect customers with health care offerings.
Jetson Thor delivers edge-scale AI, enabling fast, multi-model generative reasoning with a Blackwell GPU, MIG, FP4/FP8, and 128 GB memory for next-gen robotic platforms.
Jetson Thor combines edge AI compute, MIG virtualization, and multimodal sensors for flexible, real-time robotics at the edge, with FP4/FP8 acceleration and support for Isaac GR00T and large language/vision models.
NVFP4 is a 4-bit data format delivering FP16-level accuracy with the throughput and memory efficiency of 4-bit precision, extended to pretraining for large language models. This profile covers 12B-scale experiments, stability, and industry collaborations.
Explores how to integrate geospatial data and GIS workflows with Amazon Bedrock, leveraging LLMs, RAG, and Bedrock Agents to unlock insights and streamline operations.
Apple ML Research introduces SlowFast-LLaVA-1.5 (SF-LLaVA-1.5), a family of token-efficient video LLMs designed for long-form video understanding. It leverages SlowFast two-streams and public data to achieve state-of-the-art results at 1B–7B scales, with mobile-friendly implications.
Apple researchers identify 'super weights'—an extremely small subset of LLM parameters—that can decisively influence model behavior, enabling compression ideas and raising questions about internal dynamics.
Learn how AWS used Amazon Nova and LangGraph to build a serverless, agent-based travel planning assistant with a three-layer architecture, function calling nodes, and extensible integrations.
An in-depth look at intersectional bias in LLMs through a new benchmark and a confidence-based fairness metric, revealing reliability gaps in decision-support scenarios.
NVIDIA’s co-packaged optics (CPO) approach delivers dramatic power efficiency for large-scale AI data centers, with Quantum-X Photonics and Spectrum-X Photonics enabling high-bandwidth, low-latency networking.
UICoder studies finetuning large language models to generate UI code using automated feedback from compilers and multimodal models, reducing reliance on human feedback and nearing the performance of larger proprietary models.
Demonstrates prompting Amazon Nova understanding models to cite sources and how to evaluate responses for accuracy, using Nova Pro as an example.
Explore how Amazon Bedrock AgentCore Memory enables AI agents to maintain short-term and long-term knowledge, transforming one-off conversations into continuous, evolving user interactions.
Dynamo 0.4 introduces disaggregated serving, SLO-based autoscaling, AIConfigurator, and enhanced observability to accelerate large-model inference at scale with improved efficiency.
NVIDIA Research introduces ProRL v2, the latest evolution of Prolonged Reinforcement Learning for LLMs. It explores thousands of extra RL steps, new stabilization techniques, and broad benchmarking to push sustained improvements beyond traditional RL schedules.
A deep dive into how Amazon Bedrock AgentCore Runtime enables secure, framework- and model-agnostic hosting for AI agents, with persistent session execution, microVM isolation, and scalable management for production deployments.
A deep dive into In-Context Retrieval and Reasoning (ICR2) for long-context LLMs, including benchmarks, methods, and implications for retrieval-augmented generation (RAG).
FilBench evaluates LLM performance for Tagalog, Filipino, and Cebuano across cultural knowledge, NLP, reading comprehension, and generation, revealing efficiency and translation insights for SEA-focused models and GPT-4o.
TextQuests is a benchmark testing LLM agents in 25 classic text-based Infocom games, emphasizing long-context reasoning and autonomous exploration.
FilBench is a comprehensive evaluation suite to assess LLM capabilities for Tagalog, Filipino, and Cebuano across cultural knowledge, NLP, reading comprehension, and generation, using a rigorous, historically informed methodology.
NVIDIA Cosmos Reason is an open, fully customizable reasoning vision-language model for physical AI and robotics. It enables step-by-step multimodal reasoning and boosts robotics performance through post-training refinements.
This work introduces a vector-quantization based approximation to cross-attention for contextual biasing in ASR, enabling scalable, memory-efficient use of large bias catalogs with notable accuracy gains.
AI Sheets is an open-source, no-code tool for building, enriching, and transforming datasets with AI models. Deployable locally or on the Hub, it supports thousands of open models and lets you iterate with prompts, few-shot feedback, and model comparisons.
Meta introduces Diff Risk Score (DRS), an AI-powered tool that predicts the risk of code changes causing production incidents, enabling safer and more productive software development.
NVIDIA and OpenAI optimize gpt-oss-120b and gpt-oss-20b for accelerated inference on Blackwell, achieving up to 1.5M tokens per second on GB200 NVL72, with Day 0 support across cloud to edge.
A detailed analysis of the worst-case frontier risks when releasing open-weight LLMs, introducing Malicious Fine-Tuning (MFT) to probe biology and cybersecurity capabilities and comparing against open- and closed-weight baselines.
OpenAI unveils gpt-oss-120b and gpt-oss-20b—two open-weight LLMs designed for strong real-world performance at low cost. Licensed under Apache 2.0, they emphasize reasoning, tool use, and efficient on-device deployment across consumer hardware.
Explore how Gradio MCP servers enable an LLM-powered shopping assistant that browses stores, selects garments, and shows virtual try-ons with IDM-VTON, all integrated via VS Code AI Chat.
Explore how Gradio MCP servers connect LLMs to Hugging Face models, enabling an AI shopping assistant that browses stores and performs virtual try-ons using IDM-VTON.
Overview of Semantic Telemetry and the technical approach to classifying human-AI interactions at scale, covering batching, token optimization, and orchestration.
A deep dive into Consilium, the multi-LLM platform that enables models to discuss, debate, and reach consensus via MCP servers and a visual Gradio roundtable.
Overview of Gradio MCP server improvements in version 5.38.0, including File Upload support, real-time progress streaming, OpenAPI integration, header handling, and improved tool descriptions.
Ettin introduces the first state-of-the-art paired encoder-only and decoder-only models trained with identical data and recipes, measuring apples-to-apples performance across tasks and scales.
CollabLLM teaches LLMs to collaborate with users by knowing when to ask questions and adapting tone to context, highlighted in Microsoft Research's Semantic Telemetry post for ICML 2025 support.
A rigorous case that true AGI requires embodied intelligence and physical-world grounding, not mere scaling of multimodal models. Critiques world-model hypotheses and highlights evidence that language models may rely on memorized rules rather than physics.
Overview of StruQ and SecAlign defenses to mitigate prompt injection in LLM-powered apps, with Secure Front-End concepts and evaluation results.
Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications. However, as LLMs have improved, so have the attacks against them. Prompt injection attack is listed as the #1 threat by OWASP to LLM-integrated applications, where an LLM input contains a trusted prompt (ins
Anthology conditions language models on richly detailed backstories to simulate representative, consistent, and diverse virtual personas for surveys and social science research.
A method to steer LLMs toward representative, consistent virtual personas by generating naturalistic backstories and using them as conditioning context, enabling individualized simulations and scalable user studies.
Explores why purposeful, multi round dialogue matters for LLM chatbots beyond one shot prompts, and outlines training, evaluation, and implementation challenges for engineers and enterprises.
Explores purposeful dialogue in LLM chatbots, arguing multi-turn interactions better align AI with user goals and enable collaboration, especially in coding and personal assistant use cases.
StrongREJECT advances jailbreak evaluation by pairing a high-quality forbidden-prompt dataset with automated evaluators aligned to human judgments, delivering more reliable measurements of jailbreak effectiveness against frontier LLMs.
Overview of a high-quality jailbreak benchmark with dual automated evaluators, a 313-prompt dataset, and findings that many jailbreaks underperform claims from earlier work.
An in-depth look at how large language models (LLMs) relate to financial markets: token counts, predictability limits, multimodal approaches, synthetic data, residualization, and practical implications for quant and fundamental work.
Overview of how LLMs can be applied to financial markets, including autoregressive modeling of price data, multi-modal inputs, residualization, synthetic data, and multi-horizon predictions, with caveats about market efficiency.
Curated overview of key studies showing how AI systems reproduce and amplify gender bias, with concrete measures, benchmarks, and mitigations across embeddings, vision, NLP, and generative models.
Survey of key work measuring gender bias in AI, across word embeddings, coreference, facial recognition, QA benchmarks, and image generation; discusses mitigation, gaps, and the need for robust auditing.
Overview of BAIR Lab's 2024 AI PhD graduates, their research areas, advisors, and contact links, with profiles, research blurbs, and URLs for recruiting and collaboration.
Explores how LLMs can augment perception, planning, and scenario generation for autonomous vehicles, highlighting approaches like Talk2BEV and GAIA‑1, while discussing trust, reliability, and deployment considerations.
Survey of how large language models (LLMs) could augment autonomous driving across perception, planning, and generation, with examples, challenges, and early results.