https://techgazette.com.br/learn/inference/ New guides on inference and gpus, by the people who build it. Serving models fast and cheap. en 180 Fri, 16 Oct 2026 20:48:00 GMT https://aws.amazon.com/blogs/machine-learning/building-trust-in-ai-agents-a-production-ready-evaluation-framework-with-amazon-bedrock-agentcore/ https://aws.amazon.com/blogs/machine-learning/building-trust-in-ai-agents-a-production-ready-evaluation-framework-with-amazon-bedrock-agentcore/ Fri, 16 Oct 2026 20:48:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — Autonomous AI agents fail softly: latency stays flat while answer quality quietly erodes. This post shares a nine-stage evaluation framework, built on Amazon Bedrock AgentCore, that catches silent quality regressions acr… https://www.digitalocean.com/community/tutorials/serverless-inference-throughput-benchmark https://www.digitalocean.com/community/tutorials/serverless-inference-throughput-benchmark Thu, 15 Oct 2026 11:53:00 GMT DigitalOcean Tutorials Inference and GPUs DigitalOcean Tutorials — We tested serverless inference with 1,200+ requests. Real throughput, latency, costs, and concurrency limits. Data for AI startup infrastructure decisions. https://developer.nvidia.com/blog/rewrite-apache-spark-udfs-gpu-acceleration-ai-agents/ https://developer.nvidia.com/blog/rewrite-apache-spark-udfs-gpu-acceleration-ai-agents/ Tue, 13 Oct 2026 21:33:00 GMT NVIDIA Developer Inference and GPUs NVIDIA Developer — Enterprise applications have become increasingly demanding. Agentic AI has compounded the need for vast amounts of enterprise data with frequent, complex... https://blog.cloudflare.com/rewinding-the-exponential/ https://blog.cloudflare.com/rewinding-the-exponential/ Tue, 13 Oct 2026 15:51:00 GMT Cloudflare Inference and GPUs Cloudflare — Our strategies to optimize our data pipeline throughput without compromising data quality. https://aws.amazon.com/blogs/machine-learning/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai/ https://aws.amazon.com/blogs/machine-learning/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai/ Tue, 13 Oct 2026 15:17:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilat… https://wf.coreweave.com/blog/tutorial-benchmarking-gpt-6-astra-vs-claude-fable-5-1-vs-gpt-5-6-sol-using-w-b-weave https://wf.coreweave.com/blog/tutorial-benchmarking-gpt-6-astra-vs-claude-fable-5-1-vs-gpt-5-6-sol-using-w-b-weave Tue, 13 Oct 2026 14:47:00 GMT CoreWeave Inference and GPUs CoreWeave — Learn how to benchmark GPT-6 Astra, Claude Fable 5.1, and GPT-5.6 Sol with Weave, comparing model quality, latency, cost, and performance across practical evaluation tasks. https://devblogs.microsoft.com/foundry/more-than-a-schema-content-understanding-prebuilt-analyzers/ https://devblogs.microsoft.com/foundry/more-than-a-schema-content-understanding-prebuilt-analyzers/ Tue, 13 Oct 2026 14:20:00 GMT Microsoft Foundry Inference and GPUs Microsoft Foundry — Learn how Azure Content Understanding prebuilt analyzers combine domain knowledge with grounding and confidence, and what Advanced Contextualization preview benchmarks show about quality, modeled cost, and latency. The p… https://newsroom.arm.com/blog/building-local-ai-for-the-intelligent-edge https://newsroom.arm.com/blog/building-local-ai-for-the-intelligent-edge Tue, 13 Oct 2026 10:00:00 GMT Arm Newsroom Inference and GPUs Arm Newsroom — As AI moves closer to devices and physical systems, developers need to build applications that can sense, reason, and act locally while balancing latency, privacy, power, connectivity, and software complexity. The post H… https://clickhouse.com/blog/chasing-32ms-tail-latency-clickhouse https://clickhouse.com/blog/chasing-32ms-tail-latency-clickhouse Tue, 13 Oct 2026 00:00:00 GMT ClickHouse Inference and GPUs ClickHouse — A customer saw an extra ~32 ms on simple SELECT queries after idle periods. We traced it to unbatched Native protocol metadata packets overflowing a reset TCP congestion window, reproduced it with network namespaces, and… https://aws.amazon.com/blogs/machine-learning/accessing-openai-gpt-models-on-amazon-bedrock-with-global-cross-region-inference-in-thailand-malaysia-singapore-indonesia-and-taiwan/ https://aws.amazon.com/blogs/machine-learning/accessing-openai-gpt-models-on-amazon-bedrock-with-global-cross-region-inference-in-thailand-malaysia-singapore-indonesia-and-taiwan/ Mon, 12 Oct 2026 17:52:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — Learn how to run OpenAI GPT-6 Sol, Luna, and Astra on Amazon Bedrock through global cross-Region inference from Singapore, Jakarta, Thailand, Malaysia, and Taipei. This post covers the Converse and OpenAI-compatible APIs… https://engineering.fb.com/2026/10/12/ml-applications/turning-opportunistic-ai-inference-capacity-into-better-ads-recommendations/ https://engineering.fb.com/2026/10/12/ml-applications/turning-opportunistic-ai-inference-capacity-into-better-ads-recommendations/ Mon, 12 Oct 2026 16:00:00 GMT Meta Engineering Inference and GPUs Meta Engineering — At Meta, we’ve built a system that enhances ads recommendations by leveraging idle inference capacity in our fleet. Two elasticity levers in the system allow ads recommendation work to expand across opportunistic capacit… https://aws.amazon.com/blogs/machine-learning/run-multi-node-inference-workloads-using-ray-llm-deep-learning-containers/ https://aws.amazon.com/blogs/machine-learning/run-multi-node-inference-workloads-using-ray-llm-deep-learning-containers/ Mon, 12 Oct 2026 15:50:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — Learn how to deploy one large language model across two GPU nodes on Amazon EKS using the multi-node Ray LLM Deep Learning Containers, with separate prefill and decode nodes behind one OpenAI-compatible endpoint. The pre… https://azure.microsoft.com/en-us/blog/deep-dive-into-azure-maia-300-architecture/ https://azure.microsoft.com/en-us/blog/deep-dive-into-azure-maia-300-architecture/ Mon, 12 Oct 2026 13:00:00 GMT Microsoft Azure Inference and GPUs Microsoft Azure — Maia 300 operates as a single platform to meet the demands of cloud-scale agentic inference. The post Deep dive into Azure Maia 300 architecture appeared first on Microsoft Azure Blog. https://www.digitalocean.com/community/tutorials/deployment-isnt-the-bottleneck-inference-economics-is https://www.digitalocean.com/community/tutorials/deployment-isnt-the-bottleneck-inference-economics-is Fri, 09 Oct 2026 20:01:00 GMT DigitalOcean Tutorials Inference and GPUs DigitalOcean Tutorials — Deploying an AI model takes an afternoon. Keeping it affordable is the hard part. Learn how cold starts, GPU utilization, and cost per request decide the bill. https://cohere.com/blog/shared-or-dedicated-inference-for-embed-rerank https://cohere.com/blog/shared-or-dedicated-inference-for-embed-rerank Fri, 09 Oct 2026 18:54:00 GMT Cohere Inference and GPUs Cohere — Shared or dedicated AI inference? Learn how embedding and reranking workloads, traffic patterns, latency, and cost determine the right deployment choice. https://www.baseten.co/blog/what-makes-a-high-quality-nvfp4-quantization/ https://www.baseten.co/blog/what-makes-a-high-quality-nvfp4-quantization/ Fri, 09 Oct 2026 14:16:00 GMT Baseten Inference and GPUs Baseten — NVFP4 makes models smaller and faster, but quantizing some layers can hurt quality. An intuitive look at how to choose which layers can run at 4 bits. https://realpython.com/huggingface-transformers/ https://realpython.com/huggingface-transformers/ Fri, 09 Oct 2026 14:00:00 GMT Real Python Inference and GPUs Real Python — Get hands-on with Hugging Face Transformers in Python. Download, run, and inspect pretrained open-source AI models, and speed up inference on GPUs. https://www.digitalocean.com/community/tutorials/should-you-use-spot-or-on-demand-gpus-for-model-training https://www.digitalocean.com/community/tutorials/should-you-use-spot-or-on-demand-gpus-for-model-training Fri, 09 Oct 2026 00:00:00 GMT DigitalOcean Tutorials Inference and GPUs DigitalOcean Tutorials — Spot vs on-demand GPUs for model training: measured reclaim and resume costs on MI355X and B300, and the break-even discount at which spot actually saves money. https://aws.amazon.com/blogs/machine-learning/pay-per-inference-for-ai-agents-how-blockrun-and-incarna-use-amazon-bedrock-agentcore-payments/ https://aws.amazon.com/blogs/machine-learning/pay-per-inference-for-ai-agents-how-blockrun-and-incarna-use-amazon-bedrock-agentcore-payments/ Thu, 08 Oct 2026 18:33:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — Amazon Bedrock AgentCore payments gives AI agents a managed way to pay for services on demand, with spending limits enforced by the infrastructure. See how Incarna's agents pay BlockRun for model inference one request at… https://pytorch.org/blog/session-aware-agentic-inference-with-nvidia-dynamo/ https://pytorch.org/blog/session-aware-agentic-inference-with-nvidia-dynamo/ Thu, 08 Oct 2026 18:13:00 GMT PyTorch Inference and GPUs PyTorch — TL;DR Agentic workloads change the traffic an inference server sees. Unlike single-turn chat, an agent session can involve a large initial prefill, repeated model calls, and subagents running in parallel,... The post Ses… https://aws.amazon.com/blogs/machine-learning/share-gpu-clusters-across-teams-with-isolation-and-fairness-using-amazon-sagemaker-hyperpod/ https://aws.amazon.com/blogs/machine-learning/share-gpu-clusters-across-teams-with-isolation-and-fairness-using-amazon-sagemaker-hyperpod/ Thu, 08 Oct 2026 16:20:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — A reference architecture for securely sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams, using AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernetes namespaces for is… https://rocm.blogs.amd.com/artificial-intelligence/primus-hybrid-llm/README.html https://rocm.blogs.amd.com/artificial-intelligence/primus-hybrid-llm/README.html Thu, 08 Oct 2026 00:00:00 GMT AMD ROCm Inference and GPUs AMD ROCm — In this blog you will learn how to pre-train hybrid LLMs from scratch with Primus on AMD Instinct™ MI355X GPUs. In particular, we will cover how to declare a hybrid layer stack with a single configuration string, how thr… https://rocm.blogs.amd.com/software-tools-optimization/di-kimi-k3-mxfp4/README.html https://rocm.blogs.amd.com/software-tools-optimization/di-kimi-k3-mxfp4/README.html Thu, 08 Oct 2026 00:00:00 GMT AMD ROCm Inference and GPUs AMD ROCm — This post is part of the AMD Disaggregated Inference (DI) Blogs Series, which takes frontier open models from a single-node demo to production, multi-node serving on AMD Instinct™ GPUs https://rocm.blogs.amd.com/software-tools-optimization/mi455x-vs-mi355x/README.html https://rocm.blogs.amd.com/software-tools-optimization/mi455x-vs-mi355x/README.html Thu, 08 Oct 2026 00:00:00 GMT AMD ROCm Inference and GPUs AMD ROCm — At Advancing AI 2026, AMD shared a set of performance results comparing the new AMD Instinct™ MI455X (Helios) accelerator against the current-generation AMD Instinct™ MI355X. This post explains what went into those numbe… https://developer.nvidia.com/blog/validate-ai-factory-changes-with-digital-twins-and-ai-agents/ https://developer.nvidia.com/blog/validate-ai-factory-changes-with-digital-twins-and-ai-agents/ Wed, 07 Oct 2026 16:00:00 GMT NVIDIA Developer Inference and GPUs NVIDIA Developer — AI factories are some of the most complex operations in the world, combining GPUs, CPUs, switches, DPUs, and SuperNICs alongside schedulers, orchestration... https://developer.nvidia.com/blog/faster-scientific-image-analysis-with-nvidia-cuphoton/ https://developer.nvidia.com/blog/faster-scientific-image-analysis-with-nvidia-cuphoton/ Wed, 07 Oct 2026 15:00:00 GMT NVIDIA Developer Inference and GPUs NVIDIA Developer — Observatories and telescopes, lasers and X-ray light sources, and other high-throughput instruments generate image data faster than CPU-bound pipelines can... https://www.digitalocean.com/community/tutorials/semantic-cache-verification-jev-vs-cosine-similarity https://www.digitalocean.com/community/tutorials/semantic-cache-verification-jev-vs-cosine-similarity Wed, 07 Oct 2026 11:00:00 GMT DigitalOcean Tutorials Inference and GPUs DigitalOcean Tutorials — We tested cosine similarity, prompted LLMs, and a purpose-built verifier (Jev) on 500 labeled query pairs, and cosine was clearly the least accurate, serving near-misses like "capital of Kenya" for "capital of Jamaica."… https://pytorch.org/blog/modernizing-table-batched-embeddings-with-fbtriton/ https://pytorch.org/blog/modernizing-table-batched-embeddings-with-fbtriton/ Tue, 06 Oct 2026 22:57:00 GMT PyTorch Inference and GPUs PyTorch — This post explores the FBTriton kernel design for Table Batched Embedding (TBE) forward and backward passes. These core operators handle embedding lookups across thousands of sharded GPUs within recommendation systems...… https://developer.nvidia.com/blog/doca-gpunetio-gda-ki-unified-gpu-networking/ https://developer.nvidia.com/blog/doca-gpunetio-gda-ki-unified-gpu-networking/ Tue, 06 Oct 2026 19:07:00 GMT NVIDIA Developer Inference and GPUs NVIDIA Developer — GPU applications increasingly need networking and data movement to behave like first-class GPU-controlled operations rather than host-driven services. When the... https://kubernetes.io/blog/2026/10/06/kubernetes-cgroups-v2-shift/ https://kubernetes.io/blog/2026/10/06/kubernetes-cgroups-v2-shift/ Tue, 06 Oct 2026 18:00:00 GMT Kubernetes Inference and GPUs Kubernetes — In Linux, cgroups (control groups) are a kernel feature used for managing system resources. Kubernetes uses cgroups to allocate resources like CPU and memory to containers, ensuring that applications run smoothly without… https://gpuopen.com/learn/rga-source-to-isa-correlation-gpu-code-objects/ https://gpuopen.com/learn/rga-source-to-isa-correlation-gpu-code-objects/ Tue, 06 Oct 2026 17:00:00 GMT AMD GPUOpen Inference and GPUs AMD GPUOpen — Learn how AMD Radeon GPU Analyzer 2.15 correlates pre-compiled AMD GPU code objects back to source lines, making ISA analysis and optimization far easier. https://www.percona.com/blog/dbtrail-analytical-reports-on-your-mysql-data-without-running-them-on-mysql-part-1/ https://www.percona.com/blog/dbtrail-analytical-reports-on-your-mysql-data-without-running-them-on-mysql-part-1/ Tue, 06 Oct 2026 16:28:00 GMT Percona Inference and GPUs Percona — MySQL is very good at serving application workloads, but things get more complicated when somebody decides to run a large report on the same server. A scan can push hot pages out of the buffer pool. A long-running read c… https://developer.nvidia.com/blog/control-how-your-gpu-shares-work-with-green-contexts/ https://developer.nvidia.com/blog/control-how-your-gpu-shares-work-with-green-contexts/ Tue, 06 Oct 2026 15:00:00 GMT NVIDIA Developer Inference and GPUs NVIDIA Developer — GPU applications increasingly consist of multiple independent components running at the same time within a single process: a latency-sensitive operator... https://cloud.google.com/blog/topics/developers-practitioners/networking-for-ai-inference-model-serving-gke-only-and-for-all-other-backends/ https://cloud.google.com/blog/topics/developers-practitioners/networking-for-ai-inference-model-serving-gke-only-and-for-all-other-backends/ Tue, 06 Oct 2026 12:00:00 GMT Google Cloud Inference and GPUs Google Cloud — Enterprises and individual developers frequently run multiple AI inference models. The right architecture can simplify how the models are called while also providing centralized governance. In this post, we'll look at tw… https://rocm.blogs.amd.com/artificial-intelligence/taf-blog/README.html https://rocm.blogs.amd.com/artificial-intelligence/taf-blog/README.html Tue, 06 Oct 2026 00:00:00 GMT AMD ROCm Inference and GPUs AMD ROCm — Modern AI accelerators can execute an extraordinary number of floating-point operations every second. But a single FLOPs number rarely tells the full performance story. In the first two blogs in this series, we discussed… https://www.databricks.com/blog/nearest-join-scaling-vector-search-databricks-runtime https://www.databricks.com/blog/nearest-join-scaling-vector-search-databricks-runtime Mon, 05 Oct 2026 20:00:00 GMT Databricks Inference and GPUs Databricks — Vector search originated as a serving problem. The classical use case is a chatbot... https://www.ai21.com/blog/how-ai21-manages-its-gpu-fleet-with-kueue/ https://www.ai21.com/blog/how-ai21-manages-its-gpu-fleet-with-kueue/ Sun, 04 Oct 2026 11:37:00 GMT AI21 Labs Inference and GPUs AI21 Labs — An independent verifier takes agentic search past published SOTA. Our trained 8B does it for almost $0 per question. https://pytorch.org/blog/building-a-high-performance-and-portable-vllm-linear-backend-with-helion/ https://pytorch.org/blog/building-a-high-performance-and-portable-vllm-linear-backend-with-helion/ Fri, 02 Oct 2026 19:55:00 GMT PyTorch Inference and GPUs PyTorch — TL;DR We integrated Helion into vLLM’s linear backend to explore how an autotuned, high-level kernel DSL can improve LLM inference performance while reducing kernel implementation complexity. A single Helion general... T… https://arize.com/blog/prompt-caching-benchmark/ https://arize.com/blog/prompt-caching-benchmark/ Fri, 02 Oct 2026 18:01:00 GMT Arize Inference and GPUs Arize — We benchmarked prompt caching across DeepSeek, GLM, GPT, and Claude using Harbor evals and Phoenix traces, so we could compare cache reuse, estimated cost, and latency on the same multi-turn shopping assistant agent. The… https://aws.amazon.com/blogs/machine-learning/fine-tune-a-search-agent-with-multi-turn-rl-on-amazon-sagemaker-ai/ https://aws.amazon.com/blogs/machine-learning/fine-tune-a-search-agent-with-multi-turn-rl-on-amazon-sagemaker-ai/ Fri, 02 Oct 2026 15:44:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — Fine-tuning teaches a small search agent your tools and environment, giving it the reliability of a frontier model at lower latency and cost. In this post, we fine-tune an LLM-powered search agent with multi-turn reinfor… https://wf.coreweave.com/blog/why-open-models-can-win-on-enterprise-cost-and-control https://wf.coreweave.com/blog/why-open-models-can-win-on-enterprise-cost-and-control Fri, 02 Oct 2026 14:12:00 GMT CoreWeave Inference and GPUs CoreWeave — See how enterprise teams cut inference costs and keep control of fine-tuning, distillation, and guardrails by running open models on their own infrastructure. https://clickhouse.com/blog/direct-io-managed-postgres-backups https://clickhouse.com/blog/direct-io-managed-postgres-backups Fri, 02 Oct 2026 13:27:00 GMT ClickHouse Inference and GPUs ClickHouse — ClickHouse Managed Postgres uses direct I/O and stripe-sized reads to keep backups fast while protecting the page cache and query latency. https://www.datadoghq.com/blog/key-metrics-for-databricks-monitoring/ https://www.datadoghq.com/blog/key-metrics-for-databricks-monitoring/ Fri, 02 Oct 2026 00:00:00 GMT Datadog Inference and GPUs Datadog — Learn about key metrics for monitoring the health, performance, and costs of Databricks data engineering, analytics, and Model Serving workloads. https://pytorch.org/blog/optimizing-jagged-flash-attention-with-tlx-the-road-toward-sota-fa4-on-blackwell/ https://pytorch.org/blog/optimizing-jagged-flash-attention-with-tlx-the-road-toward-sota-fa4-on-blackwell/ Thu, 01 Oct 2026 22:26:00 GMT PyTorch Inference and GPUs PyTorch — TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) — the attention kernel behind Meta’s Generative Ads Model (GEM) — on NVIDIA Blackwell (B200), built... The post Optimizing Jagged Flash Attenti… https://developer.nvidia.com/blog/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples/ https://developer.nvidia.com/blog/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples/ Thu, 01 Oct 2026 17:59:00 GMT NVIDIA Developer Inference and GPUs NVIDIA Developer — Adding AI models to local applications requires a portable model format, a reliable runtime, and acceleration that works across target systems. Do Inference Now... https://www.digitalocean.com/community/tutorials/ttft-latency-audit-guide https://www.digitalocean.com/community/tutorials/ttft-latency-audit-guide Thu, 01 Oct 2026 16:03:00 GMT DigitalOcean Tutorials Inference and GPUs DigitalOcean Tutorials — Real time-to-first-token numbers from 380 test calls across six models on DigitalOcean Serverless Inference, including two multi-minute delays on one model. https://clickhouse.com/blog/linkedin-observability-at-scale https://clickhouse.com/blog/linkedin-observability-at-scale Thu, 01 Oct 2026 15:50:00 GMT ClickHouse Inference and GPUs ClickHouse — LinkedIn expanded its ClickHouse observability stack from distributed tracing to metric discovery and analytics, consolidating 13B+ metrics into one index serving 150k+ queries per minute. https://www.lancedb.com/blog/how-jev-compares-to-other-rerankers https://www.lancedb.com/blog/how-jev-compares-to-other-rerankers Thu, 01 Oct 2026 03:58:00 GMT LanceDB Inference and GPUs LanceDB — See how Jev performs in a benchmark of 19 reranker configurations across five datasets, covering accuracy, latency, and prompt-driven relevance for LanceDB search. https://www.digitalocean.com/community/tutorials/multi-tenant-gpu-isolation https://www.digitalocean.com/community/tutorials/multi-tenant-gpu-isolation Thu, 01 Oct 2026 00:00:00 GMT DigitalOcean Tutorials Inference and GPUs DigitalOcean Tutorials — Multi-tenant GPU isolation, measured on an H100 Droplet: which of 7 serving layers 'dedicated' covers, which it doesn't, and how to read it off telemetry. https://aws.amazon.com/blogs/machine-learning/build-a-multi-agent-music-production-pipeline-on-amazon-bedrock-agentcore-runtime-instances/ https://aws.amazon.com/blogs/machine-learning/build-a-multi-agent-music-production-pipeline-on-amazon-bedrock-agentcore-runtime-instances/ Wed, 30 Sep 2026 15:21:00 GMT AWS Machine Learning Inference and GPUs AWS Machine Learning — Amazon Bedrock AgentCore Runtime Instances gives multi-agent workflows AWS managed EC2 infrastructure with GPUs, persistent volumes, and multi-day sessions. In this post, we deploy a three-agent music production pipeline…