CloudHero Weekly / Issue 003
Iceberg migration, AI spend controls and cloud recovery
Developments from to
AWS’s managed Hive-to-Iceberg migration gives data teams a reason to revisit estimates for custom migration tooling. This week also brings consequential details about AI spending controls, European VMware renewals, agent infrastructure and what happened after Google restored service during October’s London outage.
S3 Tables changes the migration estimate, not the acceptance criteria
On 4 November, AWS launched managed migration from Glue-registered Hive tables stored as Parquet into S3 Tables. Source and destination must share an AWS account and region. European launch regions include Frankfurt, Ireland and Stockholm. AWS handles compatibility checks, conversion, recompression and compaction, with a brief interruption at cutover rather than zero downtime.
For an existing modernization project, reconsider custom copy-and-conversion work before commissioning it. Preserve engineering capacity for query validation, writer coordination and recovery.
The destination remains read-only until completion, which enables writes and ends further migration operations. Setting reviewMigrationPlan=true allows compatibility-report review before copying.
An intact source is not necessarily a current rollback target. In a hypothetical billing migration, records written only to Iceberg after cutover would be absent if readers switched back to Hive.
Before completion, specify how those records would be replayed or reconciled. Compare representative query results and performance, not just row counts. Easier conversion improves an existing business case; it does not create one.
Snowflake’s AI budgets are not immediate spending ceilings
On 3 November, Snowflake made Cortex AI Gateway inference, observability and cost management generally available. It supports OpenAI Chat Completions and Anthropic Messages API formats, request tracing and user-level spending attribution. Trace analytics and tool connectivity remain Preview.
The important distinction is enforcement. Shared resource budgets provide notifications and can invoke customer-written procedures. Evaluation can take up to 6.5 hours, or one hour with the low-latency option. Per-user quotas provide blocking within minutes and are not pooled across users.
Platform and finance teams should agree on acceptable overshoot. In a proposed staging test, set a small quota, generate bounded traffic and measure the interval between reaching the threshold and requests being rejected.
For existing Snowflake users, this may reduce bespoke attribution and control work. It is not independently a reason to add another gateway. Confirm model availability and permitted processing locations before routing European workloads.
Google’s London outage exposes a second recovery clock
The 4 November response from the Bank of England, PRA and FCA adds evidence about Google Cloud’s 20 October London outage. Google’s detailed analysis appeared on 27 October; this week’s development is the regulators’ initial assessment, not a new incident.
Two findings deserve attention. Regulators initially learned about the disruption through affected firms rather than Google. Some firms also needed additional time to restore their services after reconnecting to cloud infrastructure. Almost all normal operations resumed by day-end.
For critical services, measure both provider restoration and business recovery. A hypothetical utility payment service is not recovered merely because its endpoint responds if transactions remain unreconciled.
At the next exercise, test who detects the incident, who receives notification and what proves that transactions work correctly again. This is a transferable operating lesson, not a new utility or telecom regulation. The UK financial-sector review remains underway, with a fuller response due by 20 January 2027.
EEA VMware customers gain another procurement option
Broadcom’s 4 November update says vSphere Standard 8 and vSphere Enterprise Plus 8 remain available for sale to customers in the European Economic Area until 31 October 2027. Active subscriptions include Extended Support until the earlier of their expiry or 31 October 2028.
If product availability or support timing is driving a migration, revisit the comparison before contract approval. Obtain written confirmation of eligibility, subscription dates, support scope and pricing.
The announcement does not establish that staying is cheaper. It creates another option to price against migration effort, compatibility testing and disruption. A move justified by capabilities or economics may still be the right decision.
Agent Substrate reaches GA, but still needs an operating team
Google announced Agent Substrate 1.0 and general availability on GKE on 5 November. It separates persistent agent state from execution workers, allowing idle agents to suspend and resume.
You install it onto GKE Standard. Autopilot is unsupported; control planes and node pools require GKE 1.37 or later, and the Extended release channel is unsupported. There is no additional feature charge, but underlying resources remain billable.
This is a platform-engineering decision, not simply an API substitution. Budget for runtime operation, workers, snapshot storage and monitoring.
For fleets with substantial waiting time, benchmark cost per completed task and tail resume latency against the existing deployment. Include warm workers and operational effort. Google’s density claims are vendor results, not demonstrated savings for your workload. Small fleets may not justify the additional machinery.
What deserves attention first
- For critical cloud-dependent services: service owners should test notification paths and business-level recovery criteria at the next resilience exercise.
- Before an EEA VMware commitment: procurement and infrastructure owners should obtain an entitlement-specific quote before approving renewal or migration.
- Before expanding Snowflake-routed AI traffic: platform and finance teams should test quota enforcement and agree on acceptable overshoot.
- Before commissioning Iceberg migration tooling: data leaders should revise implementation estimates while preserving validation and post-cutover recovery work.
- Before choosing an agent runtime: platform leaders should verify prerequisites and measure whether idle-resource savings exceed additional operating costs.
Sources
- AWS: Managed Hive-to-Iceberg migration announcement, 4 November 2026
- AWS: S3 Tables migration workflow and review controls
- Snowflake: Cortex AI Gateway GA release, 3 November 2026
- Snowflake: Gateway budgets, quotas and enforcement timing
- Bank of England, PRA and FCA: Google Cloud disruption response, 4 November 2026
- Broadcom: Portfolio update and EEA vSphere conditions, 4 November 2026
- Google Cloud: Agent Substrate 1.0 and GKE availability, 5 November 2026
- Google Cloud: Agent Substrate installation requirements and pricing basis
Continue the weekly briefing
Follow CloudHero Weekly
Cloud, AI and DevOps developments assessed against reliability, security, cost and delivery decisions.
Follow the weekly briefing