Why AI inference data centers should NOT share a site with your ERP

Your CFO approved a budget for an AI initiative. Your CIO bought GPUs to run inference. Your CTO installed the servers in the same data center where your ERP runs. Three months later you have problems: unpredictable latency in financial transactions, electrical noise on the network that affects inference, and disputes between teams over the cooling budget. The root cause: mixing workloads of completely different nature in the same physical infrastructure.

Why the workloads are incompatible

A transactional ERP system and an AI inference cluster have fundamentally different operating profiles. The difference matters because it affects electrical load, electrical noise, network profile, maintenance cycle and SLAs.

Concrete operational differences

Electrical consumption profile

A typical ERP server consumes a constant 200-400W. Its consumption is predictable, with no significant peaks. A GPU server for inference consumes a constant 1-3 kW but has peaks of 200-300% when a batch of queries arrives. Those peaks propagate to the UPS, the generator and the electrical distribution system. An ERP that shares a UPS with a GPU cluster sees those peaks as momentary voltage drops — the kind of thing that generates transaction errors without triggering traditional alarms.

Electrical noise and harmonics profile

High-density GPU power supplies generate current harmonics that degrade power quality. A cluster of 20-50 GPUs introduces total harmonic distortion (THD) of 8-15% on its circuit — double the acceptable standard. For SQL database engines and sensitive storage systems, that electrical noise translates into unpredictable I/O latency and checksum errors in critical operations.

Network profile

Modern inference uses east-west traffic between GPUs (tensor parallel, pipeline parallel). A single LLM query can generate 5-20 GB of traffic between cluster nodes. A conventional 10 GbE network saturates. The typical response is to add 25 or 100 GbE uplinks to the AI cluster — but that consumes core switch capacity that also serves the ERP. The result: both systems see increased latency during inference peaks.

The business case for separating

Beyond the technical problems, there are three commercial reasons that justify separate infrastructure:

1. Differentiated SLA. A corporate ERP needs 99.95% uptime with an RTO of minutes. An inference application typically tolerates 99% with an RTO of hours. Sharing infrastructure forces you to operate to the stricter standard, which raises the total cost.
2. Security and compliance. The data that feeds an inference model (internal documents, code, customer data) has a different security classification than a financial ERP. Mixing both in the same network segment complicates compliance with ISO 27001, SOC 2 or LFPDPPP.
3. Maintenance cycles. The AI cluster requires GPU driver updates every 2-4 weeks, more frequent network firmware maintenance, and maintenance windows coordinated with the ML team. The ERP is only patched once a month with pre-approved change windows. Sharing an operations team leads to calendar conflicts that delay everything.

How to separate without building a new DC

There are three levels of separation, from lowest to highest cost:

1. Logical separation. Separate VLAN segments, firewalls between both, traffic prioritization with QoS. Low cost, moderate benefit. It does not solve the physical problems of electrical noise.
2. Physical separation within the same data center. Dedicated rack rows, separate PDUs, dedicated UPS or derived circuit. Medium cost, solves the electrical noise problems and allows maintaining two operating budgets.
3. Separate site. Two data centers in different locations (it can be the same building on different floors, or a twin building 5 km away). High cost, maximum operational independence. Useful when the AI workload has a different SLA from the ERP.

How to diagnose whether your current DC has the problem

Three quick measurements that tell you whether you are already suffering from this problem:

1. Measure the harmonic distortion (THD) on the ERP circuit with a power quality analyzer (Fluke 435, Dranetz PowerVisa or equivalent). If THD exceeds 5%, you already have a problem. If it exceeds 8%, the problem is severe.
2. Compare the P99 latency of your SQL database before and after the GPU deployment. If the ERP’s P99 latency rose 20-50% after installing the AI cluster, the correlation is strong. The root cause is electrical noise and network contention.
3. Ask your finance team how many manual reconciliations they have done in the last 3 months. If the number went up, you are probably having transaction errors from undetected voltage drops.

When the cost of separation is lower than the cost of the problem

The typical threshold where physical separation is financially justified: when the monthly cost of the problems (operations team overtime, manual reconciliations, incident response time, deployment delay hours) exceeds the monthly amortized capex cost of separation. For a mid-sized ERP in Mexico, that threshold is usually between $80-150k MXN monthly. If your CFO is signing manual reconciliations every week, you are already above the threshold.

Practical recommendation

If you have fewer than 10 racks total or your budget is limited, logical separation with good instrumentation may be enough to start. If you exceed 20 racks and inference is a production workload, physical separation within the same data center with dedicated PDUs and UPS is worth the investment. Site separation (two separate data centers) is only justified if the AI workload is critical or has different compliance requirements. Do not mix loads unless you have instrumentation to measure latency and electrical noise on each circuit.

Sources

1. NVIDIA — homepage — https://www.nvidia.com/en-us/data-center/
2. Uptime Institute — Tier Classification framework (resilient infrastructure) — https://uptimeinstitute.com/tier-certification
3. TIA-942 — Telecommunications Infrastructure Standard for Data Centers — https://www.tiaonline.org/
4. IEEE 1100 — homepage — https://www.ieee.org/