• Human Infrastructure
  • Posts
  • Human Infrastructure 398: Manhattan or Apollo for AI, Elephant Flows, Trippy Tools, and More

Human Infrastructure 398: Manhattan or Apollo for AI, Elephant Flows, Trippy Tools, and More

AuthorAuthor

Drew Conry-Murray & Ethan Banks
April 24, 2025

THIS WEEK’S MUST-READ BLOGS 🤓

Toni Pasanen continues his “AI for Network Engineers” series (he’s writing a book) by discussing the GPU-to-GPU elephant flow problem, why traditional ECMP is a bottleneck, and new load-balancing methods that address the problem.

“This chapter introduces two alternative load-balancing methods to traditional Flow-Based with Layer 3 ECMP: 1) Flowlet-Based Load Balancing with Adaptive Routing, and 2) Packet-Based Load Balancing with Packet Spraying. Both aim to improve traffic distribution in RoCEv2-based AI backend networks, where conventional flow-based routing often leads to congestion and underutilized links.”

You’ll get a great education reading this post, as Toni explains RDMA write operations with diagrams, and illustrates how data flows from GPU to GPU across the backend network. He then explains flowlet-based and packet-based load balancing, again with diagrams. - Ethan

Geoff dives into the question of “how big should a packet be?” As usual, Geoff reviews history, so we know why we have the generally accepted values of between 20 and 1500 octets. He also reviews practical aspects of the physical medium across which packets flow and their limitations.

For instance, Geoff points out that “The speed of light in glass or fiber-optic cable is … approximately 194,865 kilometers per second. The speed of voltage propagation in copper is 224,844 kilometers per second.” He uses math from there to explain how we ended up at 1,500 octets (a compromise value).

So what of jumbo frames? What of 802.11 Wi-Fi specs? And can’t we go bigger now that we’re rapidly pushing Ethernet speeds up to 1Tbps or higher? Geoff shares his highly informed perspective. - Ethan

Andrej explains “the Ansible way” (my words, not his), highlighting what it does well, but the constraints it then places upon the user.

“[Ansible] is not as pure of an IaC tool as its competitors. You don’t define the complete machine configuration in code and have agents enforce it, you have some code ensuring something is in a certain state when you run it, not more and not less. This is a less powerful, but also requires way less commitment, which is what makes it so appealing.”

He’s not an Ansible apologist—not every nail calls for an Ansible-shaped hammer. Instead, the piece highlights what Ansible does, and compares it with some other IaC approaches that might fit your needs better. - Ethan

Claudia de Luna wants to make you aware of a Python package called uv that aims to replace multiple tools and actions including pip, pyenv, initializing a git repository, and more. Why should you, a network engineer, care? If you’ve written some useful Python scripts and would like others on your team to be able to use them, there’s a lot of groundwork that has to happen. Claudia writes “If you have gone through the heavy lifting of getting your teammates to install the version of Python you need and then create the virtual environment and then install the necessary modules across whichever operating system they are using (and then deal with the one or two installations that just won't work) you know this can be challenging.” The uv package aims to make this much easier, and Claudia shows you how it works. - Drew

This post argues that the Apollo program is a better model for the US government to spur development of Artificial General Intelligence (AGI) than the Manhattan Project. Why? Because the Apollo program was “a public, civilian-led, whole-of-society initiative that developed dual-use technologies—such as advanced guidance and propulsion techniques—that benefited both civilian and military applications.” By comparison, the Manhattan Project was classified military project with the single aim of producing a weapon of mass destruction.

Generally speaking, if we’re going to have a taxpayer-funded effort that attempts to achieve AGI, I’d rather it be open, public, and aimed at improving human flourishing. Whether we can achieve AGI, and whether this machine mind will give a damn about what we want from it, remains to be seen.  - Drew 

MORE BLOGS

Unlock Faster AI-Powered Security Insights
Don't let massive network data slow down your threat detection.

NetFlow Logic’s NetFlow Optimizer supercharges your AI-native security by intelligently reducing and enriching raw NetFlow data into high-quality intelligence. This empowers your AI to identify subtle anomalies and predict threats more accurately, without data overload. Imagine: eliminating redundancy, aggregating flows, and reconstructing conversations for a complete view. Beyond reduction, we enrich data with application identification, geolocation, user details, threat intelligence, and VM names. This context-rich data fuels your AI, enabling it to learn normal network behavior and pinpoint malicious activity precisely.

Seamlessly integrate your optimized NetFlow data with your SIEM and monitoring systems, creating a central security hub. Detect and respond to sophisticated threats faster and more effectively.

With our NetFlow Optimizer, you're not just collecting data – you're unlocking the full potential of your AI-powered security, staying one step ahead of evolving threats.

TECH NEWS 📣

TL;DR. A breakup of GOOG really seems to be happening this time. We’ll see, but if it does, will it set a precedent (in tone if not in legal terms) to drive a breakup of Meta?

I’ve read enough about what happens behind the scenes at these huge tech companies that I’d like to see them broken up. Dystopian shards keep breaking through to reality in their wake. Society is not better for what some of these companies have become. I’m ready for something…other. - Ethan

Unstructured data vendor Hammerspace has been around for several years, but the exploding AI infrastructure build outs are driving recent growth.

“Hammerspace, as we’ve noted before, is named after the concept first coined from cartoons and comics, where characters pull objects they need out of thin air.

This is, in effect, what Hammerspace does. The startup provides a way of making large amounts of data, regardless of where it lives or how it is used, accessible and available to an organization just when they need it, and keeping it out of the way when they do not.

As [CEO and co-founder David] Flynn describes it, typically the way that enterprises would have worked with data would be to port it from wherever it is to where it needs to be processed. “You need to install stuff on every system,” he said. “It’s a mess.”

It’s also slow. “The AI arms race is such a sprint,” he said. With “time to value” now a key priority for these companies, Hammerspace is signing up a lot of customers that are anxious about idle time.“ - Ethan

TL;DR. For certain Synology NAS storage solutions, you now have to install Synology-branded drives for full functionality. Patrick Kennedy thinks this is just a cash grab by the Synology folks. He doesn’t see how they can remain competitive considering other NAS solutions that feature better hardware and place no such restrictions on drive choice. - Ethan

This is a great article and well worth reading, but if you don’t have time, here’s the TL;DR. First, end-to-end encryption doesn’t help if you, a government official, use a personal phone that’s vulnerable to spyware. Second, there’s a vast for-profit surveillance industry operating worldwide that can produce disturbingly specific and accurate “dossiers” on people, including government officials, by analyzing information collected by smartphone apps and combining it with additional information from data brokers. 

Both of these lessons are valuable not only for those concerned about national security, but also for us regular people who never consented to this degree of surveillance. - Drew

MORE NEWS

FOR THE LULZ 🤣

RESEARCH & RESOURCES 📒

Andrew’s made a great resource available for people prepping for the CCIE Service Provider track. Listen for Andrew on a future episode of the Heavy Networking podcast. He’s on the calendar to record! - Ethan

From the README. “Trippy combines the functionality of traceroute and ping and is designed to assist with the analysis of networking issues. Trippy runs on Linux, BSD, macOS, and Windows.” - Ethan

From the README. “Dublin Traceroute is a NAT-aware multipath traceroute tool. In a few words, you can run traceroutes in multi-path networks (i.e. with ECMP load-balancing enabled), recognize NATs, have nice diagrams like the one below, export to JSON, and do this with a command-line tool, a C++ library or a Python library.” - Ethan

From the README. “A Python-based tool for automatically generating visual diagrams of Azure virtual networks using topology data exported from the Azure API. This script creates .drawio diagram files representing Hub-and-Spoke network architectures, making it easier to audit, present, and understand complex Azure network infrastructures.” - Ethan

Holly’s posted her slide deck and YouTube from her KubeCon Europe talk about zombies. And what are zombies? “Zombies are servers which aren’t doing useful work. They’re everywhere, costing money, eating electricity, and belching carbon. And they’re useless!” - Ethan

Another great Humble Bundle! 15 ebooks for $25 or more - you set the price. Topics include infrastructure as code, Terraform, KCNA, CKAD & CKA certs, DevSecOps, Ansible, Python and OpenShift.

Top tip. Upload your PDF book collection to an LLM such as Google’s NotebookLM. Then you can ask your library questions and get back answers with the sources cited. - Ethan

MORE RESOURCES

  1. Kagi Assistant is now available to all users! - Tales from Kagi

  2. Regex Helper (test, learn, experiment with regular expressions)

  3. Phosphor Icons (flexible icon family for interfaces, diagrams, presentations)

  4. Grab the Free TCP Analysis Poster - Chappell University

  5. Prepper Disk Premium - 512GB of Survival Content - Prepper Disk

Get AI-Native Security that moves at the speed of your business

Don’t let network protection slow you or your business down.

Juniper’s new SRX4700 Firewall is designed to protect data in motion, bringing security and networking together in a single, streamlined platform.

That means you can mitigate risk without interruption. Making sure users aren’t just well protected, but enjoying the very best network experience.

And with the Mist AI Predictive Prevention Feature, it’s never been easier to keep the network safe from potential, initial, and subsequent attacks.

That means you, future cybersecurity hero, can detect and stamp out bad actors and sophisticated threats before they wreak havoc on the business.

With Juniper, you’re always one step ahead.

<Explore Our Solutions Now>

INDUSTRY BLOGS & VENDOR ANNOUNCEMENTS 💬 

Hank Preston shares his concerns about the “human-in-the-loop” model of AI automation, in which a human operator is simply there to review, approve, or modify AI output. To Hank, that sounds like a boring career. It also raises a problem: where do you get future network engineers who have the knowledge and experience to recognize when an AI gets something wrong?

Good question. The answer seems to be: AI. The difference is in how AI is used. Rather than a network engineer stamping “approved” on AI output, Hank hopes that network engineers will do the network engineering, and use AI as a tool or resource to support their own work. He writes “We should stay in the driver’s seat. We should stay in control of the networks and the network engineering. We should embrace the capabilities of AI to improve our network engineering work. AI shouldn’t be using us to improve its network engineering work—we should be using AI as a resource to become more effective network engineers—now and into the future.”

I agree with Hank that this seems like an optimal use of AI. But I’m not confident that the industry sees it this way. My gut feeling is that the tech leaders driving AI development, and the CEOs anxious to consume it, don’t see AI as a hammer or a saw to be put in the hands of an able craftsperson. They envision a factory, fully automated, that runs 24-7 with nary a craftsperson to be seen. - Drew

Forward Networks collects configuration and state data from network devices (routers, switches, load balancers, SD-WAN gateways, etc.) to build a digital twin of your networks: data centers, enterprise LANs, and enterprise WANs. Network engineers can use this digital twin to understand how traffic flows through the network, check to see if actual traffic flows align with policy (very useful in a security context), and provide a more accurate inventory of network devices than that spreadsheet you update now and then.

Forward Networks has announced the ability to add endpoint devices to its inventory of your network, including laptops and PCs, printers, and IoT devices. While this feature is helpful to get a sense of what’s connected to your network, at present Forward’s endpoint capabilities are limited. If you’re thinking of an NMAP scan that discovers and fingerprints devices on the network, this isn’t it. You can get the device’s IP address and MAC address, and that’s it. Endpoint devices also aren’t included in the digital twin. Forward Networks says it plans to enhance its endpoint capabilities over time, so stay tuned. - Drew 

NetBrain has announced new features and capabilities in its 12.1 release. From the press release: “NetBrain Next-Gen 12.1 combines no-code automation and AI into a single platform that automatically diagnoses and assesses your network for real-time observability and proactive remediation. It gives you a single source of truth and automation that empowers you to proactively identify risks, resolve issues, and ensure your network runs “outage-free”—every day.” - Drew 

Arelion (formerly Telia Carrier) has announced an expanded fiber connection among multiple PoPs in Mexico. From the release: “Arelion today announced a network expansion connecting its Point-of-Presence (PoP) at KIO QRO1 in Querétaro, Mexico, to KIO MEX 5 in Tultitlán and KIO MEX2 CDMX (Santa Fe) in Mexico City. This expansion delivers high-quality transmission capabilities through completely underground fiber construction, enhancing scalability, availability and resiliency. Arelion's route from Tultitlán to Querétaro offers the shortest, lowest-latency path between these markets, supporting customers' connectivity requirements for AI/ML, cloud and content applications.” - Drew  

MORE INDUSTRY NOISES

  1. Why do we need Q#? - Microsoft Quantum Blog

DYSTOPIA IRL 🐙

TOO MANY LINKS WOULD NEVER BE ENOUGH 🐳

LAST LAUGH 😆

Shared on the Packet Pushers Slack by Chris. You can join for free!