Human Infrastructure 334: Do Routers, Switches Need AI Silicon?

Author

Drew Conry-Murray
December 14, 2023

Do Routers And Switches Need AI Silicon?

by Greg Ferro

Most infrastructure topics are boring except for AI. AI enables new features across a wide range of existing technologies as addons or upgrades. Growth-oriented companies see AI as a way to displace incumbents. Incumbent companies see AI as a chance to upsell existing products.

In a data networking context I’d say that there are three product verticals to consider in 2024. 

The first is cybersecurity. Security teams need better operations. The ability to detect/mitigate/respond becomes ever more complex. Analysis of encrypted data streams needs metadata from dozens of sources. 

The second is visibility and observability. The modern network has overlays, underlays, dynamic endpoints, fluid users, and connects more applications than ever. The network silos are converging with SDWAN / Campus / WAN / Branch / Wired / Wireless to reduce the operational load but, again, more complexity through diversity. 

We have spent the last decade working out data generation and collection so that principles of telemetry are  established (but perhaps not widespread). AI could take this data and turn it into improved visibility into performance, responsiveness and tracking of known events.

The third topic is on-device intelligence. The first wave to AI-enhanced products are services hosted externally “in the cloud” and delivered via SaaS. While Apple Mac computers and mobile devices have AI processors, generally most edge devices do not. I’m watching vendors announce AI hardware for servers and desktops so that inference operations can be performed at the user. Does this signal that AI processors will be embedded into routers, switches, and wireless devices? 

AI can be generalized into two parts - inference and models. Analyzing data and locating relationships / patterns / meaning are intensive processes. The result is a model. So Large Lanaguage Models like HuggingFace are generated but the resulting model can be downloaded to a personal computer. Models are often hosted in the cloud to enable charging and wrap other applications around it, but that's not the only way. 

I expect that AI models will be generated and sold by infrastructure vendors as part of existing products. There does not seem to be new products that AI creates in the infrastructure space. The consumption of AI at the edge will be done using inference on pre-generated models and this can be accelerated using ‘AI silicon’. General purpose CPUs are slow, hot, expensive and consume a lot of power. Comparatively, a small-ish AI ASIC is worthwhile when manufacturing can do them cheaply enough. 

Back to the question “Will AI Silicon be a part of routers and switches?” The answer is yes. The process for designing and producing AI silicon is common to most chip companies and manufacturing them is not a restricted or new process. 

What's not clear is what we can do with AI Silicon. Identify data flows at line rate? Buffer management? QOE/QOS improvements? Enable SASE as better security appliances run AI models at the branch edge? Application inspection in the branch/campus LAN?

I’m interested to hear what you think AI could do for you. Send me an email. But get ready for some confusion as the vendors roll out products that you may or may not actually need. As I’ve outlined, adding AI features is straightforward but are they worth the money to have them? 

THIS WEEK’S MUST-READ BLOGS 🤓

Istio Networking Part 2  https://jamali.hashnode.dev/istio-networking-part-2
Karim has a two-part series on Istio, a service mesh widely used in microservices architectures. What’s a service mesh? Essentially it serves as a control point for connecting and monitoring all the containers that make up a service or application. With a service mesh you can enforce access and security policies, monitor performance, help manage workloads, and more. If you’re a network engineer, service meshes are an area where you can apply your fundamental knowledge and bring value to the cloud-native realm. This two-part series provides a comprehensive overview for those new to service meshes and those who’d like a refresher. - Drew

The hunt for the hidden probe - The Goodwi.fi https://goodwi.fi/posts/2023/12/hunt-for-hidden-probe/
This blog is like a detective story that starts with a simple question: “Does a hidden SSID generate a probe response to a wildcard probe (“null”) request?” Much detecting ensues. - Drew

Bash One-Liners for LLMs - Justine LOL https://justine.lol/oneliners/
Open source installation of LLAMA LLM from Mozilla: “I spent the last month working with Mozilla to launch an open source project called llamafile which is the new best way to run an LLM on your own computer.” Once you have LLAMA installed the post shows five bash prompts to generate LLM responses for code completion, chatbots and email drafts. This might be your thing and thanks to Mozilla Foundation for doing it. - Greg

Questions about your cloud spend? DoiT is here to help.

You’ve got questions about your cloud costs, and DoiT’s technology makes it easy to get the answers. Ensure you’re spending what you should be – and stop overspend in its tracks.

From cost optimization to cloud migration, machine learning and CloudOps, DoiT is here to make the public cloud easy — without the costs.

Learn why DoiT is trusted by thousands of digital natives at DoiT.com.

TECH NEWS 📣

Quantum computers: what are they good for? - Nature.com https://www.nature.com/articles/d41586-023-01692-9
This piece going back to May 2023 is a good overview of the state of quantum computing. You get a good explanation of how quantum computers work, why anyone cares, and how far we are from quantum technology being able to deliver on all our hopes and dreams. - Ethan

Nokia and Windstream Wholesale achieve 800GE service readiness with 800Gb/s long-haul transport on single wavelength - Nokia https://www.nokia.com/about-us/news/releases/2023/12/13/nokia-and-windstream-wholesale-achieve-800ge-service-readiness-with-800gbs-long-haul-transport-on-single-wavelength/
There are many claims around “Internet traffic records” lately coming from various sources. This time around it’s Japan and Finland. First, the optical vendors and undersea cable deployments are rolling out new technology that lifts the link speeds to 800G and with 12/20/24/whatever fibres per cable you get >tens of terabits over the cable. Not really a surprise but always nice to know that more bandwidth is coming. Second, I’d say there is a marketing trend around claiming to be ‘biggest / fastest/ mostest’ on an undersea link. Consider this the last one I’ll write up and just assume that this happens every week or two from now on. - Greg

Ukraine's top mobile operator hit by biggest cyberattack of war - Reuters https://www.reuters.com/technology/cybersecurity/ukraines-biggest-mobile-operator-suffers-massive-hacker-attack-statement-2023-12-12/
Ukraine’s largest mobile operator was taken down by a cyberattack and took two days to recover. Russian actors claimed responsibility. The impact to Ukrainians includes missing air raid warnings and being unable to use public transport. This demonstrates that taking down critical national infrastructure networks can be done. We live in turbulent geopolitical times and you may need to consider this situation for personal and professional purposes. Modern lifestyles are fragile. - Greg 

Open source forkers stick an OpenBao in the oven - The Register https://www.theregister.com/2023/12/08/hashicorp_openbao_fork/
OpenBao is an open source fork of Hashicorp’s Vault project. Like OpenTofu before it, OpenBao was created in response to Hashicorp changing their licensing terms in a bid to { survive | be profitable | whatever }. OpenBao is under the increasingly large umbrella of the Linux Foundation. - Ethan

Cloud engineer gets 2 years for wiping ex-employer’s code repos - Bleeping Computer https://www.bleepingcomputer.com/news/security/cloud-engineer-gets-2-years-for-wiping-ex-employers-code-repos/
The engineer in question was fired for legit reasons when he violated company policy, then went absolutely nuts in retaliation. He was living out a revenge fantasy. He burned down the bridge while standing on it. The article details everything he did. - Ethan

FOR THE LULZ 🤣

RESEARCH & RESOURCES 📒

Awesome Network Automation - Network To Code via GitHub https://github.com/networktocode/awesome-network-automation
This repo is a collection of information about network automation from all over. Blogs, programming topics, open source projects, books, events, training, presentations, and a great deal more information is included here. Contributions are welcomed. - Ethan

Trippy https://trippy.cli.rs/
Trippy is a combination traceroute and ping tool that gives you lots of control over how the probe is sent. From the features list: “Customizable tracing options:

  • packet size & payload pattern

  • start and maximum time-to-live (TTL)

  • minimum and maximum round duration

  • round end grace period & maximum number of unknown hops

  • source & destination port (TCP & UDP)

  • source address and source interface

  • TOS (aka DSCP + ECN)”

Trippy is free. I installed it on macOS with homebrew and tested it immediately. The TUI is well done. - Ethan

OpenSpeedTest - The New Stack https://thenewstack.io/openspeedtest-check-the-speed-of-your-lan-via-web-browser/
OpenSpeedTest offers the ability to test the speed of a local network, and not just speed to an outside server on the Internet somewhere. “OpenSpeedTest is a free, open source HTML5 network performance estimation tool that doesn’t require any client-side software or plugin to function. Once deployed, the tool can be accessed from a standard, modern web browser. Even better, OpenSpeedTest can be deployed with Docker. It uses a combination of NGINX and Alpine Linux to use very little resources on your Docker server.” - Ethan

NetworkX - Network Analysis In Python https://networkx.org/
“NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.” NetworkX is for any sort of network, not just the type of network you think of when I say “router”. That said, there are use cases for infrastructure engineers and computer scientists. - Ethan

INDUSTRY BLOGS & VENDOR ANNOUNCEMENTS 💬 

Two New Marvell OCTEON 10 Processors Bring Server-Class Performance to Networking Devices - Marvell https://www.marvell.com/company/newsroom/two-new-marvell-octeon-10-processors-for-networking-devices.html
You probably haven’t heard about Marvell as an ASIC maker. They don’t do much to promote their brand to end users, but they are a major supplier. Their Octeon product line has been around for many years and you will find inside many networking appliances doing the heavy lifting for packet forwarding and inspection. The updated DPU ASIC has eight ARM cores and three times the performance using 50% less power. Vendors will run entirely onboard or use them as accelerators in an x86 system. - Greg

Zyxel Networks Announces Availability of 22Gbps WiFi 7 Access Point - Zyxel https://finance.yahoo.com/news/zyxel-networks-announces-availability-22gbps-130500618.html
Zyxel is the second company I’ve seen to release a pre-standard Wi-Fi 7 AP. Ratification of the standard is expected in May 2024, but if you can’t wait, Zyxel is ready for you. The company is positioning this AP for MSPs and small and medium-size businesses. From the press release: “The enterprise-grade WBE660S features a triple radio BE22000 architecture and utilizes a wider 320MHz channel to deliver speeds up to five times faster than WiFi 6/6E solutions. WBE660S enhances the user experience by providing seamless, latency-free connectivity to optimize high-bandwidth applications such as video streaming, broadcasting, online gaming, and VR/AR.” - Drew

Spoofing DNS Records by Abusing DHCP DNS Dynamic Updates - Akamai https://www.akamai.com/blog/security-research/spoofing-dns-by-abusing-dhcp
The core issue is that, by default, Microsoft DHCP servers will create a DNS record for any client that requests the DHCP server make a dynamic DNS update on its behalf. That means any DHCP client with the right flags set can create a DNS record, effectively using the DHCP server as a proxy. While not the easiest vulnerability to exploit, Akamai outlines a dead hostname spoof that results in a man-in-the-middle attack. I find this write-up interesting because of the large number of Microsoft DNS and DHCP servers out there. Many of you reading this might be vulnerable to this exploit. - Ethan

VMware by Broadcom Dramatically Simplifies Offer Lineup and Licensing Model - VMware https://news.vmware.com/company/vmware-by-broadcom-business-transformation
Broadcom announces the end of perpetual licenses. It’s realistic to assume that Broadcom wants to be able to increase prices at a time that fits their business model and to package/bundle products as suits their shareholder requirements. Less choice for customers will reduce negotiation time / costs and speed deal completion - not a bad thing. Check with VMware for details and book a meeting with accounting to explain why Capex spending for IT purchases is finished. - Greg

LAST LAUGH 😆