In this blog I'll share with you a step-by-step guideline on how to keep your servers protected from possible hardware hacks.
I will also summarize Bloomberg's post that started the conversation about hardware cybersecurity and explain how hacked hardware is a real threat with actual customer use cases.
Bloomberg's Unforgettable Post
I doubt that you haven't heard about it by now, but just in case:
Chinese spies manage to pull off the biggest hardware hack in history using tiny microchips to infiltrate almost 30 U.S. companies, including Amazon and Apple, and compromising America's supply chain. This report, published by Bloomberg Businessweek on October 4, 2018, sent a massive shockwave through the tech world.
Moments later, the news reach all corners of the earth.
Of those mentioned in the article, stocks simply plummeted, many others are up in arms about the possible implications, and interestingly enough, all during National Cybersecurity Awareness Month.
The Day After
24 hours after the initial shock took the internet by storm, we seem to have a house divided: the believers and the skeptics.
Let's start with the Skeptics
These are individuals who doubt every claim or dismiss the story altogether. Valid proof is the theme behind their arguments. After all, it's just impossible for U.S. companies of that size to purchase hacked hardware implanted with spy chips.
By Chinese spies?
...AND it goes unnoticed for multiple years?
I mean, really? Who'd believe any of that?
Let's meet the Believers
Labeled as conspiracy theorists, panic-inducing social media trolls, or simply put: liars - by every skeptic out there.
[I've had my fair share of insults for simply sharing the article on social media]
The theme here: Nothing is impossible.
Here's a snip from the article to think twice about the realm of possibility:
"The security of the global technology supply chain had been compromised, even if consumers and most companies didn’t know it yet."
"The companies’ denials are countered by six current and former senior national security officials, who—in conversations that began during the Obama administration and continued under the Trump administration—detailed the discovery of the chips and the government’s investigation."
The article lists numerous others - highly recommend you read it for yourself to form your own opinion.
Who's right and who's wrong?
I don't think that's the question we need to be asking ourselves.
No matter which side of the fence you're on, one thing is crystal clear - it IS possible and if there is ANY truth to this story - the problem of hacked hardware is a matter of national security.
Our personal data is at stake.
The government's sensitive files could be compromised.
We could all be at great risk if the information landed in the wrong hands.
The question we need to ask ourselves instead is...
How do we prevent The Big Hack from happening (again)?
In our world of rugged high-performance computing, hardware hacks are a real concern and a very real threat. For us and for our customers.
Let me answer that with a use case. [Client's names will not be listed for confidentiality purposes]
Customer A purchases numerous systems on an annual basis. These systems hold sensitive data that cannot be compromised under any circumstances. The client has implemented numerous processes to secure the data no matter where it is stored physically or what state it is in. These processes are intricate and involved, both from a logistics as well as a systems infrastructure standpoint. Numerous software applications are installed to protect the front-end from being attacked - but the hardware is still at risk. The customer's rugged computers are exposed to unauthorized parties and thus placing the physical servers/systems at risk of an intrusion - hardware side. To ensure their utmost security, they rely on us to design, manufacture, assemble, and integrate their newly built systems with anti-hardware hacking guidelines already in place. This protects them from any unwanted party gaining access to the system via a physical connection (think LAN ports, USBs, etc.) or foreign chip on the hardware (the case in Bloomberg's article).
How does this really help you?
Set expectations for your Manufacturer - on first contact
When you design, manufacture, assemble, and support the very hardware your customers depend on, it's vital - scratch that - it is a MUST to understand possible backdoors a hardware hacker would infiltrate to gain access to an otherwise protected system.
This is no arena for cutting corners or taking shortcuts.
Ensuring absolute security is P1.
End of story.
Your manufacturer needs to understand these threats and protect you from a possible hardware hack in the future.
How do you choose the right computer manufacturer?
Start with a US-made manufacturer.
I don't mean someone who is physically located in the US yet offshores all other aspects of the business.
When I say US-made, I mean the entire process is done state-side.
Here's what I mean:
Some manufacturers hide in the details.
They'll make the rugged chassis in the USA but integrate a 3rd party motherboard, which may come from other parts of the world that does not have strict rules & regulations into the manufacture/assembly process.
Others only have a Sales Team that is state-side, while all product/service related matters are offshore.
This can greatly impact your system security and put you and your customers at risk of a hardware hack.
The best way to filter your list of manufacturers is by asking the right questions up front.
What questions should you ask the manufacturer to determine if that's the right choice for you?
This is no small task. Keep in mind, no matter which manufacturer you choose, you are looking at a long-term relationship - few years at least.
These sorts of partnerships require you to think long-term strategy and total cost of ownership. A significant amount of your time, money, and resources will be spent - choose wisely and remember to weigh all of your options.
As promised, here are some questions to help you screen your list of manufacturers:
- Who designs your bare boards?
- [if not done by manufacturer] Where is this company located?
- Do you have an engineering team in-house?
- Do you have mechanical and electrical engineers that work on your products in-house?
- What steps do you take to ensure your products are not prone to hardware hacks?
- Can you disable the BMC if I needed you to? How would you do so? Are your engineers on this sort of task or do you outsource?
- Is your facility ITAR certified?
- Do you outsource any component-related work to 3rd party vendors?
- Who manufactures your products?
- Who integrates your products?
- Is your Sales Team state-side as well or do you have them all over the globe?
- Is your Support Team based in the USA or do you offshore these services?
- [If Support Team is US-based] Is your support team in the same building as your Engineers or do you have another company handle these requests?
Don't be alarmed if these questions eliminate a large number of manufacturers from your list. At the end of the day, you need to choose the security of your hardware and ultimately your computer systems above all else.
I am already in a long-term relationship with the incumbent, now what?
Chances are, you already have someone providing your high-performance computers. Possibly even, they've been your go-to for years.
Still, you can't shy away from asking them the same questions above; and let it not surprise you if they don't meet the qualifications either.
Breaking up is never easy. What should you do if you need to jump ship?
If you have to jump ship, then take some precautions and let the new manufacturer of choice absorb some of your pain points. This will ease the transition and provide smooth sailing henceforth.
Ask the new manufacturer about their integration capabilities. Describe your system in detail, give as much info as you can, and list your need-to and would-like-to haves - a top tier manufacturer will take both into consideration. After all, they should be delighted to showcase their products and services to you.
[first impressions last forever]
A lot of your worries should be thoroughly answered by their team. Not only Sales, but Engineering, Production, and all the other aspects that are of concern to you. Make sure to find out how their teams interact with each other, as that is usually a great indicator of how they will interact with you.
Send e-mails, start chats, give them a call - whatever it takes to measure their response rate as it is going to be vital during onboarding.
Final thoughts & summary
At the end of the day, your choices about which products you purchase and whom you choose to do business with is going to impact how prone your hardware is to hacks. If you don't do your homework, you are only hurting yourself.
The comfort of staying with the incumbent, no matter the practices in place, can cost you tremendously in the long-run. On the other hand, moving to a new manufacturer might be much easier than you think.
Asking the right questions at the right time will filter out those who hide behind every asterisk on their datasheets and web pages vs. those who stand by what they say.
Know what you want and what you need and don't be afraid to demand it - those willing to fight for your business with a passion to solve your problems before they arise are your true contenders.
Evaluate their products, services, and if you have the chance, meet in person.
If there's one thing I'd like you to take away from this post it's this:
Hacked hardware is real. It happens more often than you think and it can wreak havoc on your infrastructure if it is compromised. Don't ignore it. Take the necessary steps to ensure you're protected and have the right resources in place that fight the problem before it occurs.