You did everything right. But…
despite well-documented ITSM Security Policies, enforced user training, and expensive tools, the worst has happened. You have the best firewalls with really smart people writing the rules. You have a really grumpy auditor who lets NOTHING slip and is so irritating you absolutely… never mind! You have protection, you have password hygiene, SUPER is locked down, a state-of-the-art Active/Active disaster recovery and business resumption plan, and your users no longer believe Bill Gates or the Nigerian Prince want to give them money if they just “click here”. Oh, and you’ve managed to convince animal lover “Bob” not to click on cat videos, no matter what!

Then this happened

You got hacked. Some digital extortionist wants a sack of money to decrypt your data, or they threaten in 12 hours to post your sensitive data on a public site somewhere. You consider, “maybe I’ll just give up the money.” Sometimes that works. And sometimes they come back for more. THEY have you. And THEY are holding your corporate (and/or personal) information hostage. But your situation is special—it would take a very skilled and well-funded, sophisticated hacker, not some kid in a basement. But THEY are a nation-state hacker, or a narco-terrorist, or a broken individual fallen into the cult-like ideology of some unknown group bent on destruction and you are the target.

That’s Ransomware. And, by the way, it’s available on the “dark web” as-a-Service. Where is this dark web thing? Well, it’s just the regular web that you and your employees (and kids) use every day. Just like walking down the sidewalk in any big city (or small town) there are good places and bad places you can visit. Sometimes you don’t even know it’s a bad place until you find your phone missing. Or a couple of jerks treating you like they are playground bullies stealing your lunch money as they demand your purse or wallet.

What do you do now? Call the FBI? CrowdStrike? Google? You have 12 hours. Nobody is coming to save you. Superman is only in the comics. Depression and terror sink in.

Do you have backups? Are they good? Are they recent? Are they corrupted? When did your system become compromised? Is there a virus embedded lurking in your hardware? Is the firmware corrupted on your servers? Is there malicious code in the space for deleted files on your storage devices? Do these digital terrorists have an open backdoor into your system? Is the DR system compromised, too? The Class 1 data centers are connected by a turnpike of data—and the same applications run in each data center. Can I trust the DR? What will tomorrow’s headlines say about your situation? Can you hide it from your customers and the regulators? How long until you are out of business completely?

Unless you have planned in advance and employed a “certain set of skills”, you may be toast.

Who are these people? Who does this sort of thing? Why are they picking on you?

Russia is a potential attacker. Putin is desperate as his war and his health are crumbling. But not just Putin. Fanatics and fascists merge together in countries that hate the Western world. Whether it’s for money, extremist cause, or revenge for some ancient sin, the Nation-State is the most prevalent hacker. Iran, North Korea, and even al-Qaeda have demonstrated social media and technological skills.

Some hackers are just thugs. They’ve been criminals for a long time. An emerging class of hackers is the narco-terrorist cartels. They are world-class professional bad guys. They do extortion, human trafficking, and kidnapping for a living. Some experts see them entering the digital realm by sourcing ransomware from aaS providers. They may not be technically smart or sophisticated. But they are mean. Paying money or violence is nothing new. They want your password, and they know where you live.

There are those who believe they are doing the “Lord’s Work” by exposing data or government secrets. Maybe they see capitalism as the ultimate evil they want to punish. Ideologically, they don’t seem to mind throwing promising careers down the digital commode for some perceived revenge or harm to “the system” that offends them.

It is said that “everyone has a price.” What if a nefarious individual bribes one of your employees? Maybe the contractor working in the data center running cable. Or maybe a trusted sysadmin who has been presented with compromising photos or threatening pictures of their children. “Load this program on the server, we have your daughter.”

Recent Events

…57% of security leaders expect to be compromised within the next year. …remote work and the cloud have made it more difficult to spot a ransomware attack, as well as how deploying behavioral-anomaly-based detection can help mitigate ransomware risk. –Hacker News

The first quarter of 2022 saw phishing attacks hit a record high, topping one million for the first time, according to data from the Anti Phishing Working Group (APWG).

The financial sector was the worst hit, accounting for 24% of all detected attacks, although webmail and SaaS providers were also popular targets. –Infosecurity Magazine

Foxconn confirms ransomware attack disrupted production in Mexico—Foxconn release

DOJ seizes 3 web domains used to sell stolen data and DDoS services. The database consisted of seven billion indexed records featuring names, email addresses, usernames, phone numbers, and passwords for online accounts that could be accessed through different subscription tiers. —US DOJ

Cloud suppliers use undisclosed software, subject to attack. Users, who didn’t know about the middleware, are unable to take defensive measures … “add new potential attack surfaces that, because customers don’t know about them, can’t be defended against.”—The Register, Brandon Vigliarolo

Malware is spreading to IoT, PDFs, and other places it was never seen before.

Russian man pleads guilty in Nevada to plot to extort Tesla. “Ransomware hackers are trying to bribe your trusted employees.”—Velta, and ABC News

Ransomware gangs have become team players forming cartels and specializing in hunter/hacker roles. Attack vectors are RDP, Drive-By websites, and phishing. –Forbes

The U.S. government has elevated ransomware threats to a level of priority like that of terrorism. In July it announced a $10M bounty in exchange for any leads that could help authorities hunt down these criminals. …. The US faces seven ransomware attacks every hour while the average ransomware demand has risen over 500% in the first half of 2021 alone. –Forbes

Cybersecurity Professionals’ Number One Concern: Ransomware
Number two: Nation-State Attacks. “I see an escalation in state-sponsored or acts in connection with state-sponsored activity” –Ian Hi, BGL Insurance

“Verizon Business CEO Tami Erwin says every business should plan for a data breach of some sort…”—quoted by Maria Bartolomeo, Mornings with Maria host in a tweet

Besides doing all the right prevention, FBI.GOV says, “Create a continuity plan in case your business or organization is the victim of a ransomware attack.”

Below is a three-part relief, recovery, and continuity reference architecture for NonStop systems. Full implementation of the architecture would protect an enterprise from these discussed attack surfaces. HPE GreenLake is flexible enough to allow any enterprise to implement some or all of this reference architecture as a Service. The author thanks Randall Hardy, DFS/PULSE Network EVP for these two requirements:

1. Return to business immediately
2. Preserve the existing environment for forensic analysis

To accomplish these goals, we will look at the three System architecture domains, Hardware, Software, and Data/Relational Database.

We need a physical system that is absolutely trustworthy. It will need the OS and the application installed and patched to match the existing production environment. We will need to restore data without compromising the new system, without bringing along malware, and we need to assure relational database integrity.

Relief

We need a factory-fresh system that has no chance of being corrupted by being connected to our data center, network, applications, or access to any human actor. The system must either be shipped new from the factory or protected from cross-contamination. Requirement #1, “Return to Business Immediately,” would preclude a factory shipment and standing up a new system. The system must already exist and be readily accessible, but non-contaminated.

Part I: A Ready-To-Go Isolated Relief System. The Relief system must be available and configured, as a digital twin to the production systems, but completely isolated—an “air-gapped” system. Assuming an active/active implementation already exists, a third Relief system with identical hardware and software to the production system but isolated from any corporate network. Installation would be with media directly from the vendors, with minimal Internet connections.

Part II A separate and ongoing management team. This team would perform the installation and periodic required updates to keep the Relief system in sync with the production system. Preferably, the air-gapped system would be in a separate data center or colo from production or DR systems and not connected to the corporate network until needed. Updates and testing are performed throughout the calendar year as required to keep in sync with production.

Finally, Part III. An Immutable Database of Change Backup with Transaction Integrity to assure the relational database is up-to-date. Realtime updates could provide a door to hackers, so a lazy update where transactions are captured in a Database of Change and forwarded to the Relief system for lazy updates. These updates would be data only—captured with HPE Shadowbase from the transaction logs. Perhaps the transactions could be in a form like JSON or XML to prevent any executable code from creeping in. Note that there are recovery appliances that take digital snapshots of the storage sector-by-sector. This type of backup would violate ACID* database principles and could result in a damaged relational database. Digital data could also capture, and transport malware executables hidden in storage media or executable code. Instead, the Database of Change Transactions would be data only and could be stored encrypted or in a blockchain format giving an immutable history with which to periodically update the Relief system.
*ACID database: https://database.guide/what-is-acid-in-databases/

Recovery and Relief as-a-Service Implementation

With HPE’s GreenLake, this solution can be offered as a service. An air-gapped system can be installed in an HPE colo, administrated and operated by the HPE GreenLake Managed Services team, separate from the customer team, but to the customer specifications. Besides the standard hardware and OS monitoring performed by the NonStop GNSC, the GreenLake services team would monitor and update the application and any middleware keeping the entire software stack current with existing production systems. Provided as a service, the Relief system would be available at a moment’s notice.

This paper has presented architecture and an as-a-Service implementation overview. Your HPE NonStop Account Manager can assist you in hosting a Ransomware Recovery workshop. Within that workshop, HPE can assist you in evaluating gaps in your individual Ransomware Recovery plans against the reference architecture presented above. The product of the workshop would be a Ransomware Relief plan with an eye towards a GreenLake as-a-Service NonStop hardware and software implementation.