Building Products for Easier Telco Big Data Analysis and Access

Telecommunications is a big data business that often lacks scalable and proven solutions for data access and analysis. Evolving technologies and privacy concerns create tangible barriers for internet service providers to monitor their networks for malicious activity or network usage insights. While most network operators are aware that they are leaving a lot of data on the table, getting the right product for the job is not easy. It is, in large part, due to the way most data products deal with ambiguity and scale. It this blog post, I’ll share some insights about how we build products that make big data analysis and access a lot more streamlined for network service providers. To give you an idea of the process of building products with telcos in mind, we’ll focus on Explorer – the easy data access and integration solution that lets providers access device and usage data.

Big Data Can Help Telecoms Do Many Things

Precise telecom data can help analytics affect many business areas, from cost reduction and management optimization to data driven product development or service offerings. Here are just some of the areas a single big data solution like Explorer can impact.

In-house Solutions Save Telecom User Research Costs

Once a company knows precisely what devices their customers are using, they can forego some data acquisition from third parties. For example, if an internet service provider knows exactly how many devices an average household has, it has absolutely no reason to buy this information from market research companies or surveyors.

Precise Risk Assessments

As our customers unlock their network data, they are able to precisely evaluate any changes that might be coming down the line. A great example is the 2020 beta release of iOS 14, which threatened any systems that relied on MAC addresses for device identification. Since our customers knew exactly how many iOS devices they had on their network, it was easy to evaluate the impact on telecommunications services iOS 14 would have. These data points were combined with other information, such as iOS upgrade adoption rates over time to figure out the best course of action to mitigate any service risk.

Trends

Are more people buying 4k TV sets? What percentage of clients are using smart thermostats? Are there new bandwidth-intensive devices gaining popularity? Would a connectivity outage for maintenance be tolerable for your clients? Once a network service provider has access to a precise big data-driven device inventory, these questions have clear-cut answers.

Data Driven Product Offers

In addition to device usage trends, big data enables telcos to improve trend forecasts for additional product offers. If a cohort of clients suddenly buys smart toys or baby monitors, their internet service provider might want to offer them a digital parenting product down the line. In some cases, network service operators discover that their clients are not using the best devices for their network setup: inefficient, high-latency climate control devices are an example, which might give the operator a profitable niche to offer a better alternative.

The Changing Telecommunications Big Data Landscape

In the past, network service providers had little actionable information about their network usage, even though they had a lot of data coming in from various sources. Getting actionable insights from that data required a lot of complex work on the provider’s side, which was focused mostly on network management use cases. What changed in recent years is that the need for better device inventory and insights about customers has increased, as more decision makers in the industry become aware that the same raw data can be processed to provide much deeper insights. Hence, it’s not the data itself that is changing, but the way it is processed and what insights are in demand. To give you an example of how narrow telco data insights used to be with legacy solutions, let’s look at device inventories. In the past, providers would only use the MAC addresses to distinguish devices connected to their gateways (i.e., routers). This meant that telecoms could not know whether a new MAC address was really a new device or a random MAC address. They also could not figure out what types of devices were on the network: whether the client bought a device they had never used before, or simply upgraded an old one. This was not that big of a problem in times of the PC, before smart devices and IoT became widespread. As the consumer internet usage landscape changes, providers can no longer afford run blind. Their networks create an ecosystem for dozens of distinct online-capable device categories with varying security, bandwidth, as well as connectivity needs. They need solutions that give them awareness of threats, improve functionality and resource allocation. This is the context we need to be aware of before building products for the telecommunications industry. In Explorer’s case, we realized that network service providers already had enough data to create proper device inventories, but the real issue they faced was transforming a myriad of data points into actionable information.

Products that Transform Telco Data into Information

Building products that help telecommunications companies uncover insights from raw data is a challenge, and a key element you must prioritize is focus on end-user (e.g., telecom customer) security and privacy. Good telco data products have these issues solved at the core, to address the exceptional challenges facing the telecommunications industry in terms of customer protection. The 3 key tenets for data products in telecommunications and how CUJO AI addresses them with Explorer:

1. Privacy – no deep packet inspection (DPI) and full GDPR, CCPA compliance for data access and management from the outset.
2. Data quality – using advanced AI models that are beyond anything on the market in terms of accuracy and number of device configurations they can identify, benefiting heavily from the scale at which CUJO AI operates.
3. Scalability – proven 1 billion+ device monitoring scale on tens of millions of end-user networks without any noticeable service interference for end-users, improving efficacy and data quality with every new connection.

A good example of such a product is our most recent addition to CUJO AI Explorer – the MAC randomization detection feature, which is a patented solution for device categorization and identification in the telecommunications landscape. Many network operators still rely heavily on MAC address signatures to identify devices and place most other services or solutions on top of those identifiers. This unreliable identifier has now artificially increased device numbers by 20-24% because devices randomize their MAC addresses, skewing their data and disrupting other services. An AI-driven product that can reliably solve the randomization issue not only allows network operators to go back to their baseline, but it also improves their databases immensely, as a good device intelligence solution like Explorer has a lot more to offer than simple device signatures.

Read the full MAC randomization white paper to find out how we developed that product.

In the case of Explorer, it brings easily manageable data access to Tier 1 telecoms without any compromise on end-user privacy. I’m extremely happy about having over 1 billion devices protected by CUJO AI solutions, and I know that the way we build our products is essential to why customers choose us for device intelligence, AI network security, and Digital Life Protection.

The Scale of Telecom Big Data

What does it mean to work with a Tier 1 telecommunications company? Well, first you have to wrap your mind around the awe-inspiring scale that these giant networks maintain on a daily basis. For instance, CUJO AI monitored over 261 billion connection requests in July 2020 alone! We’ve identified and classified (most of the time, down to the OS version) over 760 million devices in 2020. Close to 300 million of these were iOS devices that started randomizing their MAC addresses in late 2020, after Apple rolled out iOS 14, iPadOS 14, and WatchOS 7 with MAC randomization turned on by default. Network service operators that depended on this single data point for device identification felt the impact. Without our solution, carriers would have had no way of knowing which of the millions of device signatures were duplicated. Explorer’s AI-driven device profiling models work with extreme precision, and over 84% of device models are identified in the first 5 minutes. Once the device is identified, changing its MAC address is no longer an issue for the network service operator. It’s challenges like these that really let you understand the scope of our solutions. Telecom big data analytics have an exciting chance to finally have a clearer picture of their networks, and it’s all a matter of uncovering existing data and analyzing metadata points. Accessing this data in a streamlined, easy to use and integrate fashion is also a key concern when developing and maintaining cutting-edge data solutions for telecommunications providers.

Telecom Data Analytics Need Precise Information

As we focused on building Explorer, we found that using machine learning algorithms would improve the product with every connection we monitored. (Remember the giant scale of connection requests we monitor every month?) We could feasibly find out what device types and capabilities each machine on the network had simply by monitoring its packet metadata. This means that we use zero invasive methods to identify and classify a device, while giving telecom data analytics greater insights into device types, usage and other attributes, such as device updating rates and risk measurements. Most of the time, network operators want to also provide this data to the end-users through native mobile applications. Thus, Explorer allows those people to manage their own home networks more efficiently. If you find this article interesting, be sure to read other articles on cujo.com and subscribe to our newsletter below. Follow us on LinkedIn to get the latest news about the products we build for network service operators.

Want to use Explorer?

Contact us!